language model

A from-scratch ~1B parameter language model on Hugging Face with open weights, open data, training details, and loading guidance. The page is unusually candid about capabilities, limits, and evaluation.

Live demo of a prompt-injection defense for LLMs using per-position semantic overlays. The page shows how marked spans can be treated as untrusted while the base model stays frozen, with interactive examples for retrieved text, hidden instructions, language marking, and request-scoped overlays.

A GitHub repository for a transformer language model implemented on Factorio 2.0 combinators, with a standalone Python version included.