transformer

An interactive browser city that visualizes how an LLM works, with live transformer mechanics such as tokenization, attention, KV cache, residual stream, and sampling.

A desktop and web app with 51 interactive labs that teaches how modern AI works through hands-on exercises, lessons in plain or technical language, and optional local or cloud AI guidance.

A CUDA C++ project for training a tiny byte-level GPT quickly, with build, run, checkpoint, logging, and test instructions in the repo README.

A PyTorch reference implementation of DeepSeek Sparse Attention from the LLMs-from-scratch project, including a lightweight indexer, top-K token selection, and a GPT-style model with KV-cache support.

A technical blog post showing a tiny diffusion transformer running on an RP2350 microcontroller to generate 128×128 face images, with details on model design, quantization, and on-device inference.

A research blog post describing an open-source small transformer trained from scratch to reach 44% on ARC-AGI-1 at very low cost, with architecture notes, ablations, and pointers to code.

A GitHub repository for a transformer language model implemented on Factorio 2.0 combinators, with a standalone Python version included.