LLM evals

Free curriculum for learning AI agents, LLM evaluation, and benchmarking, with 42 lessons, interactive labs, runnable JavaScript, and saved progress.

A Python library for agent evals and guardrails that uses typed decision questions instead of an LLM judge. It supports one-request evaluation for traces, with examples for tool choice, grounding, relevancy, and injection checks.

A research page comparing 153 autonomous runs across 18 frontier models on the nanoGPT optimizer speedrun, with rankings, trajectories, token/compute stats, and equal-budget comparisons.