agent-evaluation

Mirrors is a staging and replay environment for AI agents. It rebuilds the systems an agent calls, replays real sessions against prompts, tools, and model changes, and surfaces regressions before deployment.

A local-first Python tool that audits Git history to quantify rework in AI-generated engineering work, including PR correction rates, churn clusters, superseded work, and line survival.

Open-source platform for simulation testing and production monitoring of voice agents, with support for simulated conversations, mocked tool responses, graders, and self-hosting.

An interactive site showcasing short AI-built arcade games and a model bench that compares coding models across the same tiny game constraints.

A prompt-injection capture-the-flag arena for testing how AI agents behave under attack. Users can play challenges, create guarded agents, add optional guards and sandboxed tools, and track scores on a leaderboard.

Senro is a WebMCP evaluation and observability platform for testing AI-agent interactions with tools and monitoring production usage. The page highlights pre-production checks like smoke testing, goal and trajectory evaluation, plus runtime observability such as tool activity, errors, session timelines, and usage metri

A research post from Prime Intellect comparing 153 autonomous runs across 18 frontier models on a nanoGPT optimizer speedrun. It presents the setup, harness, results, and discussion around autonomous AI research performance.

MineTrials is a benchmark-style writeup exploring how far AI agents can get in one hour of Minecraft, comparing models and harnesses and highlighting performance, cost, and a best run.

A DeepSeek Harness plugin that uses an LLM to verify and rank candidate outputs with select, compare, track, and rollout tools.

A blog post describing an evaluation harness for comparing agentic coding tools and prompts across realistic SWE tasks, with token-cost results and model-specific behavior notes.

A blog post from Archestra analyzing Jev on 100 real AI tool calls, with benchmark results, failure analysis, and comparisons against other models used for annotating agent traces.