testing

Excalibur is an open-source AI coding agent for product engineers. It runs from the terminal, supports multiple model providers, and covers the full workflow from discovery and planning through implementation, verification, review, and shipping.

An open-source CLI for testing LLM prompts across consistency, latency, cost, and behavior, with HTML reports and support for OpenAI and Anthropic.

Python package for recording real LLM API responses and replaying them in tests so LLM-driven code can run deterministically without live API calls.

CLI for testing and improving documentation with simulated developer agents. It checks whether docs are clear enough for agents to complete real tasks, reports where they get stuck, and can generate proposed edits.

An open-source tool that watches Playwright failures, inspects the live page, and opens a GitHub PR with a proposed fix.

Omnideck describes a self-testing browser-tool workflow where an AI agent runs daily checks on real websites, diagnoses failures against its own source code, and files GitHub issues with proposed fixes.

An open-source, self-hosted AI DevOps pipeline that ingests GitHub webhooks, generates code patches with an LLM, runs them in network-isolated Docker sandboxes, and posts validated fixes back as PR comments.

A Copilot skill suite that enforces structured coding workflows for AI-assisted development, with design, planning, TDD, debugging, verification, and cleanup steps plus a live task-graph visualization server.

An open-source enforcement layer for AI coding agents that adds frozen specs, tamper-detected tests, independent verification, and hard-blocking gates so unverified work cannot pass.

A tiny single-tasking DOS clone written from scratch in x86 assembly, built to boot and run period games in emulators. The repo includes a detailed README, build instructions, docs, and automated QEMU regression testing.

A source-available JavaScript browser agent for asserting end-to-end behavior in real user sessions. The repo explains the annotation-based approach, how it runs in staging and production, and how to install and initialize it via CDN or npm.

A GitHub repo for evaluating agentic AI pipeline systems, with guidance for defining metrics, building eval cases, running repeatable tests, and tracking regressions.

Pure Effect is a small JavaScript library for building testable business logic as data-driven commands instead of relying on mocks. It includes primitives like Success, Failure, Command, Ask, Retry, and Parallel, plus helpers like effectPipe and runEffect for composing and executing flows.