evals

Open-source AI agent monitoring platform with tracing, issue grouping, evals, semantic search, alerts, and MCP access for coding agents.

Greenflash is a product management layer for AI agents that reads production conversations, surfaces friction and intent gaps, and recommends what to fix or ship next. It includes SDKs, a REST API, Claude Code integration, and support for voice and text agents.

Armature is analytics and evals for MCP-based agent sessions. It captures how users interact with products through Claude, ChatGPT, Claude Code and other clients, then groups sessions into use cases, flags issues, replays traces, and turns real workflows into evals.

A job board focused on AI engineering roles, with first-party listings for RAG, agents, evals, inference, and MCP-related work. The page shows live filters, location and seniority breakdowns, and a nightly refreshed feed of open roles.

Selfship is an AI operations tool for agentic applications that detects recurring failures in live traffic, diagnoses the cause, and opens an eval-backed GitHub pull request for review and merge. It works with any LLM/framework and includes a demo, pricing, docs, and FAQ.

A repository of reusable AI agent skills centered on regression evals and benchmark history, with install instructions and local eval workflows.

A curated learning roadmap for building production AI agents, covering OpenAI, Anthropic, MCP, harnesses, evals, memory, and safety. The README organizes the material into phases with exercises and a prioritized reading table.