observability

An open-source control plane for AI agents that adds policy enforcement, approval gates, durable execution, audit logs, and observability for unattended workflows.

Go-native framework for building, orchestrating, and observing AI agents with native OpenTelemetry, an embedded dashboard, replay, and first-party MCP/A2A support.

An OpenTelemetry Collector distribution that turns GenAI traces into bounded metrics and keyed summaries, designed for fleet-level observability without exporting raw prompts or high-cardinality labels.

A logical KV cache profiler and leak detector for PagedAttention engines like vLLM and SGLang, with a dashboard, live engine hook, CI checks, and Perfetto trace export.

Local-first debugger for LLM agent context windows. It records each call’s context into a single-file trace so you can inspect, diff, and share what the model actually saw turn by turn.

A local-first CLI that compresses application logs into a token-optimized Markdown/JSONL summary for LLMs.

Klent is an AI agent guardrails product that sits between agents and their tools, letting teams allow, deny, pause, rewrite, or redirect tool calls with policies, human approval flows, and audit trails.

A Claude Code mod that adds debugger-like controls for tool calls, including breakpoints, step/reject, timeline inspection, and failure diagnosis.

An unofficial Grafana plugin for Hermes Agent that records LLM calls and tool executions as generations and sends OTEL traces and metrics to Grafana Cloud.

A self-hosted Python runtime for building agentic AI apps with server-driven UI, audit logging, RBAC, sandboxing, and pluggable model orchestration.

A curated GitHub resource list for production AI agent infrastructure, covering runtimes, sandboxes, tool protocols, security, observability, and evaluation.

Local-first record, inspect, and replay tool for debugging AI agents. Orchid captures LLM calls and tool traffic through a proxy, exposes traces in a web UI and MCP tools, and supports offline replay for deterministic testing.

AISBF is a self-hostable AI service broker and OpenAI-compatible proxy/router for routing requests across multiple providers, adding prompt-aware model selection, failover, quotas, analytics, and privacy-focused deployment options.

Senro is a WebMCP evaluation and observability platform for testing AI-agent interactions with tools and monitoring production usage. The page highlights pre-production checks like smoke testing, goal and trajectory evaluation, plus runtime observability such as tool activity, errors, session timelines, and usage metri

Open-source Kubernetes-native runtime for LLM agents. CLRK runs agents in gVisor sandboxes and intercepts outbound traffic for observability, policy enforcement, credential injection, and routing control.

A plugin for recording AI coding agent activity in a tamper-evident local audit trail. It stores sessions, tool calls, prompts, and integrity checkpoints in SQLite and exposes CLI and UI views for status, reports, anomalies, spend, inventory, exports, and proof verification.

vernLLM is a TypeScript LLM call framework that keeps retries, fallback, rate limiting, circuit breaking, caching, and middleware inside your own process. It supports OpenAI-compatible providers plus Anthropic, Gemini, Bedrock, and others via adapters.

Selfship is an AI operations tool for agentic applications that detects recurring failures in live traffic, diagnoses the cause, and opens an eval-backed GitHub pull request for review and merge. It works with any LLM/framework and includes a demo, pricing, docs, and FAQ.

AgentSight is a zero-instrumentation observability tool for AI agents that uses eBPF to trace LLM API calls, token usage, and process behavior on Linux.

OpenSmith is a local-first, open-source alternative to LangSmith for tracing Python LLM pipelines. It supports decorators, context managers, console output, autopatching, CLI commands, local SQLite storage, and optional Postgres and OpenTelemetry integrations.

A curated GitHub resource mapping AI token costs, waste, and efficiency practices for agentic coding. It combines an awesome list with editorial notes, practices, concepts, claims, and copy-paste setups.

A single-binary OpenTelemetry trace generator that simulates realistic microservice and AI-agent traces, with chaos injection and multiple topology sizes.

Python toolkit for building LLM agents with MCP servers, skills folders, memory, streaming, and metrics support.

Open-source CLI that analyzes AUDR traces to find avoidable LLM calls, wasted tool usage, context growth, and other execution inefficiencies in agent runs.