openai-compatible

An OpenAI-compatible LLM proxy that routes requests to the fastest healthy provider. It runs as a small sidecar, supports latency/error-aware routing, session affinity, and Vertex AI, and can be deployed as a single static binary or Docker container.

A reverse proxy for OpenAI-compatible agent traffic that detects runaway tool loops and enforces hourly and daily spend caps with zero payload retention.

A Python SDK that provides LiteLLM-style model routing and translation with a smaller surface area. It supports completions, streaming, embeddings, tool calling, async variants, provider mapping, and local OpenAI-compatible backends.

Subpool is a self-hosted control plane for managing AI subscription and API capacity across teams. It pools authorized accounts, issues employee-specific keys, exposes OpenAI-compatible endpoints, and tracks usage without storing prompts or responses.

FAAAH is a dependency-free, OpenAI-compatible local proxy that routes requests through text files and AI coding agents. It is aimed at developers who want to reuse an existing agent subscription as a backend for tools that expect the OpenAI API.

A lightweight Flask-based LLM proxy that forwards Ollama, OpenAI, and llama.cpp-style requests to NVIDIA’s OpenAI-compatible API, with streaming support, retries, caching, auth, and usage stats.

Self-hosted LLM gateway and model router with OpenAI and Anthropic API compatibility, outbound provider routing, and eval-gated smart routing. It’s presented as a single static Go binary with zero telemetry and support for failover, caching, metrics, and local deployment.

AI agent that runs plain-text manual test cases in a real browser. It pastes step-by-step tests into Chrome, executes them live, and reports pass, fail, or warn per step with support for captcha and OTP pauses.

Brick is an AI model routing gateway that sends each query to the most suitable LLM based on capability and complexity. The repository includes quickstart steps, Claude Code integration, an OpenAI-compatible endpoint, and a short explanation of when to use it.

Cachet is a local semantic cache for LLM APIs that sits in front of OpenAI- or Anthropic-compatible endpoints and serves repeated or rephrased requests from cache. It includes a dashboard for hit rate and estimated savings, supports streaming responses, and is implemented as a single Rust binary.

AISBF is a self-hostable AI service broker and OpenAI-compatible proxy/router for routing requests across multiple providers, adding prompt-aware model selection, failover, quotas, analytics, and privacy-focused deployment options.

Lightport is a lightweight AI gateway that makes LLM providers OpenAI-compatible. The repo shows supported endpoints, a long provider list, quickstart commands, request examples, and active maintenance signals.

TokenDelivery.ai is an OpenAI-compatible API for open-weight LLMs with deterministic outputs, prepaid per-token billing, and a playground for testing prompts, images, video, and sampling settings.

Local privacy gateway that redacts sensitive text before sending prompts to external LLMs, then reconstructs the final answer locally.

Open-source resilience and debugging proxy for autonomous AI agents. It checkpoints LLM and tool calls, supports cache recovery, human-in-the-loop approval, multi-model fallback, and exports trajectories for fine-tuning.

A fail-closed execution firewall for AI agents that quarantines MCP tools, proxies OpenAI-compatible requests, and emits signed receipts for offline verification.

A Node.js proxy server that rotates API keys for LLM APIs like OpenAI and Gemini when rate limits are hit. It also includes an admin panel, request logging, provider management, and a Telegram bot for chatting with configured models.

vernLLM is a TypeScript LLM call framework that keeps retries, fallback, rate limiting, circuit breaking, caching, and middleware inside your own process. It supports OpenAI-compatible providers plus Anthropic, Gemini, Bedrock, and others via adapters.

Self-hosted, Kubernetes-native platform for distributed multi-model LLM inference. It includes an OpenAI-compatible API, installer flow, and infrastructure-as-code deployment with support for air-gapped environments.

Pacific Slate is a self-hosted, model-agnostic multi-agent AI assistant with background work, long-term memory, source tracing, and an interface built around cards and a demo monitor. The page also explains the architecture, agent roles, storage boundaries, and how it connects through MCP and an OpenAI-compatible endpo

A lightweight CLI for running multiple local MLX models on Apple silicon. It serves OpenAI-compatible endpoints, supports both text-only and multimodal models, and can browse/download from Hugging Face.

Bifrost is a high-performance AI gateway that provides a single OpenAI-compatible API across 23+ model providers, with automatic failover, load balancing, semantic caching, and enterprise deployment features.

OpenAI-compatible LLM API with a flat $6/month plan, unlimited tokens, and no prompt storage. It’s aimed at developers running coding agents, automation, and other high-volume workflows.

relaxAI is a UK sovereign LLM inference API with OpenAI-compatible endpoints and official SDKs for Python, TypeScript, and Go. The docs page covers installation, REST usage, and links into models, pricing, authentication, and endpoint sections.