api-proxy

A drop-in proxy for Ollama that automatically tunes local LLM requests to reduce RAM use and speed up first-token latency. The page shows install steps, benchmark results, a live dashboard, and a local-only admin UI.

A C++ OpenAI-compatible LLM gateway designed for very low latency, with support for translating requests to providers like Anthropic, Gemini, and Cohere.

A local Rust API gateway that lets tools like Claude Code and OpenAI/Anthropic SDKs talk to OAuth-backed providers through OpenAI- and Anthropic-compatible endpoints.

Cachet is a local semantic cache for LLM APIs that sits in front of OpenAI- or Anthropic-compatible endpoints and serves repeated or rephrased requests from cache. It includes a dashboard for hit rate and estimated savings, supports streaming responses, and is implemented as a single Rust binary.

Lightport is a lightweight AI gateway that makes LLM providers OpenAI-compatible. The repo shows supported endpoints, a long provider list, quickstart commands, request examples, and active maintenance signals.

A Node.js proxy server that rotates API keys for LLM APIs like OpenAI and Gemini when rate limits are hit. It also includes an admin panel, request logging, provider management, and a Telegram bot for chatting with configured models.

Bifrost is a high-performance AI gateway that provides a single OpenAI-compatible API across 23+ model providers, with automatic failover, load balancing, semantic caching, and enterprise deployment features.

An OpenAI-compatible API proxy for using DeepSeek models with standard OpenAI SDKs, billed pay-as-you-go and accessible without phone verification.