performance

A web calculator for estimating whether local LLMs fit on specific hardware and how fast they may run. It covers VRAM, token throughput, latency, power, and cost across multiple GPUs and devices.

A drop-in proxy for Ollama that automatically tunes local LLM requests to reduce RAM use and speed up first-token latency. The page shows install steps, benchmark results, a live dashboard, and a local-only admin UI.

LiteLLM is moving its AI gateway to Rust and claims major gains in throughput, memory use, and request overhead while keeping the same config, API, and provider coverage.