deterministic inference

TokenDelivery.ai is an OpenAI-compatible API for open-weight LLMs with deterministic outputs, prepaid per-token billing, and a playground for testing prompts, images, video, and sampling settings.

CTGT’s research post shows how deterministic LLM serving can still produce inconsistent pipeline outcomes, and introduces Concordance, an Apache 2.0 instrument for tracing where AI workflows diverge. The page includes the problem setup, benchmark details, findings, limitations, and a linked repo for reproduction.