llm

Uno is a research repository for speeding up LLM inference with discrete diffusion and lossless multi-token decoding. The repo includes inference, training, and evaluation code, plus installation steps, checkpoints, and example workflows.

Local privacy gateway that redacts sensitive text before sending prompts to external LLMs, then reconstructs the final answer locally.

Dependency-free Python library that compiles a declarative JSON query format into safe, parameterized SQL for LLM workflows, REST APIs, and frontend-backed data tables.

ContextSpy is an LLM context profiler that intercepts agent traffic, breaks down token usage, and visualizes how prompts change across requests and sessions.

An AI-powered Obsidian plugin that turns notes into a self-organizing wiki with auto-generated entity pages, bidirectional links, traceable answers, and local-first support.

A governance plugin for OpenClaw that registers instances with a Recursant registry and intercepts tool calls, LLM prompts, and chat messages for auth, PII redaction, rate limiting, and auditing.

Single-binary viewer for HTML reports generated by AI agents, with local browsing, live previews, search, and shareable links.

A hands-on workshop repository that walks developers through building a small GPT-style language model from scratch, including tokenization, transformer blocks, training, and text generation.

Browser-based RAG app that runs entirely client-side, using BM25/Lunr.js for retrieval and direct LLM provider APIs for answering questions without a backend.

A GitHub repository for a 7-node ESP32-S3 cluster that runs a small LLM using 1.58-bit BitNet-style quantization over an SPI daisy-chain.

A text-first, hands-on guide to agentic AI for practicing engineers, with runnable local-first code samples and a module-based structure covering LLM basics, tool calling, RAG, memory, multi-agent patterns, evaluation, safety, and enterprise topics.

A step-by-step tutorial that walks readers through making a first Anthropic Claude API call in Python, including environment setup, client creation, token usage, and printing the response.

Open-source LLM uncertainty and hallucination detection tool that estimates epistemic risk in microseconds using exact-match normalized entropy, with Rust gateway and Python package support.

vernLLM is a TypeScript LLM call framework that keeps retries, fallback, rate limiting, circuit breaking, caching, and middleware inside your own process. It supports OpenAI-compatible providers plus Anthropic, Gemini, Bedrock, and others via adapters.

Live demo of a prompt-injection defense for LLMs using per-position semantic overlays. The page shows how marked spans can be treated as untrusted while the base model stays frozen, with interactive examples for retrieved text, hidden instructions, language marking, and request-scoped overlays.

A terminal-based agentic coding harness for engineers, optimized for z.ai GLM models and built to hot-swap across multiple LLM providers mid-session.

Open-source agent harness that starts from a tiny loop, gives the model one exec tool, and stores the agent’s state and sessions inside its own directory. It includes setup commands, model configuration notes, and design rationale.

A small Rust-based LLM proxy for forwarding chat, messages, and responses requests to providers with gateway-key handling and optional protocol translation between Anthropic and OpenAI styles.

Open-source Python library that lets developers integrate an LLM into the Python runtime so missing attributes, methods, and typed values can be inferred on demand with confidence scores.

A Rust-based coding agent built for speed, low memory use, and Claude Code compatibility. The project highlights support for skills, hooks, MCP, plugins, sub-agents, and multiple LLM providers, with terminal, headless, and IDE-facing entry points.

A curated GitHub resource mapping AI token costs, waste, and efficiency practices for agentic coding. It combines an awesome list with editorial notes, practices, concepts, claims, and copy-paste setups.

NakshGuard is a self-hosted reverse proxy for OpenAI and Anthropic chat APIs that detects and blocks runaway AI agent loops before they waste tokens or API spend. It includes per-agent session tracking, configurable thresholds, shadow mode, and hot config reload.

Local-first governance layer for AI systems that monitors prompts and responses to block secrets, PII, and runaway LLM spending before requests leave the machine.

Python library for managing LLM conversation history with cache-friendly message ordering, dynamic context, and compaction.