llm

Minimal LLM post-training experiments on a single 8GB GPU, covering SFT, DPO, and GRPO with a focus on forgetting, seed variance, and RL behavior.

A lightweight Flask-based LLM proxy that forwards Ollama, OpenAI, and llama.cpp-style requests to NVIDIA’s OpenAI-compatible API, with streaming support, retries, caching, auth, and usage stats.

CLI utility that counts LLM tokens in files or directories, with support for stdin, recursive scans, file-type grouping, profiles, JSON output, and multiple encodings.

Google’s announcement for new Gemini models focused on agentic workflows, coding, multimodal work, and cybersecurity use cases. The page includes performance claims, pricing, safety notes, and example applications.

A local-first AI reading assistant for Chrome that rewrites pages into customizable briefs, supports reusable reading recipes, and can run with local or hosted models.

A local Rust-plus-Python middleware layer for LLM apps that claims to reduce token usage and add in-process security checks before prompts are sent to a provider.

An open-source tool that watches Playwright failures, inspects the live page, and opens a GitHub PR with a proposed fix.

AI-powered security scanner for modern web apps that combines SAST, DAST, and LLM-assisted code review in a single command. The repository includes install steps, quick-start commands, supported stacks, and examples for scanning live URLs or source code.

Soup is a Python library that routes Agent Skills into LLM prompts, injecting only the skills relevant to each request. It supports local skills, remote repositories, and multiple providers including OpenAI, Anthropic, Gemini, Ollama, LiteLLM, and LangChain.

Local-first Python SDK for AI agents that compresses web pages before LLM use, flattens Shadow DOM, builds action maps, and extracts structured data from tables, forms, and lists.

An open-source, self-hosted AI DevOps pipeline that ingests GitHub webhooks, generates code patches with an LLM, runs them in network-isolated Docker sandboxes, and posts validated fixes back as PR comments.

Cachet is a local semantic cache for LLM APIs that sits in front of OpenAI- or Anthropic-compatible endpoints and serves repeated or rephrased requests from cache. It includes a dashboard for hit rate and estimated savings, supports streaming responses, and is implemented as a single Rust binary.

Open-source MCP server and CLI that lets an LLM control hardware workflows like compiling, flashing, and reading serial output for firmware development and device diagnostics.

Rudi is a Python tool for LLM memory management that replaces a growing chat transcript with a causal dependency graph, aiming to keep token usage flat across long sessions. The README shows benchmarking results, fold behavior, and example usage with get_slice, store_decisions, and run_turn.

Go packages for building agents in Go, with a streaming runner, tool dispatch, guardrails, compaction, provider adapters, and verified completion helpers.

Open-source MCP server that exposes SQL databases as tools and prompts for AI agents, with support for MSSQL and PostgreSQL, read-only defaults, and local or remote transport.

Swift package for building provider-agnostic AI agents on Apple platforms, with tools for runtime permissions, hooks, context compaction, session state, validation loops, and traceable execution.

TypeScript middleware that trims or compresses LLM chat history to fit a token budget, with support for OpenAI, Anthropic, and AI SDK integrations.

Open-sourced local multi-agent OS that runs three agents on consumer hardware, lets them form goals from environmental pressure, and supports self-modifying workflows through tool creation and change requests.

A research-style demo showing LLM agents evolving through a genetic algorithm to play Pokémon in parallel sandboxed VMs. The page explains the setup, scoring loop, generations, results, and links to the repo and run artifacts.

A ~1000-line C workflow runtime where an LLM can modify a DAG during execution using a restricted four-verb grammar, with every change written to an append-only audit log for replay and inspection.

Open-source Rust coding agent with LLM-native code understanding, shell safety, and support for multiple LLM providers with automatic failover.

Lightport is a lightweight AI gateway that makes LLM providers OpenAI-compatible. The repo shows supported endpoints, a long provider list, quickstart commands, request examples, and active maintenance signals.

A blind listening arena for AI-composed music. Users compare two pieces against the same brief, vote A/B/tie/neither, and then reveal results and rankings.