local-llm

A research prototype for a BitTorrent-style mesh where coding agents share idle capacity, delegate tasks, and settle credits across peers.

A drop-in proxy for Ollama that automatically tunes local LLM requests to reduce RAM use and speed up first-token latency. The page shows install steps, benchmark results, a live dashboard, and a local-only admin UI.

An interactive guide for choosing local AI hardware and matching open-weights LLMs to specific builds. It offers budget-based build recommendations, a hardware picker, model-to-hardware compatibility, and a state-of-the-local-AI snapshot.

PrivateRedact is a local-first PII redaction app that uses regex plus a local LLM to detect sensitive information in documents, then lets you review and export redacted files without sending data to the cloud.

Open-source job search agent that searches multiple job boards, scores listings against a user profile with a local LLM, and includes a terminal UI plus a web dashboard.

CrankGPT is a fully offline voice assistant built on a hand-crank-powered Raspberry Pi setup. The page explains the hardware stack, local speech recognition, on-device LLMs, text-to-speech, power smoothing, and latency measurements for running the system without cloud dependency.

Tokenstead helps people match AI models to their own hardware, with support for Macs, GPUs, and custom rigs. It shows which models can run locally, estimates speed, compares cloud pricing, and includes a source-cited tracker of organizations using different models.

Lightweight AI inference engine written in Rust for embedding local LLMs in mobile and desktop apps, with bindings for Python, Flutter, and React Native.

A web tool that estimates when a local LLM machine pays for itself versus using an API. It compares hardware options, model fit, speed, and break-even usage, with notes on assumptions and methodology.

A lightweight CLI for running multiple local MLX models on Apple silicon. It serves OpenAI-compatible endpoints, supports both text-only and multimodal models, and can browse/download from Hugging Face.

A Mojo port of the Pi AI Agent Toolkit with a set of runnable examples for chat, coding agents, tool calling, web research, and long-running agent loops.

Writekin is a Mac app that fine-tunes a local language model on your own writing and uses it to draft and rewrite text in your voice, with all processing kept on-device.

A GitHub repository guide to running state-of-the-art LLMs on local hardware, with concrete build notes, cost breakdowns, and runnable configs for large models.

A detailed walkthrough for running a local coding agent on macOS with llama.cpp, Gemma 4, MTP speculative decoding, image support, and Pi as the agent interface.

A local AI coding harness that turns business-rule specs into a constrained JSON AST, validates it, and compiles it into Clojure code while logging generation and repair artifacts.

A curated GitHub list and benchmark leaderboard for ultra-lightweight, zero-dependency AI engines in C/C++/Rust/Assembly, covering local LLMs, vector search, memory, GEMM, and agent tooling.

Enough is a beta personal language system for planning, writing, reviewing, and translation. It supports local models and OpenRouter, and is aimed at users who want more control over their data while building a flexible personal knowledge workflow.

Humbug is an open-source, GUI-based agentic development platform for human-AI collaboration. The page says it supports multi-model workflows, local and cloud LLMs, tool use, approvals for risky actions, and local-first data handling.

An open-source Python project that uses a local 7B diffusion language model to make real-time Gymnasium decisions with zero training. The README shows install steps, a code example, and benchmark-style results across several environments.

A blog post about an experimental local LLM system where three qwen3.5:9b agents run autonomously, maintain stress states, self-modify capabilities, and coordinate through a shared OS-like layer.