apple-silicon

Shoehorn is a Rust library and CLI for fitting GGUF LLMs into available Mac VRAM by choosing a per-tensor mixed-precision quantization plan. It can probe usable GPU memory, account for KV cache and compute buffers, generate or use an imatrix, write a fitted GGUF, and launch llama.cpp inference.

Privacy-first AI meeting notes and writing assistant for macOS, with on-device transcription, smart minutes, action items, and AI writing tools. The repo also shows an opt-in plain Markdown storage mode and full source for the app.

A terminal coding agent for Mac that runs on local models through MLX, edits code, runs tests, and keeps everything on-device.

Native Zig inference server for Apple Silicon Macs that runs local LLMs with MLX and GGUF support, plus OpenAI- and Anthropic-compatible APIs. The page highlights one-click local setup, Ollama compatibility, agent tools, MCP support, and performance claims against LM Studio.

A macOS command-line tool for on-device AI and voice tasks, including speak, listen, transcribe, chat, and an OpenAI-compatible local server.

LocalCode turns plain English into CLI commands on Apple Silicon Macs using Apple’s on-device Foundation Models framework, then asks for approval before running them.

Experimental page-aware Metal runtime and reproducibility harness for running oversized sparse mixture-of-experts models on Apple Silicon. The repo documents GPT-OSS 120B support, performance results on 64 GiB and 16 GiB machines, and the runtime/harness structure.

A lightweight CLI for running multiple local MLX models on Apple silicon. It serves OpenAI-compatible endpoints, supports both text-only and multimodal models, and can browse/download from Hugging Face.

An Apple Silicon–optimized inference build of Bonsai 1.7B with custom Metal kernels, benchmark results, quick-start instructions, and a bundled OpenAI-compatible server.