hugging-face

A pure C runtime for Gemma 4 E2B CPU inference, with export, benchmarking, and numerical validation tools included.

Shoehorn is a Rust library and CLI for fitting GGUF LLMs into available Mac VRAM by choosing a per-tensor mixed-precision quantization plan. It can probe usable GPU memory, account for KV cache and compute buffers, generate or use an imatrix, write a fitted GGUF, and launch llama.cpp inference.

unYOLO is a framework for building credential brokers and proxies for AI agents. It keeps provider credentials out of the agent, applies local policy rules, supports timed grants and operator approvals, and includes brokers for GitHub, Hugging Face, and approved Unix commands.

A 150M ModernBERT-based token classification model that extracts verbatim evidence spans for RAG and related pipelines, reducing the need for an LLM call to highlight answers.

A from-scratch ~1B parameter language model on Hugging Face with open weights, open data, training details, and loading guidance. The page is unusually candid about capabilities, limits, and evaluation.

A lightweight CLI for running multiple local MLX models on Apple silicon. It serves OpenAI-compatible endpoints, supports both text-only and multimodal models, and can browse/download from Hugging Face.

UltraCompress is a Python-based compression tool for large language models. The repo describes lossless 5-bit transformer compression, verification via SHA-256, a CLI on PyPI, and published model packs on Hugging Face.

Browser-only local AI app for running compatible Hugging Face ONNX community models with WebGPU. The page shows a model library, local text generation, and an inference console in the browser.