hugging-face

A pure C runtime for Gemma 4 E2B CPU inference, with export, benchmarking, and numerical validation tools included.

Shoehorn is a Rust library and CLI for fitting GGUF LLMs into available Mac VRAM by choosing a per-tensor mixed-precision quantization plan. It can probe usable GPU memory, account for KV cache and compute buffers, generate or use an imatrix, write a fitted GGUF, and launch llama.cpp inference.

unYOLO is a framework for building credential brokers and proxies for AI agents. It keeps provider credentials out of the agent, applies local policy rules, supports timed grants and operator approvals, and includes brokers for GitHub, Hugging Face, and approved Unix commands.

A 150M ModernBERT-based token classification model that extracts verbatim evidence spans for RAG and related pipelines, reducing the need for an LLM call to highlight answers.

A from-scratch ~1B parameter language model on Hugging Face with open weights, open data, training details, and loading guidance. The page is unusually candid about capabilities, limits, and evaluation.

Official implementation of LittleBit and LittleBit-2 for sub-1-bit LLM compression via latent factorization and quantization-aware training.

Open-weight System One model for machine-intuition style judgment tasks, built on a T5Gemma 2 encoder-decoder base. The page shows what the model does, how it is used in Python and via CLI/HTTP API, and includes benchmark and example sections.

A lightweight CLI for running multiple local MLX models on Apple silicon. It serves OpenAI-compatible endpoints, supports both text-only and multimodal models, and can browse/download from Hugging Face.

UltraCompress is a Python-based compression tool for large language models. The repo describes lossless 5-bit transformer compression, verification via SHA-256, a CLI on PyPI, and published model packs on Hugging Face.

A practical article explaining how to run AI models directly in the browser with Transformers.js, including setup, how it works, browser support, and example use cases like translation and grammar checking.

Browser-only local AI app for running compatible Hugging Face ONNX community models with WebGPU. The page shows a model library, local text generation, and an inference console in the browser.

A Hugging Face dataset about an experiment in agentic autonomy protocols, built from output generated by Qwen 3.8 27B running autonomously on a single RTX 3090 for three weeks.

A Hugging Face paper page describing a memory layer for LLMs that stores KV state to encrypted NVMe and reloads it later without recomputing, with results from 50M-token experiments on Gemma models.