quantization

AutoRound is an open-source quantization toolkit for LLMs and VLMs, focused on high-accuracy low-bit inference across CPU, XPU, CUDA, and multiple deployment backends.

UltraCompress is a Python-based compression tool for large language models. The repo describes lossless 5-bit transformer compression, verification via SHA-256, a CLI on PyPI, and published model packs on Hugging Face.

An open-source Llama2 inference engine written in x86 real-mode assembly that fits in 1277 bytes and can boot directly from disk before any OS loads.