CUDA

AutoRound is an open-source quantization toolkit for LLMs and VLMs, focused on high-accuracy low-bit inference across CPU, XPU, CUDA, and multiple deployment backends.

Open-source C++ and CUDA LLM inference engine inspired by vLLM, with a teaching-focused course that walks through model serving, batching, KV cache, and attention kernels.

A pull request adding AMD GPU support to tiny-vLLM through ROCm/HIP while keeping the existing CUDA build path unchanged. The snapshot describes the compatibility header, CMake option, architecture selection, and validation on multiple AMD GPUs.

Open-source AI infrastructure project focused on GPU memory management and serving, with a Python control plane, C++ data plane, and optional CUDA kernels.