CUDA

AutoRound is an open-source quantization toolkit for LLMs and VLMs, focused on high-accuracy low-bit inference across CPU, XPU, CUDA, and multiple deployment backends.

Open-source C++ and CUDA LLM inference engine inspired by vLLM, with a teaching-focused course that walks through model serving, batching, KV cache, and attention kernels.

Open-source repository for a NanoGPT training speedrun, focused on getting a 124M model to target loss on 8x H100 GPUs in under 90 seconds. The README includes run commands, Docker instructions, and a detailed list of model and systems optimizations.

A LangGraph-based workflow that generates, benchmarks, validates, and iterates CUDA kernels using a C++/CUDA harness. It targets workload-specific optimization with support for NVIDIA research lookup and optional Nsight Compute profiling.

A curated GitHub resource list for learning AI performance engineering, covering GPU fundamentals, kernel optimization, inference engines, and distributed inference.

A CUDA C++ project for training a tiny byte-level GPT quickly, with build, run, checkpoint, logging, and test instructions in the repo README.

Open-source research workbench for fail-closed LLM inference routing based on measured GPU workloads. The repo documents Phase 1 results, reproducible experiments, quick-start commands, and a policy that only promotes exact measured workload buckets.

A pull request adding AMD GPU support to tiny-vLLM through ROCm/HIP while keeping the existing CUDA build path unchanged. The snapshot describes the compatibility header, CMake option, architecture selection, and validation on multiple AMD GPUs.

Open-source AI infrastructure project focused on GPU memory management and serving, with a Python control plane, C++ data plane, and optional CUDA kernels.

A research-oriented optimizer project claiming to cut AdamW optimizer-state memory for LLM training by 50% through factorized variance and frequency-domain compression. The repo includes a Python POC, benchmark claims, and a stated drop-in usage pattern for PyTorch training.