LLM training

LoongForge is an open-source training framework for LLMs, VLMs, diffusion, and embodied models. It targets NVIDIA GPUs and Kunlun XPUs and emphasizes faster training with multi-backend support, model-specific optimizations, and ready-to-run configs.

A closed-loop controller for LLM training that adapts batch size, learning rate, checkpointing, and pruning to reduce OOM crashes on limited VRAM hardware.

A research-oriented optimizer project claiming to cut AdamW optimizer-state memory for LLM training by 50% through factorized variance and frequency-domain compression. The repo includes a Python POC, benchmark claims, and a stated drop-in usage pattern for PyTorch training.