Developer Tools / AI Inference / LLM Serving
Open-source C++ and CUDA LLM inference engine inspired by vLLM, with a teaching-focused course that walks through model serving, batching, KV cache, and attention kernels.
The page clearly describes an AI infrastructure project: a high-performance LLM inference engine in C++ and CUDA. The README gives concrete implementation details, mentions Safetensors and Llama 3.2 1B Instruct support, and lists engine features like KV cache, static and continuous batching, online softmax, and PagedAttention. It is useful for AI builders and readers who want both code and an educational walkthrough.
The snapshot does not show installation prerequisites, runtime examples, benchmark results, API usage, or maintenance signals beyond the file list and commit count. A visitor still cannot tell how to run the server or what interfaces it exposes.
Last evaluated 3 days ago. Current rank #44. Down 3 spots in the rankings.
CodeGraph is a local code knowledge graph for AI coding agents like Claude Code, Cursor, Codex, OpenCode, and Hermes Agent. It aims to cut token use, tool calls, and runtime by letting agents query pre-indexed code structure instead of scanning files repeatedly.
Claude Code plugin that blocks dependency edits until a fresh, source-cited version check is recorded, helping prevent hallucinated or stale package versions across npm, pip, Poetry/uv, Cargo, and NuGet.
An OWASP incubator project that protects AI agent memory from prompt injection, secret leakage, and tampering. It includes a Python library, policy-based controls, benchmarks, and integrations for agent frameworks like LangChain and AutoGen.
CLI for catching AI-generated code smells and regressions in code. It scans changes with 40+ rules across 7 languages, offers fixes, CI gating, hooks, and MCP tools.
