AI Hardware / On-device LLM demo
A live demo and write-up showing a 3.16M-parameter INT4 transformer running entirely in on-chip memory on a Xilinx Kria KV260 FPGA, with measured token rates and hardware details.
The page clearly presents a real AI-powered demo, not just a discussion post: it shows a live chat widget, measured throughput, and explains how the model runs fully on-chip without DRAM in the token loop. The snapshot gives enough evidence for a useful listing because it includes the hardware platform, model size, quantization, performance numbers, and the core technical idea behind the project.
The demo is narrow in scope: it behaves as a story generator rather than a general assistant, and the crawl snapshot does not show installation steps, source code, or a reproducible setup. It is also not clear from the snapshot how to access the full demo beyond the embedded widget.
Last evaluated 21 days ago. Current rank #771. Down 6 spots in the rankings.
CrankGPT is a fully offline voice assistant built on a hand-crank-powered Raspberry Pi setup. The page explains the hardware stack, local speech recognition, on-device LLMs, text-to-speech, power smoothing, and latency measurements for running the system without cloud dependency.
A blog post documenting an open-source, under-$40 take on OpenAI’s Codex Micro, built as an AI-assisted PCB routing and keyboard/macropad project.
