Developer Tool / AI Data Preparation

doc-scraper

A Go-based crawler for scraping documentation sites into clean Markdown and JSONL for LLM ingestion, RAG pipelines, and offline searchable doc corpora.

Clear28/30
Useful26/30
Specific16/20
Complete16/20
doc-scraper screenshot

Why it was accepted

The page clearly describes an AI-adjacent developer tool with a concrete purpose: crawling technical documentation, cleaning it up, and exporting it for LLM use. The snapshot includes features, setup steps, CLI usage, output formats, and evidence of extras like MCP server mode and offline search, which is enough for a useful public listing.

Weakness

The snapshot doesn’t show a live demo, screenshots, benchmark results, or examples of the generated output beyond a small quick-start snippet. It also doesn’t make the licensing, release cadence, or current maintenance status easy to assess from the page alone.

Review status

49 days ago #685 ↑ +2

Last evaluated 49 days ago. Current rank #685. Up 2 spots in the rankings.

Score history

86

Related listings

bdinfo-rs screenshot
92

Developer Tool / Media Analysis

A Rust Blu-ray disc analyzer that reimplements BDInfo for BDMV folders and ISO images, with CLI, desktop, and browser/WASM front ends.

WebLLM screenshot
#12 WebLLM
92

Developer Tool / AI SDK / In-browser LLM inference

WebLLM is a high-performance in-browser LLM inference engine that runs locally in the browser with WebGPU acceleration. It exposes an OpenAI-compatible API, supports streaming and JSON mode, and includes examples for building chat apps and browser extensions.

simulator-use screenshot
90

Developer Tool / AI Agent Tooling

An open-source MCP and CLI tool that lets AI agents read and control the iOS Simulator, build and test Xcode apps, and handle common iOS interactions with a compact screen representation.

Magnitude screenshot
#49 Magnitude
90

Developer Tool / AI Inference Engine

Open source inference engine for agents that self-tunes on the user’s hardware to run open models faster, with desktop app support and one-click connections to popular agents.