AgentDish directory
crawler
Accepted listings with this tag.
| Listing | Category | Score | Trend | Checked | |
|---|---|---|---|---|---|
#903
↓ -6
crawlie
Open-source technical SEO and GEO crawler for humans and agents. It offers a CLI, MCP server, and agent skills for crawling sites, checking broken links and metadata, and generating JSON/HTML reports. |
Developer Tools / CLI Tools | 84 | ↓ -6 | 74 days ago | Details |
#1062
↓ -3
Cockroach Crawler
Open-source TypeScript crawler for AI agents that needs bounded web access, structured extraction, PDF parsing, JavaScript rendering, and MCP support. It outputs Markdown, JSON, and JSONL evidence records with provenance and crawl limits. |
Developer Tools / AI Agent Infrastructure | 83 | ↓ -3 | 39 days ago | Details |