mcp-server-webcrawl
Advanced search and retrieval for web crawler data. With mcp-server-webcrawl, your AI client filters and analyzes web content under your direction or autonomously. The server includes a full-text search interface with boolean support, and resource filtering by type, HTTP status, and more.
mcp-server-webcrawl provides the LLM a complete menu with which to search your web content, and works with a variety of web crawlers:
Crawler/Format |
Description |
Setup Guide |
|---|---|---|
Standard web archive format |
||
CLI website mirroring tool |
||
GUI crawler and analyzer |
||
CLI security-focused crawler |
||
GUI crawler and analyzer |
mcp-server-webcrawl is free and open source, and requires Claude Desktop, Python (>=3.10). It is installed on the command line, via pip install:
pip install mcp-server-webcrawl