Package
Scrape-LE
Analyse robots.txt content and report whether a path may be crawled.
io.github.nolindnaidoo/scrape-leSource repository
Overview
What this server provides
Analyse robots.txt content and report whether a path may be crawled.
Install
1Packages
scrape-le-mcp2.6.0
Runtime
npxTransport stdio