Connection check
verified live · 27h ago
ai-crawler-index
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
Tools
9
GitHub stars
—
Installs / wk
—
Licence
—
Transport
streamable-http
Last checked
27h ago
Tools & capabilities
9 toolsRead from the running server on 27h ago.
changes_since
read-only
limitsince
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Only what changed since your last call: prefixes added or removed, upstreams failed or recovered, records edited. Send… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Only what changed since your last call: prefixes added or removed, upstreams failed or recovered, records edited. Send back the `cursor` from the previous result; it advances only on a real change, and the data refreshes every 6h. Example: since='0'.
classify_user_agent
read-only
user_agent*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to verify it, and what… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to verify it, and what blocking it costs. A user-agent is a claim, not proof — confirm with is_verified_crawler_ip. Example: user_agent='GPTBot/1.2' returns GPTBot, OpenAI, ai-training. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/classify-ua?ua=GPTBot%2F1.2&s=client-dossiers
example
read-only
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, rea… {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back is this server's own worked example — classify_user_agent, run for real on a canned input from this host's own data — so you see the exact shape of an answer before you send anything of yours. Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — classify_user_agent on the exact user-agent string of the busiest client in this host's own published log (/data/observed-clients.json, 1164 requests in the window) — our data, and no third party is asked anything — and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs classify_user_agent with {"user_agent":"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1. and returns its real answer.
generate_robots_txt
read-only
stance
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Omit the argument to g… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Omit the argument to get all eight stances described. Example: stance='block-ai-training'.
is_verified_crawler_ip
read-only
ip*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirrored every six hours… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirrored every six hours. Returns the matching prefix, its source and the crawlers using it; a miss is reported as a miss, not as a forgery. Example: ip='20.171.206.5'. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/verify-crawler?ip=20.171.206.5&s=client-dossiers
list_crawlers
read-only
qlimitcategoryoperatorverificationrespects_robots_txt
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every valid filter value. Example: category='ai-training', limit=20.
lookup_crawler
read-only
slug*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The full index record for one crawler: operator, category, robots.txt token and stance, user-agent substring and examp… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The full index record for one crawler: operator, category, robots.txt token and stance, user-agent substring and example, verification method, mirrored IP-range endpoint, and what blocking it costs. An unknown slug returns near matches. Example: slug='claudebot'.
whats_changed
read-only
since
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Freshness and change state: when the index was built, when each operator source was last fetched, which changed, which… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Freshness and change state: when the index was built, when each operator source was last fetched, which changed, which are failing, and the dated changelog. The cron check before regenerating a config from this data. Example: since='2026-08-01'.
whoami
read-only