Connection check
verified live · 27h ago
robots-policy-lint
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
Tools
7
GitHub stars
—
Installs / wk
—
Licence
—
Transport
streamable-http
Last checked
27h ago
Tools & capabilities
7 toolsRead from the running server on 27h ago.
audit_ai_access
read-only
pathrobots_txt*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Evaluate a pasted robots.txt against every AI crawler in this index and return the two lists that matter: blocked and… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Evaluate a pasted robots.txt against every AI crawler in this index and return the two lists that matter: blocked and allowed, per operator and category. Also names the tokens in your file that match no known crawler (a typo blocks nothing) and separates the crawlers that document obedience from the ones observed ignoring robots.txt, which need an IP or WAF rule instead. Example: path='/' with your robots_txt pasted in — the verdict is per crawler, at that path. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/ai-access?robots_txt=<urlencoded>&s=client-dossiers
check_path_allowed
read-only
pathsrobots_txt*user_agent*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Evaluate a pasted robots.txt for one crawler and one or more paths under RFC 9309: longest token match for the group,… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Evaluate a pasted robots.txt for one crawler and one or more paths under RFC 9309: longest token match for the group, longest path pattern for the rule, Allow breaking a tie, * and $ supported. Returns allowed/disallowed per path with the exact line that decided it, and flags the cases where a merge-groups parser and a first-group-wins parser would disagree. Example: user_agent='GPTBot', paths=['/', '/blog'], with your robots_txt pasted in. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/robots-allowed?robots_txt=<urlencoded>&ua=GPTBot&path=/blog&s=client-dossiers
diff_robots_txt
read-only
pathafter*before*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Compare two versions of a robots.txt and report only the crawlers whose verdict actually changes at a given path — not… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Compare two versions of a robots.txt and report only the crawlers whose verdict actually changes at a given path — not the text difference. Answers 'did my edit do what I meant, and did it do anything else', including sitemap additions and whether the parse errors went up or down. Example: before='User-agent: *\nAllow: /\n', after=your edited file, path='/'.
example
read-only
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots and you are done: fr… {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp/robots and you are done: free, read-only, no API key, no account, no consent prompt, nothing for you to invent, and what comes back is this server's own worked example — lint_robots_txt, run for real on a canned input from this host's own data — so you see the exact shape of an answer before you send anything of yours. Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — lint_robots_txt on three groups lifted verbatim from this host's own published /robots/block-ai-training.txt and /robots.txt — our data, not yours, and nothing is fetched — and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs lint_robots_txt with {"robots_txt":"# lifted from https://www.pathwren.workers.dev/robots/block-ai-training.txt and returns its real answer.
lint_robots_txt
read-only
robots_txt*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Parse a robots.txt you paste and report every fault that makes it do something other than what it looks like: misspell… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Parse a robots.txt you paste and report every fault that makes it do something other than what it looks like: misspelled directives, a full UA string where a product token belongs, rules before any User-agent line, duplicate groups, noindex (unsupported since 2019), relative Sitemap URLs, BOM. Each finding carries the line number and the fix. Example: robots_txt='User-agent: GPTBot\nDisallow: /\n\nUser-agent: *\nAllow: /\n' — paste the whole file, it is never fetched for you. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/robots-lint?robots_txt=<urlencoded>&s=client-dossiers — or POST the file as the raw body to the same URL.
merge_policy
read-only
stance*robots_txt*
FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Merge one of eight maintained robots.txt stances (block-ai-training, allow-ai-search-only, block-all-ai, block-dataset… FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Merge one of eight maintained robots.txt stances (block-ai-training, allow-ai-search-only, block-all-ai, block-datasets, block-disputed, block-seo-tools, allow-all, maximum-ai-visibility) into a robots.txt you already have, without touching a single rule you wrote: a token you already name keeps your rules and the stance's version is reported instead of applied. Example: stance='block-ai-training', robots_txt='User-agent: *\nAllow: /\n'.
whoami
read-only