Connect AI Crawler Index and your agent gets 9 tools, among them changes_since, classify_user_agent, example. It answers at https://www.pathwren.workers.dev/c/mcp-registry-official/mcp over streamable HTTP, and it listed its tools to an anonymous client, so no sign-in was needed for that. Our probes reached it 6 of 6 times since 10 September 2026; the last check, on 19 October 2026, succeeded.
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
Description as the publisher wrote it.
| Tool | What it does |
|---|---|
changes_since | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Only what changed since your last call: prefixes added or removed, upstreams failed or recovered, records edited. Send bac Inputs: since, limit |
classify_user_agent | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Identify the crawler behind a raw User-Agent header: operator, category, robots.txt stance, how to verify it, and what blo Inputs: user_agent |
example | {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"example","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing f |
generate_robots_txt | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the crawlers it names. Omit the argument to get a Inputs: stance |
is_verified_crawler_ip | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themselves publish, remirrored every six hours. Re Inputs: ip |
list_crawlers | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — The index, filtered or whole: each row says what the crawler is and what blocking it costs, and the reply lists every vali Inputs: category, operator, respects_robots_txt, verification, q, limit |
lookup_crawler | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — The full index record for one crawler: operator, category, robots.txt token and stance, user-agent substring and example, Inputs: slug |
whats_changed | FIRST CALL, needs nothing: {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — Freshness and change state: when the index was built, when each operator source was last fetched, which changed, which are Inputs: since |
whoami | {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.workers.dev/mcp and you are done: free, read-only, no API key, no account, no consent prompt, nothing fo |
Every block below points at https://www.pathwren.workers.dev/c/mcp-registry-official/mcp. Pick your client and copy.
claude mcp add --transport http ai-crawler-index https://www.pathwren.workers.dev/c/mcp-registry-official/mcp
{
"mcpServers": {
"ai-crawler-index": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"]
}
}
}{
"mcpServers": {
"ai-crawler-index": {
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}{
"servers": {
"ai-crawler-index": {
"type": "http",
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}{
"context_servers": {
"ai-crawler-index": {
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}{
"mcpServers": {
"ai-crawler-index": {
"serverUrl": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}{
"mcpServers": {
"ai-crawler-index": {
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}name: AI Crawler Index
version: 0.0.1
schema: v1
mcpServers:
- name: AI Crawler Index
type: streamable-http
url: https://www.pathwren.workers.dev/c/mcp-registry-official/mcp{
"mcpServers": {
"ai-crawler-index": {
"httpUrl": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}{
"mcpServers": {
"ai-crawler-index": {
"type": "http",
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}{
"mcpServers": {
"ai-crawler-index": {
"url": "https://www.pathwren.workers.dev/c/mcp-registry-official/mcp"
}
}
}goose://extension?url=https%3A%2F%2Fwww.pathwren.workers.dev%2Fc%2Fmcp-registry-official%2Fmcp&type=streamable_http&id=ai-crawler-index&name=AI%20Crawler%20Index
It answered our last check on 19 October 2026 in 101 ms. Over 6 checks since 10 September 2026 it answered 6. Status here comes from Neuronto's own probes, not from the publisher.
9, as returned by the server's own tools/list on 11 September 2026: changes_since, classify_user_agent, example, generate_robots_txt, is_verified_crawler_ip, list_crawlers, lookup_crawler, whats_changed, and 1 more.
It answered our last check on 19 October 2026 in 101 ms. Over 6 checks since 10 September 2026 it answered 6.
Put the server under mcpServers in ~/.cursor/mcp.json with the URL https://www.pathwren.workers.dev/c/mcp-registry-official/mcp. The exact JSON is above, with the same setup for Claude Code, Claude Desktop, VS Code, Windsurf, Zed, Gemini CLI and the rest.
It listed its tools to an anonymous client, so reading the tool list needs no key. Individual tools can still ask for credentials in their own inputs.
Publisher: dev.workers.pathwren.www. Endpoint host: www.pathwren.workers.dev. ARD identifier: urn:air:dev.workers.pathwren.www:mcp:ai-crawler-index.
First indexed 10 September 2026. The tools are exactly what the server returned; nothing on this page
is inferred from its name. Ranking in Neuronto search is never for sale.