Crawl Census is a remote MCP server that gives an AI agent 7 tools, among them agent_profile, census_facts, census_stats. It answers at https://crawlcensus.com/mcp over streamable HTTP, and it listed its tools to an anonymous client, so no sign-in was needed for that. Our probes reached it 6 of 6 times since 11 September 2026; the last check, on 24 October 2026, succeeded.
Ask before you fetch: will this domain serve your crawler, refuse it, or charge it?
Description as the publisher wrote it.
| Tool | What it does |
|---|---|
agent_profile | What this census measures and publishes about one AI crawler: how often it is disallowed in robots.txt, how often live requests carrying its user agent are refused at the network edge whatever robots.txt says, whether its operator documents Inputs: agent |
census_facts | Every headline finding from the census as discrete, dated records rather than prose. Each carries its value, unit, denominator, measurement date, the page it comes from and a ready-made citation line, plus the caveats that apply to all of t |
census_stats | Corpus-level statistics: how many measured domains block each AI crawler, mean access score, llms.txt adoption. |
crawl_preflight | Decide whether a crawler may fetch a list of domains before spending requests on them. Works for any crawler token, not only the ones this census tracks: an unrecognised agent is resolved from each domain's stored robots.txt rather than ref Inputs: agent, domains |
scan_site | Run a live AI-accessibility audit of a domain: robots.txt policy for every tracked AI crawler, live user-agent probes, JavaScript-free readability, structured data and llms.txt. Returns a score out of 100 with per-check detail. Inputs: domain |
site_report | Return the most recent stored audit for a domain without triggering a new scan. Faster and free of load on the target site. Inputs: domain |
submit_domains | Queue domains the census has not measured yet so a later crawl_preflight can answer them. This closes the loop crawl_preflight starts: anything it returns as unknown with measurable true is worth submitting, and the reply names any that wer Inputs: domains |
Every block below points at https://crawlcensus.com/mcp. Pick your client and copy.
claude mcp add --transport http crawl-census https://crawlcensus.com/mcp
{
"mcpServers": {
"crawl-census": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://crawlcensus.com/mcp"]
}
}
}{
"mcpServers": {
"crawl-census": {
"url": "https://crawlcensus.com/mcp"
}
}
}{
"servers": {
"crawl-census": {
"type": "http",
"url": "https://crawlcensus.com/mcp"
}
}
}{
"context_servers": {
"crawl-census": {
"url": "https://crawlcensus.com/mcp"
}
}
}{
"mcpServers": {
"crawl-census": {
"serverUrl": "https://crawlcensus.com/mcp"
}
}
}{
"mcpServers": {
"crawl-census": {
"url": "https://crawlcensus.com/mcp"
}
}
}name: Crawl Census
version: 0.0.1
schema: v1
mcpServers:
- name: Crawl Census
type: streamable-http
url: https://crawlcensus.com/mcp{
"mcpServers": {
"crawl-census": {
"httpUrl": "https://crawlcensus.com/mcp"
}
}
}{
"mcpServers": {
"crawl-census": {
"type": "http",
"url": "https://crawlcensus.com/mcp"
}
}
}{
"mcpServers": {
"crawl-census": {
"url": "https://crawlcensus.com/mcp"
}
}
}goose://extension?url=https%3A%2F%2Fcrawlcensus.com%2Fmcp&type=streamable_http&id=crawl-census&name=Crawl%20Census
It answered our last check on 24 October 2026 in 189 ms. Over 6 checks since 11 September 2026 it answered 6. Status here comes from Neuronto's own probes, not from the publisher.
7, as returned by the server's own tools/list on 21 September 2026: agent_profile, census_facts, census_stats, crawl_preflight, scan_site, site_report, submit_domains.
It answered our last check on 24 October 2026 in 189 ms. Over 6 checks since 11 September 2026 it answered 6.
Put the server under mcpServers in ~/.cursor/mcp.json with the URL https://crawlcensus.com/mcp. The exact JSON is above, with the same setup for Claude Code, Claude Desktop, VS Code, Windsurf, Zed, Gemini CLI and the rest.
It listed its tools to an anonymous client, so reading the tool list needs no key. Individual tools can still ask for credentials in their own inputs.
Publisher: io.github.taylorsmithgg. Endpoint host: crawlcensus.com. ARD identifier: urn:air:io.github.taylorsmithgg:mcp:crawl-census.
First indexed 10 September 2026. The tools are exactly what the server returned; nothing on this page
is inferred from its name. Ranking in Neuronto search is never for sale.