Index / Agent Readiness Index

Agent Readiness Index

Can an AI agent find out what the most visited websites offer, without scraping their pages? We check the Tranco top 10,000 (list V354N) every month for every file and record built for that purpose. Of the 6,935 domains that answered as a website, 74 publish a machine-readable agent file. Measured 2026-11-02 19:04:56 UTC.

74 of 6,935
publish a machine-readable agent file: an ARD manifest, an MCP or A2A card, or an API catalog
727
serve an llms.txt (10.5%)
29
publish an ARD manifest at either well-known path
1,095
block at least one named AI crawler in robots.txt (15.8%)

By signal and by rank

SignalTop 100
48 sites
Top 1,000
598 sites
Top 10,000
6,935 sites
llms.txt
/llms.txt, a Markdown summary of the site written for language models
6 (12.5%)87 (14.5%)727 (10.5%)
MCP server card
/.well-known/mcp.json or /.well-known/mcp/server-card.json
1 (2.1%)7 (1.2%)50 (0.7%)
A2A agent card
/.well-known/agent-card.json or /.well-known/agent.json
1 (2.1%)2 (0.3%)11 (0.2%)
ARD manifest
/.well-known/ard.json, the Agentic Resource Discovery manifest
0 (0.0%)2 (0.3%)11 (0.2%)
ARD manifest, older path only
/.well-known/ai-catalog.json, the name used before ARD v0.91
1 (2.1%)2 (0.3%)18 (0.3%)
API catalog (RFC 9727)
/.well-known/api-catalog, a linkset of the site's APIs
0 (0.0%)9 (1.5%)53 (0.8%)
Link: rel="ard"
a header on the root pointing at the ARD manifest
0 (0.0%)1 (0.2%)3 (0.0%)
SVCB _index._agents
a DNS record naming where the ARD manifest is served
0 (0.0%)0 (0.0%)11 (0.2%)
Any agent file
ARD manifest, MCP card, A2A card or API catalog
2 (4.2%)11 (1.8%)74 (1.1%)

Shares are of the sites that answered over HTTPS. Of the 10,000 ranked domains, 2,090 answered no web request at all (most are CDN, DNS and telemetry hosts with no website) and 975 redirect to a different domain, so they are left out of every denominator.

A 200 is not a file

596 of the 6,935 sites answer /.well-known/ard.json with 200 and an HTML page: their homepage or an app shell, served for any path. A scanner that counts status codes reports every one of them as an ARD publisher. This index counts a file only when its content has the right shape, which is why its numbers are small.

Who publishes what

Every site in the population with each signal, with its Tranco rank.

llms.txt: 727 sites

cloudflare.com #2, github.com #34, adobe.com #62, wordpress.org #66, digicert.com #73, opera.com #77, sentry.io #102, intuit.com #115, samsung.com #122, dropbox.com #134, webex.com #144, gravatar.com #167, ubuntu.com #168, shopify.com #171, kaspersky.com #172, stripe.com #206, hcaptcha.com #228, weather.com #238, roku.com #239, amplitude.com #246, sourceforge.net #252, hubspot.com #260, linktr.ee #264, datadoghq.com #282, slack.com #304, etsy.com #334, unisys.com #337, bilibili.com #348, ea.com #350, dell.com #361, cm.com #366, paloaltonetworks.com #367, okta.com #394, name.com #399, wps.com #405, dynatrace.com #415, calendly.com #416, dailymotion.com #445, bluehost.com #470, taboola.com #475, businessinsider.com #477, shein.com #483, trendmicro.com #496, foxnews.com #507, braze.com #513, capcut.com #542, klaviyo.com #554, cloudns.net #565, dreamhost.com #579, newrelic.com #586, squarespace.com #594, atlassian.com #602, target.com #610, wix.com #613, sophos.com #637, onetrust.com #648, stackadapt.com #663, optimizely.com #666, bitrix24.ru #672, redhat.com #680, ring.com #682, surveymonkey.com #690, fontawesome.com #731, gumgum.com #741, asus.com #746, mailchimp.com #777, zoho.com #794, moloco.com #802, prnewswire.com #810, fiverr.com #818, allegro.pl #819, agora.io #841, auth0.com #849, frontiersin.org #870, unpkg.com #873, pendo.io #887, cloudinary.com #898, porkbun.com #909, lg.com #916, otto.de #925, seedtag.com #928, classlink.com #929, tailscale.com #930, repubblica.it #941, instructure.com #952, trendyol.com #985, flightradar24.com #1,000, xing.com #1,010, kwai.com #1,018, moe.video #1,060, netgear.com #1,063, typeform.com #1,135, kslawin.com #1,165, jimdo.com #1,188, siemens.com #1,205, coursera.org #1,213, one.com #1,214, intercom.com #1,215, home.pl #1,222, marktplaats.nl #1,226, savefrom.net #1,260, rajasthan.gov.in #1,282, polybuzz.ai #1,291, oxylabs.io #1,336, haberler.com #1,341, bookmyshow.com #1,378, sportybet.com #1,388, notion.com #1,411, protothema.gr #1,414, ilmeteo.it #1,425, coinbase.com #1,426, sozcu.com.tr #1,446, loopia.se #1,454, sondakika.com #1,496, revenuecat.com #1,504, programme-tv.net #1,528, markt.de #1,536, musicdel.ir #1,542, rakuten-sec.co.jp #1,549, alura.com.br #1,564, therapservices.net #1,586, chordtela.com #1,607, nustargame.com #1,621, adroll.com #1,636, eff.org #1,712, sambad.com #1,776, sattamatkano1.me #1,788, rapid7.com #1,791, bigcommerce.com #1,803, xfinity.com #1,813, dns-shop.ru #1,828, apkpure.com #1,830, pewresearch.org #1,834, hostinger.com #1,837, bitdefender.com #1,843, screenconnect.com #1,848, utorrent.com #1,853, wpastra.com #1,879, clevelandclinic.org #1,916, meetup.com #1,923, oreilly.com #1,939, surfshark.com #1,950, umeng.com #1,983, squareup.com #1,987, mongodb.com #1,997, trip.com #1,998, poste.it #1,999, sendbird.com #2,004, amocrm.ru #2,021, logitech.com #2,027, simpli.fi #2,053, customer.io #2,063, rustdesk.com #2,069, getbootstrap.com #2,082, yotpo.com #2,084, posthog.com #2,087, verify.hn #2,090, acronis.com #2,099, lenta.com #2,100, crowdstrike.com #2,119, coinmarketcap.com #2,145, iubenda.com #2,154, typing.com #2,158, americanexpress.com #2,174, unity.com #2,181, adkernel.com #2,195, mgid.com #2,200, nielsen.com #2,209, expedia.com #2,213, websupport.sk #2,221, nodejs.org #2,229, fliphtml5.com #2,246, gdemoideti.ru #2,267, dmac-solutions.net #2,269, sattamatka.email #2,283, adp.com #2,294, semrush.com #2,299, grafana.com #2,300, evernote.com #2,327, dondominio.com #2,346, verkada.com #2,389, ringcentral.com #2,394, mindbox.ru #2,405, goguardian.com #2,406, egnyte.com #2,432, canonical.com #2,446, dpbossss.com #2,456, gameanalytics.com #2,459, dpbosssss.mobi #2,495, su89-cdn.net #2,499, polymarket.com #2,506, kbz0pwvxmv.com #2,507, fotmob.com #2,513, atom.com #2,532, nordvpn.com #2,558, x-cdn-static.com #2,567, c6gj-static.net #2,568, digg.com #2,574, okx.com #2,578, ifeng.com #2,599, yg5sjx5kzy.com #2,625, tenable.com #2,640, parallels.com #2,664, slashdot.org #2,665, weplayapp.com #2,695, fullstory.com #2,707, zscaler.com #2,709, padlet.com #2,723, databricks.com #2,737, here.com #2,747, hindustantimes.com #2,750, cookieyes.com #2,756, malwarebytes.com #2,762, webflow.com #2,772, robinhood.com #2,778, dnsimple.com #2,781, suno.com #2,811, cutt.ly #2,831, rudderstack.com #2,840, guard.io #2,847, shazam.com #2,864, prezi.com #2,865, kpmg.com #2,879, newsbreak.com #2,914, clickup.com #2,919, gumroad.com #2,925, loom.com #2,938, redis.io #2,942, lokalise.com #2,948, tutu.ru #2,953, phys.org #2,958, fox.com #2,962, crealitycloud.com #2,968, crazyegg.com #2,980, libsyn.com #2,993, zapier.com #2,999, dantri.com.vn #3,025, ucoz.ru #3,029, ip.sb #3,033, sardine.ai #3,037, askona.ru #3,039, vercel.com #3,059, snapcraft.io #3,065, ihc.ru #3,079, drpc.org #3,092, cvent.com #3,107, fool.com #3,128, laracasts.com #3,170, imagemagick.org #3,174, cursor.com #3,202, walkme.com #3,221, monday.com #3,225, clickpetroleoegas.com.br #3,229, gldn.net #3,244, iterable.com #3,269, revcontent.com #3,271, elevenlabs.io #3,289, backblaze.com #3,291, clerk.com #3,294, proximus.be #3,311, hotmart.com #3,339, lavanguardia.com #3,342, razorpay.com #3,395, honor.com #3,396, postman.com #3,441, e-recht24.de #3,445, mobilefuse.com #3,449, twingate.com #3,456, jio.com #3,458, hootsuite.com #3,460, xero.com #3,472, joinmassive.com #3,506, pushwoosh.com #3,546, dinahosting.com #3,553, si.com #3,559, 17track.net #3,572, qnap.com #3,580, osano.com #3,582, sprinklr.com #3,585, voximplant.com #3,587, smartsheet.com #3,600, easydns.com #3,639, vkusvill.ru #3,642, bitwarden.com #3,652, arduino.cc #3,661, redbull.com #3,666, replit.com #3,670, demandbase.com #3,673, verisoul.ai #3,676, piano.io #3,684, lovable.dev #3,698, qualcomm.com #3,711, adyen.com #3,723, edx.org #3,727, dev.to #3,742, farlightgames.com #3,784, colorlib.com #3,786, familysearch.org #3,788, sanity.io #3,791, rokt.com #3,798, rome2rio.com #3,810, blackboard.com #3,832, domaintools.com #3,833, nazwa.pl #3,839, trimble.com #3,849, mux.com #3,880, sproutsocial.com #3,887, mysterium.network #3,893, elementor.com #3,904, nanit.com #3,907, remotepc.com #3,913, openrouter.ai #3,921, gamma.app #3,923, g2.com #3,930, admaster.cc #3,931, tum.de #3,939, mygaru.com #3,946, netlify.com #3,967, razer.com #4,012, elespanol.com #4,015, dramaboxdb.com #4,029, rapidssl.com #4,037, op.gg #4,053, lazada.com.ph #4,058, ctrip.com #4,077, tanium.com #4,097, similarweb.com #4,098, db.com #4,099, veeam.com #4,112, wiki.gg #4,135, garant.ru #4,151, grandviewresearch.com #4,155, ubs.com #4,163, helpshift.com #4,165, ts.net #4,200, xe.com #4,201, 1und1.de #4,209, mailerlite.com #4,220, giantpanda.com #4,224, zoneedit.com #4,225, lazada.sg #4,228, growthbook.io #4,268, livemint.com #4,285, producthunt.com #4,318, buffer.com #4,336, higgsfield.ai #4,364, hyundai.com #4,372, crypto.com #4,390, cloudflarestatus.com #4,392, victoriassecret.com #4,411, rediff.com #4,412, paylocity.com #4,431, picsart.com #4,435, ollama.com #4,464, 3ds.com #4,510, samsara.com #4,527, kaggle.com #4,534, terabox.com #4,564, lufthansa.com #4,565, vinted.co.uk #4,567, changeip.com #4,592, kayak.com #4,594, parentsquare.com #4,605, adspower.net #4,607, livechat.com #4,609, stlouisfed.org #4,634, recruit.co.jp #4,651, zoominfo.com #4,683, impact.com #4,684, salesmartly.com #4,690, yango.com #4,703, thetrainline.com #4,736, boredpanda.com #4,737, oppo.com #4,762, tally.so #4,767, westernunion.com #4,772, freshworks.com #4,778, dashlane.com #4,788, patch.com #4,806, vedomosti.ru #4,817, localytics.com #4,840, supabase.com #4,844, opencode.ai #4,864, kion.ru #4,869, thescore.com #4,913, wolfram.com #4,952, qualys.com #4,958, securly.com #4,960, dynadot.com #4,996, depositphotos.com #5,006, infobip.com #5,023, theweathernetwork.com #5,024, classdojo.com #5,025, zalando.de #5,115, framer.com #5,118, serverwatch.com #5,185, findmykids.org #5,214, airtel.in #5,241, bankrate.com #5,263, prodigygame.com #5,265, imperial.ac.uk #5,274, superbet.ro #5,295, zebra.com #5,303, uservoice.com #5,319, granola.ai #5,329, kijiji.ca #5,343, remitly.com #5,361, bittorrent.com #5,363, easybell.de #5,381, hola.org #5,384, skylinewebcams.com #5,402, yesstyle.com #5,409, bambuser.com #5,433, umt.edu #5,461, immowelt.de #5,477, lesoir.be #5,485, monster.com #5,490, narvar.com #5,504, tradedoubler.com #5,513, tradeinn.com #5,517, freeipapi.com #5,518, tuoitre.vn #5,543, techrepublic.com #5,545, vietnamnet.vn #5,568, nownodes.io #5,580, getcomposer.org #5,582, vuejs.org #5,583, outlookindia.com #5,602, glance.com #5,609, canadiantire.ca #5,612, swarovski.com #5,625, hostnet.nl #5,656, justgiving.com #5,659, imunify360.com #5,673, warwick.ac.uk #5,700, turbify.com #5,702, decodo.com #5,704, pubnub.com #5,710, uzh.ch #5,714, empik.com #5,716, maryland.gov #5,720, thinkific.com #5,724, credit-agricole.fr #5,729, mobalytics.gg #5,733, lazada.co.th #5,759, beehiiv.com #5,765, crossref.org #5,768, azbyka.ru #5,773, kucoin.com #5,778, zee5.com #5,801, pixai.art #5,815, groupon.com #5,826, n8n.io #5,847, ravelry.com #5,855, oilprice.com #5,857, milanuncios.com #5,863, ziprecruiter.com #5,866, boingboing.net #5,870, corel.com #5,872, mentalfloss.com #5,873, manageengine.com #5,882, airbridge.io #5,883, 9gag.com #5,912, netshort.com #5,924, mexc.com #5,929, thriftbooks.com #5,962, konghq.com #5,971, navyfederal.org #5,973, klook.com #5,975, wyze.com #5,984, formstack.com #5,989, transcend.io #6,011, ricoh.com #6,034, onamae.com #6,037, airwallex.com #6,055, cleverreach.com #6,060, ifixit.com #6,069, cosme.net #6,088, jooble.org #6,102, beacons.ai #6,119, solitaired.com #6,126, motorola.com #6,130, esri.com #6,140, goodnotes.com #6,162, dnb.com #6,185, playerok.com #6,202, aylanetworks.com #6,203, claro.com.br #6,209, camscanner.com #6,214, logitechg.com #6,225, genially.com #6,232, icims.com #6,262, uu.se #6,272, mynet.com #6,283, tsn.ua #6,295, piwik.pro #6,296, rtbhouse.com #6,301, 1984.is #6,315, tenjin.com #6,329, optum.com #6,347, td.com #6,359, muumuu-domain.com #6,365, fantasypros.com #6,378, tripo3d.ai #6,408, imo.im #6,411, dm.de #6,434, deepmind.google #6,460, mistral.ai #6,467, allegro.cz #6,484, hextom.com #6,487, thanhnien.vn #6,489, gmw.cn #6,504, webnode.com #6,508, wyndhamhotels.com #6,530, narto-drama.com #6,593, cimri.com #6,596, b92.net #6,632, pair.com #6,634, gameforge.com #6,645, cricketwireless.com #6,646, quran.com #6,666, prismic.io #6,680, nykaa.com #6,687, heute.at #6,716, openweathermap.org #6,717, smile.io #6,744, maersk.com #6,748, kiro.dev #6,773, yoast.com #6,780, onedio.com #6,834, 2dehands.be #6,843, orangelogic.com #6,867, tag24.de #6,869, drift-pixel.ai #6,880, mango-office.ru #6,887, me-qr.com #6,907, bamboohr.com #6,940, batch.com #6,942, trovaprezzi.it #6,958, claro.com.ar #6,971, programmaticx.ai #6,977, fastly.com #6,980, medicalxpress.com #6,986, noip.com #6,999, ixigo.com #7,007, qoo10.jp #7,010, carvana.com #7,045, tchibo.de #7,055, mql5.com #7,076, cafef.vn #7,080, bain.com #7,087, nic.ai #7,103, dellin.ru #7,112, almalinux.org #7,114, ledger.com #7,116, hackernoon.com #7,120, monetixads.com #7,137, amarujala.com #7,155, toptal.com #7,191, pik.ru #7,192, akc.org #7,240, suse.com #7,242, ngrok.com #7,260, starfieldtech.com #7,289, thredup.com #7,318, thehackernews.com #7,345, simple.life #7,385, vedantu.com #7,413, opentext.com #7,422, ukrainealarm.com #7,433, gitbook.com #7,438, streamlabs.com #7,447, apollo.io #7,450, palmettostatearmory.com #7,483, ig.com #7,518, dinamalar.com #7,545, ethereum.org #7,555, elfsight.com #7,597, link.me #7,612, booklive.jp #7,616, halodoc.com #7,650, seaart.ai #7,656, tuya.com #7,672, dxc.com #7,677, openart.ai #7,687, kenh14.vn #7,694, queensu.ca #7,702, survicate.com #7,716, trezor.io #7,723, getresponse.com #7,748, bibliaonline.com.br #7,751, react.dev #7,756, district.in #7,757, thenounproject.com #7,760, annas-archive.gl #7,795, mk.co.kr #7,825, catho.com.br #7,840, blackmagicdesign.com #7,842, make.com #7,854, telcel.com #7,860, cloudamqp.com #7,867, wordstream.com #7,881, imovelweb.com.br #7,884, soha.vn #7,934, pmi.org #7,945, sojern.com #7,963, complex.com #7,968, anker.com #7,974, icicidirect.com #7,986, autotrader.co.za #7,993, cashify.in #7,996, sbicard.com #8,013, alchemy.com #8,018, mattel.com #8,049, fikfap.com #8,070, umbc.edu #8,071, arm.com #8,079, dorzeczy.pl #8,088, yesplay.bet #8,096, cars24.com #8,099, storiesig.info #8,117, speakerdeck.com #8,123, webim.ru #8,161, yad2.co.il #8,164, uptimerobot.com #8,189, huanqiu.com #8,194, 1024tera.com #8,231, pocketfm.com #8,243, booksy.com #8,248, terabox.app #8,249, bikedekho.com #8,252, tubidy.com #8,304, allegro.hu #8,326, zougla.gr #8,372, timmehosting.de #8,394, doz.pl #8,407, anonymised.io #8,408, flexport.com #8,425, bebee.com #8,438, roborock.com #8,447, linear.app #8,452, linksys.com #8,461, linklist.bio #8,469, ilbe.com #8,482, onlinejobs.ph #8,484, getjobber.com #8,523, olist.com #8,526, pipedrive.com #8,532, autozone.com #8,597, anonyig.com #8,641, daisonet.com #8,646, surahquran.com #8,666, win2day.at #8,680, lcw.com #8,723, hyros.com #8,773, sssinstagram.com #8,796, harness.io #8,809, yoomoney.ru #8,814, freejobalert.com #8,841, decathlon.in #8,863, shop-pro.jp #8,865, abhibus.com #8,881, eufy.com #8,882, illumin.com #8,913, solitairegrandharvest.com #8,933, axismaxlife.com #8,951, sexiha.cam #8,968, delicious.com #8,976, banco.bradesco #8,980, startech.com.bd #8,981, qt.io #8,990, okbet.com #9,003, sexbebin.com #9,014, photoroom.com #9,035, javgg.net #9,048, sportybet.co.za #9,061, steamunlocked.org #9,063, savefrom.co.id #9,077, nobat.ir #9,091, apuestatotal.com #9,093, chiikawamarket.jp #9,103, vivastreet.co.uk #9,129, gumtree.com #9,147, pocketcasts.com #9,151, aniagotuje.pl #9,182, gogoroyal.com #9,210, beijing.gov.cn #9,264, megagroup.ru #9,277, arabnews.com #9,298, q10.com #9,299, rosszlanyok.hu #9,302, dg-datenschutz.de #9,304, creativethemes.com #9,323, modelcontextprotocol.io #9,337, alerts.in.ua #9,409, ecoflow.com #9,419, bezeq.co.il #9,450, havadurumu15gunluk.xyz #9,452, mxtoolbox.com #9,487, classificadosx.net #9,498, cleantalk.org #9,504, riyasewana.com #9,512, travesticomlocal.com.br #9,537, garotocomlocal.com.br #9,538, modaselvim.com #9,575, august.com #9,580, enloteria.com #9,697, artsy.net #9,705, petsmart.com #9,710, arabic-porn-tube.com #9,713, rozbano.com #9,714, underarmour.com #9,756, discoverpilgrim.com #9,777, nishatlinen.com #9,780, espacioapk.com #9,800, healthychildren.org #9,825, saya.pk #9,862, megaescort.info #9,870, zellbury.com #9,874, uncenz.com #9,878, faymas.in #9,882, elgato.com #9,895, wri.org #9,920, checkout.com #9,927, wakelet.com #9,962, gaiaonline.com #9,985, xxxza.com #9,991

MCP server card: 50 sites

cloudflare.com #2, sentry.io #102, forter.com #207, amplitude.com #246, linktr.ee #264, cm.com #366, zoom.com #833, twilio.com #1,137, intercom.com #1,215, oxylabs.io #1,336, notion.com #1,411, coinbase.com #1,426, hostinger.com #1,837, wpastra.com #1,879, kraken.com #1,967, customer.io #2,063, padlet.com #2,723, cookieyes.com #2,756, webflow.com #2,772, gumroad.com #2,925, crazyegg.com #2,980, zapier.com #2,999, monday.com #3,225, elevenlabs.io #3,289, clerk.com #3,294, razorpay.com #3,395, hootsuite.com #3,460, wistia.com #3,590, openrouter.ai #3,921, netlify.com #3,967, mailerlite.com #4,220, opensea.io #4,569, supabase.com #4,844, infobip.com #5,023, granola.ai #5,329, nownodes.io #5,580, glance.com #5,609, decodo.com #5,704, beehiiv.com #5,765, icons8.com #6,207, haveibeenpwned.com #6,669, openweathermap.org #6,717, cal.com #6,897, ngrok.com #7,260, gitbook.com #7,438, getresponse.com #7,748, alchemy.com #8,018, uptimerobot.com #8,189, pocketcasts.com #9,151, modelcontextprotocol.io #9,337

A2A agent card: 11 sites

cloudflare.com #2, forter.com #207, medal.tv #1,850, cookieyes.com #2,756, monday.com #3,225, razorpay.com #3,395, opensea.io #4,569, glance.com #5,609, vidyard.com #7,816, modelcontextprotocol.io #9,337, alerts.in.ua #9,409

ARD manifest: 11 sites

cm.com #366, bluehost.com #470, gameanalytics.com #2,459, cookieyes.com #2,756, hootsuite.com #3,460, elementor.com #3,904, openrouter.ai #3,921, netlify.com #3,967, supabase.com #4,844, pixai.art #5,815, arabnews.com #9,298

ARD manifest, older path only: 18 sites

github.com #34, amplitude.com #246, huggingface.co #1,078, oxylabs.io #1,336, hostinger.com #1,837, bitdefender.com #1,843, padlet.com #2,723, rudderstack.com #2,840, airtable.com #2,929, zapier.com #2,999, vercel.com #3,059, clerk.com #3,294, beehiiv.com #5,765, haveibeenpwned.com #6,669, cal.com #6,897, ngrok.com #7,260, gitbook.com #7,438, alchemy.com #8,018

API catalog (RFC 9727): 53 sites

sentry.io #102, forter.com #207, amplitude.com #246, linktr.ee #264, cm.com #366, name.com #399, bluehost.com #470, redhat.com #680, zoom.com #833, huggingface.co #1,078, intercom.com #1,215, oxylabs.io #1,336, coinbase.com #1,426, hostinger.com #1,837, bitdefender.com #1,843, wpastra.com #1,879, kraken.com #1,967, customer.io #2,063, coinmarketcap.com #2,145, padlet.com #2,723, cookieyes.com #2,756, crazyegg.com #2,980, zapier.com #2,999, vercel.com #3,059, monday.com #3,225, elevenlabs.io #3,289, clerk.com #3,294, razorpay.com #3,395, hootsuite.com #3,460, wistia.com #3,590, sanity.io #3,791, openrouter.ai #3,921, netlify.com #3,967, mailerlite.com #4,220, zoneedit.com #4,225, buffer.com #4,336, cloudflarestatus.com #4,392, opensea.io #4,569, supabase.com #4,844, infobip.com #5,023, decodo.com #5,704, beehiiv.com #5,765, pixai.art #5,815, icons8.com #6,207, haveibeenpwned.com #6,669, onedio.com #6,834, cal.com #6,897, ngrok.com #7,260, gitbook.com #7,438, getresponse.com #7,748, uptimerobot.com #8,189, librarything.com #9,168, faymas.in #9,882

Link: rel="ard": 3 sites

cm.com #366, netlify.com #3,967, pixai.art #5,815

SVCB _index._agents: 11 sites

hostinger.com #1,837, customer.io #2,063, padlet.com #2,723, cookieyes.com #2,756, coingecko.com #3,292, domain.com #4,207, opensea.io #4,569, 5ch.io #5,619, networksolutions.com #5,800, ngrok.com #7,260, faymas.in #9,882

AI crawlers in robots.txt

4,928 of the 6,935 sites serve a robots.txt; 1,095 have a group that names at least one AI crawler and disallows the whole site to it. Named means a group names the crawler without blocking all of it.

CrawlerBlockedShareNamed, not blocked
CCBot85412.3%134
Bytespider83912.1%82
GPTBot81211.7%300
ClaudeBot76311.0%225
Google-Extended6819.8%247
meta-externalagent6609.5%96
Applebot-Extended6469.3%97
PerplexityBot4166.0%274
ChatGPT-User3354.8%306
OAI-SearchBot2353.4%307
Claude-SearchBot2193.2%135

Check your own site

curl -s https://example.com/llms.txt | head -3
curl -sI https://example.com/.well-known/ard.json | grep -i content-type
curl -s https://example.com/.well-known/mcp.json
curl -s https://example.com/.well-known/agent-card.json

Or run the free audit. To publish an ARD manifest, see how to publish or the manifest generator. How the sites that already publish one meet the new IETF drafts is tracked daily on the ARD IETF readiness tracker.

Data and method

Download the full measurement as JSON (summary, history and one row per domain) or CSV, under CC BY 4.0. Please cite it as Neuronto Agent Readiness Index, 2026-11-02, https://neuronto.com/agent-readiness-index.

The Tranco top list by its permanent id, each domain checked once over HTTPS. A file counts only when its content has the right shape: JSON with an entries array for an ARD manifest, a name plus url or skills for an A2A agent card, a linkset for an API catalog, text starting with a heading for llms.txt. A 200 answered with an HTML page is a soft 404, not a file. Redirects are followed only within the domain. Percentages use the domains that answered HTTPS as the denominator. Each domain is requested once, over HTTPS, with an identifying user agent. A JSON file counts as an ARD manifest when it has an entries array; as an A2A agent card when it has a name and a url or skills; as an API catalog when it has a linkset. An llms.txt counts when its first line is a Markdown heading and it is not HTML. A DNS answer counts only when its record type is SVCB. robots.txt is read as RFC 9309 groups; a crawler counts as blocked when a group naming it disallows /.