Agents, MCP and WebMCP
Can AI agents read your website? Every Check Website tells you.
When you check a website on our home page, we answer two questions at once: does it respond for people, and can AI agents read it? The second answer is a speedometer from 0 to 100, with every fix you can make to raise it.
Quick answer
AI agents are programs, such as ChatGPT, Claude or Perplexity, that read websites or use them for a person. They read best when a page has real text in its HTML, clear headings, open bot rules and machine-readable helpers. Our CheckWebsiteNow Agent Readability Score measures exactly that, from public files, in about ten seconds. It is our own transparent score, not an official standard.
What happens with every Check Website
- We check the website for people. Does the HTTPS homepage answer from our measurement point? You get the verdict, the reason and free fix steps.
- At the same time we check it for AI agents. We read the homepage HTML, robots.txt (the rules for bots), llms.txt, the API and MCP discovery files, the sitemap and a Markdown version. No scripts run and nothing is sent.
- You see the speedometer. One number from 0 to 100 for agent readiness, a band (Low agent readiness, Partly agent-ready, Good, Excellent), one plain sentence that says separately whether agents can read the text, four group bars (unmeasured parts drawn as "not measured") and a clear status line for MCP and for WebMCP.
- You get the fixes. “How to raise your score” lists every lost point, biggest first, each with a copy-ready snippet for your site.
This works for any public website: yours, a competitor’s, or ours. If too little could be measured, we show the reason instead of a guessed number.
MCP and WebMCP in plain English
MCP (Model Context Protocol)
A common standard that lets AI assistants call tools on a server. Think of it as one plug that fits many assistants. A website with an MCP server lets an assistant ask it directly, for example “check example.com”, instead of reading pages.
We look for MCP announcements: an MCP Server Card, an AI Catalog entry and an API catalog. Advertised, not tested: we never call your endpoint.
WebMCP (tools inside the web page)
A new browser standard that lets a web page offer its own functions, such as a search form, as tools to an AI agent working in the browser. The agent can then use the form properly instead of clicking around.
We look for WebMCP form attributes (toolname, tooldescription, toolparamdescription) and inline registration. Declared, not verified in a browser: WebMCP is experimental (Chrome origin trial). WebMCP vs MCP explained, with a free check of your site.
Our own site: CheckWebsiteNow is built to pass every check of this score: open bot rules with a Content-Signal line, llms.txt, an RFC 9727 API catalog, an MCP Server Card listed in our AI Catalog, a WebMCP-declared check form and Markdown versions of our pages. Check checkwebsitenow.com on the home page to see the evidence yourself.
How the CheckWebsiteNow Agent Readability Score works
Our own transparent score from 0 to 100 (model cwn-ars-1), not an official standard. It runs with every Check Website on our home page, through POST /api/v1/agent-check, the MCP tool check_agent_readability and the WebMCP browser tool of the same name. The same constants below compute every score; the machine-readable version is /api/v1/agent-readability-method.
Four groups, 100 points
| Group | What it covers | Points |
|---|---|---|
| A | AI bot access | 25 |
| B | Readable content | 32 |
| C | Structure and metadata | 20 |
| D | Agent help and actions: llms.txt, MCP and WebMCP | 23 |
The 14 checks and their weights
| ID | Group | Check | Weight | Rule |
|---|---|---|---|---|
| A1 | AI bot access | Public page answers with HTML | 10 | Pass: complete 2xx text/html without login or bot-challenge hints. Partial: such a hint. Fail: 404, 410, 5xx or no HTML. Unknown: 401/403/429, timeouts or framing we do not read (may only affect our probe). |
| A2 | AI bot access | Search and user agents may read the homepage | 10 | robots.txt rules for Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot, ChatGPT-User, Claude-User and Perplexity-User on the final homepage path. Pass: none blocked (also no robots.txt). Partial: some blocked. Fail: all five search bots blocked. Training bots are never scored here. |
| A3 | AI bot access | Explicit AI usage policy | 5 | Rewards clarity, not openness. Pass: a robots.txt group for an AI-specific bot or a Content-Signal line (allow and disallow count the same). Partial: only general rules. Fail: no robots.txt. |
| B1 | Readable content | Text is in the HTML (no JavaScript-only shell) | 12 | Visible text characters outside script, style, template, noscript and hidden elements. Pass: 500 or more. Partial: 100 to 499. Fail: under 100 (score capped at 49). Thresholds are our product rule. |
| B2 | Readable content | Clear heading structure | 8 | Pass: exactly one visible h1 and at least one h2. Partial: headings, but no h1, several h1 or no h2. Fail: no headings. |
| B3 | Readable content | Landmarks, language, labels and alt text | 8 | Three parts: html lang; main landmark; at least 80 % of form fields labelled and images with alt (not applicable without fields or images). Pass: all applicable parts. Partial: at least half. Fail: fewer. |
| B4 | Readable content | Markdown version for agents | 4 | Pass: the homepage answers Accept: text/markdown with text/markdown. Partial: Markdown is only advertised (alternate link or .md links in llms.txt). Fail: neither. |
| C1 | Structure and metadata | Title and description | 5 | Pass: non-empty title and meta description. Partial: one of them. Fail: neither. |
| C2 | Structure and metadata | Indexable, with a canonical address | 5 | Fail: noindex in meta robots or X-Robots-Tag. Partial: no canonical link to this HTTPS host, or nosnippet / max-snippet:0. Pass: indexable with a canonical link to this HTTPS host. |
| C3 | Structure and metadata | Structured data (JSON-LD) | 5 | Pass: at least one application/ld+json block that parses and has @type (directly or in @graph). Partial: blocks, but none valid. Fail: none. External @context is never fetched. |
| C4 | Structure and metadata | Sitemap | 5 | Pass: a same-origin sitemap (robots.txt Sitemap line, else /sitemap.xml) starts with urlset or sitemapindex (first 64 KiB, no XML parser). Partial: declared but on another origin or not confirmed. Fail: not declared and /sitemap.xml missing. |
| D1 | Agent help and actions: llms.txt, MCP and WebMCP | llms.txt guide | 5 | Optional; Google Search ignores it. Pass: /llms.txt as plain text or Markdown, first line an H1 (# ), at least one Markdown link. Partial: present without H1 or link. Fail: 404/410. |
| D2 | Agent help and actions: llms.txt, MCP and WebMCP | MCP and API discovery | 10 | Pass: valid /.well-known/api-catalog (RFC 9727 linkset), /.well-known/ard.json (Agentic Resource Discovery, a proposal: entries with identifier, type and url or data) or the older /.well-known/ai-catalog.json with entries, an MCP Server Card at /.well-known/mcp/server-card.json (name plus remote endpoint), or a same-origin rel="api-catalog" / rel="service-desc" / rel="ard" link. Partial: llms.txt only mentions OpenAPI or MCP. Fail: nothing found. Advertised, not tested: endpoints are never called. |
| D3 | Agent help and actions: llms.txt, MCP and WebMCP | WebMCP browser tools declared | 8 | Pass: a form with toolname and tooldescription where every named field has toolparamdescription. Partial: toolname without descriptions, or an inline script calling modelContext registerTool. Fail: no static hint. Declared, not verified in a browser; external scripts are not loaded. |
Calculation
- Each check: pass = full weight, partial = half, fail = 0.
- Unknown and not applicable checks are removed from both the points and the maximum; they never count as passed.
- Score = 100 x earned points / measured points, rounded down (never up).
- No number is shown when the homepage is not readable public HTML or fewer than 70 of 100 points could be measured.
- Caps: all five AI search bots blocked caps the score at 39; a JavaScript-only shell (B1 fail) caps it at 49.
Coverage rule: a number is shown only when at least 70 of 100 points could be measured and the homepage is readable public HTML. Otherwise we show the reason, never a guess.
Caps: search bots blocked: at most 39; javascript shell: at most 49.
Bands
- 0 to 39: Low agent readiness
- 40 to 69: Partly agent-ready
- 70 to 89: Good
- 90 to 100: Excellent
- No number: Not enough evidence
MCP and WebMCP
MCP discovery (D2) reads /.well-known/api-catalog, /.well-known/ard.json (Agentic Resource Discovery, a proposal), the MCP Server Card at /.well-known/mcp/server-card.json, homepage links and, last, the older /.well-known/ai-catalog.json. Advertised, not tested: endpoints are never called.
WebMCP (D3) reads declarative form attributes (toolname, tooldescription, toolparamdescription) and inline scripts. Declared, not verified in a browser: runtime registration is only visible in a supporting browser.
Training opt-outs (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent) are an owner choice and never lower the score.
Limits
- Static inspection only; no browser scripts, forms or actual provider bots were executed.
- At most 9 HTTPS requests within a shared 10-second budget: homepage, robots.txt, llms.txt, three discovery files (api-catalog, ard.json, MCP Server Card), sitemap prefix, one Markdown negotiation and, last, the older ai-catalog.json, all on the final homepage origin; their redirects are followed only on that host and count towards the nine, so the older file is the first to stay unknown.
- No proof of search ranking, AI citations, actual bot access through firewalls or CDNs, or working WebMCP tools.
- MCP and API endpoints are only read as announcements; they are never called.
- Bot rules: RFC 9309 matching (user-agent groups, longest match, * and $ wildcards) for the final homepage path only; a robots.txt that redirects to another host or is not a robots file stays unknown.
- Homepage HTML is read up to 1 MiB; a larger homepage is analysed from its first 1 MiB (homepage_truncated). 64 KiB per text file is an inspection cap; sitemap and Markdown use only the first 64 KiB.
- Only the final homepage is scored; subpages and the accuracy of llms.txt, JSON-LD or server cards are not checked.
- Plain chunked framing is supported; chunk extensions and other transfer codings remain unknown.
- Only the first validated public network address is contacted.
- Unknown checks are removed from the calculation and never count as passed.
CheckWebsiteNow Agent Readability Score: our own transparent score from static evidence (homepage HTML, robots.txt, llms.txt and discovery files). Not an official standard. It cannot see runtime WebMCP tools, firewall rules or whether a specific AI bot really gets through.
HTTP API
JSON over HTTPS. No key and no account. Send one public domain; paths, credentials, IP addresses and custom ports are rejected. The full contract is in the OpenAPI description. Step-by-step examples in curl, Python and JavaScript: API guide.
/api/v1/checkIs the homepage responding?curl -s https://checkwebsitenow.com/api/v1/check \
-H "Content-Type: application/json" \
-d '{"hostname":"example.com"}'
The response contains status (for example reachable, access_restricted, dns_error), http_status, final_url, redirect_count, redirect_chain, evidence, limitations and, when an entry matches, a diagnosis with the best fix guide and related errors.
/api/v1/agent-checkCan AI agents read the website? Score 0-100Returns score (0-100 or null with score_unavailable_reason), band, coverage, four groups, 14 checks with a fix for every deduction, deductions, and explicit mcp and webmcp fields. Homepages up to 1 MiB are read completely; a larger homepage is analysed from its first 1 MiB and homepage_truncated says so.
/api/v1/agent-readability-methodThe score method as JSON/api/v1/access-solutions?domain=example.comWebsite blocked in your country? Ordered solutions| status | Meaning |
|---|---|
| reachable | The tested final destination answered successfully. |
| redirected | The redirect could not safely be completed; inspect redirect_stop_reason. |
| access_restricted | Authentication, protection or rate limiting restricted the request. |
| http_error | The final tested request returned an HTTP error. |
| dns_error / tls_error | Name resolution or secure connection failed at an attempted destination. |
| timeout / probe_error | Inconclusive: the measurement did not complete. |
Error library endpoints
/api/v1/errors/lookup?q=ERR_CONNECTION_REFUSEDMatch pasted error text/api/v1/errors?category=http-5xx&kind=http_statusList entries, optional filters/api/v1/errors/http-503One full entry/api/v1/status-sitesCurated “Is it down?” services/api/v1/statsPublic service metadata and inventoryMachine-readable methodology · Agent discovery guide · API catalog · OpenAPI
Remote MCP endpoint
For MCP clients such as desktop assistants and coding agents. Transport: Streamable HTTP, stateless, JSON responses. No session and no authentication. Server Card: /mcp/server-card.
https://checkwebsitenow.com/mcpModel Context ProtocolAdd it to your MCP client
{
"mcpServers": {
"checkwebsitenow": {
"type": "http",
"url": "https://checkwebsitenow.com/mcp"
}
}
}
Claude Code: claude mcp add --transport http checkwebsitenow https://checkwebsitenow.com/mcp
Raw protocol example
curl -s https://checkwebsitenow.com/mcp \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"check_agent_readability","arguments":{"hostname":"example.com"}}}'
The nine tools
check_website {hostname}Is the public HTTPS homepage responding from our measurement point? Verdict, evidence, limits and fix guide.check_agent_readability {hostname}The CheckWebsiteNow Agent Readability Score 0-100 with a fix for every deduction; MCP advertised, not tested; WebMCP declared, not verified.get_agent_readability_method {}How the score works: groups, checks, weights, caps, bands and limits.get_access_solutions {domain, case?, visitor_reported?}Website blocked in a country? Free solutions first; VPN providers only when a regional block is suspected; ads labelled.explain_error {query}Match a pasted browser, HTTP, DNS or TLS error and return the fixes.list_errors {category?, kind?}List the error library.get_error {slug}One complete solution entry.list_status_sites {}Curated services with verified official status links where available.get_service_info {}Method, limits, version and knowledge inventory. No visitor analytics.
check_website answers first with agent_guidance, the result in plain English with the likely cause, the first steps and the guide link for the person, and lists the same labelled VPN ad people see after a result separately in sponsored_offers, which agents should never open automatically but may show as an optional ad.
WebMCP browser tools
Our home page check form is declared as the WebMCP tool check_website with toolname, tooldescription and toolparamdescription attributes, so a browser agent can see it without running scripts. In browsers that support the experimental WebMCP API, every page also registers the same tools as our MCP server through document.modelContext.registerTool(), including check_agent_readability and get_access_solutions. Each tool returns a short text summary plus the JSON result. On the home page the person in front of the browser sees the same result.
WebMCP is experimental. Ordinary browser use, the HTTP API and the remote MCP endpoint work without it. There is no promise of support in every browser, an agent-directory listing or improved search ranking. Chrome’s WebMCP documentation. Our own explanation with examples and a free check: WebMCP vs MCP: what it is and how to check your site.
Websites blocked in a country
Some websites block visitors from certain countries (geoblocking). Our guide for blocked websites, the endpoint /api/v1/access-solutions and the MCP tool get_access_solutions give people and agents the same answer: free solutions first, then VPN providers, but only when a regional block is suspected. The order comes from documented criteria and never from commission. Paid links are marked “Ad” and carry a disclosure; agents get the same labels and are asked to show them.
Limits
- Website checks: ten per minute per client. Agent-readability checks: ten per minute per client, so both can run for every Check Website. One running check per client and type; results are reused for sixty seconds.
- Both check types share four active probe slots. The agent check reads at most nine files within one shared ten-second budget.
- The remote MCP endpoint uses the same limits as the HTTP API, plus sixty MCP requests per minute.
- Busy-service or rate-limit errors are not verdicts about the target website.
- One measurement point, not global coverage. A successful response does not test sign-in, checkout or every page.
- No long-term uptime percentage or verified visitor count is claimed.
What the quick check measures
Every requested destination is resolved and all returned addresses must be public. We connect to the verified numeric address while checking the HTTPS certificate for the hostname. Up to five safe HTTPS redirects are followed within one ten-second measurement budget. HEAD requests can fall back to GET response headers for 403, 404, 405 or 501 responses. The quick response check does not inspect page contents. Sensitive query strings are omitted from exposed URLs.
Public evidence, private analytics
Utility observations, the score method and the error library are public. Agent access never returns visitor analytics, session recordings, input history or access data. The project cockpit and improvement proposals are not available on this public service. See the privacy details.
Helping agents use your own website
A public response, readable content and callable browser actions are separate questions. Practical steps for website owners cover clear text and controls, search versus training bots, hosting protection, optional llms.txt and experimental WebMCP. None of this is a promise of search inclusion or improved ranking.