Agents, MCP and WebMCP

Can AI agents read your website? Every Check Website tells you.

When you check a website on our home page, we answer two questions at once: does it respond for people, and can AI agents read it? The second answer is a speedometer from 0 to 100, with every fix you can make to raise it.

Quick answer

AI agents are programs, such as ChatGPT, Claude or Perplexity, that read websites or use them for a person. They read best when a page has real text in its HTML, clear headings, open bot rules and machine-readable helpers. Our CheckWebsiteNow Agent Readability Score measures exactly that, from public files, in about ten seconds. It is our own transparent score, not an official standard.

What happens with every Check Website

  1. We check the website for people. Does the HTTPS homepage answer from our measurement point? You get the verdict, the reason and free fix steps.
  2. At the same time we check it for AI agents. We read the homepage HTML, robots.txt (the rules for bots), llms.txt, the API and MCP discovery files, the sitemap and a Markdown version. No scripts run and nothing is sent.
  3. You see the speedometer. One number from 0 to 100 for agent readiness, a band (Low agent readiness, Partly agent-ready, Good, Excellent), one plain sentence that says separately whether agents can read the text, four group bars (unmeasured parts drawn as "not measured") and a clear status line for MCP and for WebMCP.
  4. You get the fixes. “How to raise your score” lists every lost point, biggest first, each with a copy-ready snippet for your site.

This works for any public website: yours, a competitor’s, or ours. If too little could be measured, we show the reason instead of a guessed number.

MCP and WebMCP in plain English

MCP (Model Context Protocol)

A common standard that lets AI assistants call tools on a server. Think of it as one plug that fits many assistants. A website with an MCP server lets an assistant ask it directly, for example “check example.com”, instead of reading pages.

We look for MCP announcements: an MCP Server Card, an AI Catalog entry and an API catalog. Advertised, not tested: we never call your endpoint.

WebMCP (tools inside the web page)

A new browser standard that lets a web page offer its own functions, such as a search form, as tools to an AI agent working in the browser. The agent can then use the form properly instead of clicking around.

We look for WebMCP form attributes (toolname, tooldescription, toolparamdescription) and inline registration. Declared, not verified in a browser: WebMCP is experimental (Chrome origin trial). WebMCP vs MCP explained, with a free check of your site.

Our own site: CheckWebsiteNow is built to pass every check of this score: open bot rules with a Content-Signal line, llms.txt, an RFC 9727 API catalog, an MCP Server Card listed in our AI Catalog, a WebMCP-declared check form and Markdown versions of our pages. Check checkwebsitenow.com on the home page to see the evidence yourself.

How the CheckWebsiteNow Agent Readability Score works

Our own transparent score from 0 to 100 (model cwn-ars-1), not an official standard. It runs with every Check Website on our home page, through POST /api/v1/agent-check, the MCP tool check_agent_readability and the WebMCP browser tool of the same name. The same constants below compute every score; the machine-readable version is /api/v1/agent-readability-method.

Four groups, 100 points

GroupWhat it coversPoints
AAI bot access25
BReadable content32
CStructure and metadata20
DAgent help and actions: llms.txt, MCP and WebMCP23

The 14 checks and their weights

IDGroupCheckWeightRule
A1AI bot accessPublic page answers with HTML10Pass: complete 2xx text/html without login or bot-challenge hints. Partial: such a hint. Fail: 404, 410, 5xx or no HTML. Unknown: 401/403/429, timeouts or framing we do not read (may only affect our probe).
A2AI bot accessSearch and user agents may read the homepage10robots.txt rules for Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot, ChatGPT-User, Claude-User and Perplexity-User on the final homepage path. Pass: none blocked (also no robots.txt). Partial: some blocked. Fail: all five search bots blocked. Training bots are never scored here.
A3AI bot accessExplicit AI usage policy5Rewards clarity, not openness. Pass: a robots.txt group for an AI-specific bot or a Content-Signal line (allow and disallow count the same). Partial: only general rules. Fail: no robots.txt.
B1Readable contentText is in the HTML (no JavaScript-only shell)12Visible text characters outside script, style, template, noscript and hidden elements. Pass: 500 or more. Partial: 100 to 499. Fail: under 100 (score capped at 49). Thresholds are our product rule.
B2Readable contentClear heading structure8Pass: exactly one visible h1 and at least one h2. Partial: headings, but no h1, several h1 or no h2. Fail: no headings.
B3Readable contentLandmarks, language, labels and alt text8Three parts: html lang; main landmark; at least 80 % of form fields labelled and images with alt (not applicable without fields or images). Pass: all applicable parts. Partial: at least half. Fail: fewer.
B4Readable contentMarkdown version for agents4Pass: the homepage answers Accept: text/markdown with text/markdown. Partial: Markdown is only advertised (alternate link or .md links in llms.txt). Fail: neither.
C1Structure and metadataTitle and description5Pass: non-empty title and meta description. Partial: one of them. Fail: neither.
C2Structure and metadataIndexable, with a canonical address5Fail: noindex in meta robots or X-Robots-Tag. Partial: no canonical link to this HTTPS host, or nosnippet / max-snippet:0. Pass: indexable with a canonical link to this HTTPS host.
C3Structure and metadataStructured data (JSON-LD)5Pass: at least one application/ld+json block that parses and has @type (directly or in @graph). Partial: blocks, but none valid. Fail: none. External @context is never fetched.
C4Structure and metadataSitemap5Pass: a same-origin sitemap (robots.txt Sitemap line, else /sitemap.xml) starts with urlset or sitemapindex (first 64 KiB, no XML parser). Partial: declared but on another origin or not confirmed. Fail: not declared and /sitemap.xml missing.
D1Agent help and actions: llms.txt, MCP and WebMCPllms.txt guide5Optional; Google Search ignores it. Pass: /llms.txt as plain text or Markdown, first line an H1 (# ), at least one Markdown link. Partial: present without H1 or link. Fail: 404/410.
D2Agent help and actions: llms.txt, MCP and WebMCPMCP and API discovery10Pass: valid /.well-known/api-catalog (RFC 9727 linkset), /.well-known/ard.json (Agentic Resource Discovery, a proposal: entries with identifier, type and url or data) or the older /.well-known/ai-catalog.json with entries, an MCP Server Card at /.well-known/mcp/server-card.json (name plus remote endpoint), or a same-origin rel="api-catalog" / rel="service-desc" / rel="ard" link. Partial: llms.txt only mentions OpenAPI or MCP. Fail: nothing found. Advertised, not tested: endpoints are never called.
D3Agent help and actions: llms.txt, MCP and WebMCPWebMCP browser tools declared8Pass: a form with toolname and tooldescription where every named field has toolparamdescription. Partial: toolname without descriptions, or an inline script calling modelContext registerTool. Fail: no static hint. Declared, not verified in a browser; external scripts are not loaded.

Calculation

  • Each check: pass = full weight, partial = half, fail = 0.
  • Unknown and not applicable checks are removed from both the points and the maximum; they never count as passed.
  • Score = 100 x earned points / measured points, rounded down (never up).
  • No number is shown when the homepage is not readable public HTML or fewer than 70 of 100 points could be measured.
  • Caps: all five AI search bots blocked caps the score at 39; a JavaScript-only shell (B1 fail) caps it at 49.

Coverage rule: a number is shown only when at least 70 of 100 points could be measured and the homepage is readable public HTML. Otherwise we show the reason, never a guess.

Caps: search bots blocked: at most 39; javascript shell: at most 49.

Bands

  • 0 to 39: Low agent readiness
  • 40 to 69: Partly agent-ready
  • 70 to 89: Good
  • 90 to 100: Excellent
  • No number: Not enough evidence

MCP and WebMCP

MCP discovery (D2) reads /.well-known/api-catalog, /.well-known/ard.json (Agentic Resource Discovery, a proposal), the MCP Server Card at /.well-known/mcp/server-card.json, homepage links and, last, the older /.well-known/ai-catalog.json. Advertised, not tested: endpoints are never called.

WebMCP (D3) reads declarative form attributes (toolname, tooldescription, toolparamdescription) and inline scripts. Declared, not verified in a browser: runtime registration is only visible in a supporting browser.

Training opt-outs (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Meta-ExternalAgent) are an owner choice and never lower the score.

Limits

  • Static inspection only; no browser scripts, forms or actual provider bots were executed.
  • At most 9 HTTPS requests within a shared 10-second budget: homepage, robots.txt, llms.txt, three discovery files (api-catalog, ard.json, MCP Server Card), sitemap prefix, one Markdown negotiation and, last, the older ai-catalog.json, all on the final homepage origin; their redirects are followed only on that host and count towards the nine, so the older file is the first to stay unknown.
  • No proof of search ranking, AI citations, actual bot access through firewalls or CDNs, or working WebMCP tools.
  • MCP and API endpoints are only read as announcements; they are never called.
  • Bot rules: RFC 9309 matching (user-agent groups, longest match, * and $ wildcards) for the final homepage path only; a robots.txt that redirects to another host or is not a robots file stays unknown.
  • Homepage HTML is read up to 1 MiB; a larger homepage is analysed from its first 1 MiB (homepage_truncated). 64 KiB per text file is an inspection cap; sitemap and Markdown use only the first 64 KiB.
  • Only the final homepage is scored; subpages and the accuracy of llms.txt, JSON-LD or server cards are not checked.
  • Plain chunked framing is supported; chunk extensions and other transfer codings remain unknown.
  • Only the first validated public network address is contacted.
  • Unknown checks are removed from the calculation and never count as passed.

CheckWebsiteNow Agent Readability Score: our own transparent score from static evidence (homepage HTML, robots.txt, llms.txt and discovery files). Not an official standard. It cannot see runtime WebMCP tools, firewall rules or whether a specific AI bot really gets through.

HTTP API

JSON over HTTPS. No key and no account. Send one public domain; paths, credentials, IP addresses and custom ports are rejected. The full contract is in the OpenAPI description. Step-by-step examples in curl, Python and JavaScript: API guide.

POST/api/v1/checkIs the homepage responding?
curl -s https://checkwebsitenow.com/api/v1/check \
  -H "Content-Type: application/json" \
  -d '{"hostname":"example.com"}'

The response contains status (for example reachable, access_restricted, dns_error), http_status, final_url, redirect_count, redirect_chain, evidence, limitations and, when an entry matches, a diagnosis with the best fix guide and related errors.

POST/api/v1/agent-checkCan AI agents read the website? Score 0-100

Returns score (0-100 or null with score_unavailable_reason), band, coverage, four groups, 14 checks with a fix for every deduction, deductions, and explicit mcp and webmcp fields. Homepages up to 1 MiB are read completely; a larger homepage is analysed from its first 1 MiB and homepage_truncated says so.

GET/api/v1/agent-readability-methodThe score method as JSON
GET/api/v1/access-solutions?domain=example.comWebsite blocked in your country? Ordered solutions
statusMeaning
reachableThe tested final destination answered successfully.
redirectedThe redirect could not safely be completed; inspect redirect_stop_reason.
access_restrictedAuthentication, protection or rate limiting restricted the request.
http_errorThe final tested request returned an HTTP error.
dns_error / tls_errorName resolution or secure connection failed at an attempted destination.
timeout / probe_errorInconclusive: the measurement did not complete.

Error library endpoints

GET/api/v1/errors/lookup?q=ERR_CONNECTION_REFUSEDMatch pasted error text
GET/api/v1/errors?category=http-5xx&kind=http_statusList entries, optional filters
GET/api/v1/errors/http-503One full entry
GET/api/v1/status-sitesCurated “Is it down?” services
GET/api/v1/statsPublic service metadata and inventory

Machine-readable methodology · Agent discovery guide · API catalog · OpenAPI

Remote MCP endpoint

For MCP clients such as desktop assistants and coding agents. Transport: Streamable HTTP, stateless, JSON responses. No session and no authentication. Server Card: /mcp/server-card.

POSThttps://checkwebsitenow.com/mcpModel Context Protocol

Add it to your MCP client

{
  "mcpServers": {
    "checkwebsitenow": {
      "type": "http",
      "url": "https://checkwebsitenow.com/mcp"
    }
  }
}

Claude Code: claude mcp add --transport http checkwebsitenow https://checkwebsitenow.com/mcp

Raw protocol example

curl -s https://checkwebsitenow.com/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"check_agent_readability","arguments":{"hostname":"example.com"}}}'

The nine tools

check_website answers first with agent_guidance, the result in plain English with the likely cause, the first steps and the guide link for the person, and lists the same labelled VPN ad people see after a result separately in sponsored_offers, which agents should never open automatically but may show as an optional ad.

WebMCP browser tools

Our home page check form is declared as the WebMCP tool check_website with toolname, tooldescription and toolparamdescription attributes, so a browser agent can see it without running scripts. In browsers that support the experimental WebMCP API, every page also registers the same tools as our MCP server through document.modelContext.registerTool(), including check_agent_readability and get_access_solutions. Each tool returns a short text summary plus the JSON result. On the home page the person in front of the browser sees the same result.

WebMCP is experimental. Ordinary browser use, the HTTP API and the remote MCP endpoint work without it. There is no promise of support in every browser, an agent-directory listing or improved search ranking. Chrome’s WebMCP documentation. Our own explanation with examples and a free check: WebMCP vs MCP: what it is and how to check your site.

Websites blocked in a country

Some websites block visitors from certain countries (geoblocking). Our guide for blocked websites, the endpoint /api/v1/access-solutions and the MCP tool get_access_solutions give people and agents the same answer: free solutions first, then VPN providers, but only when a regional block is suspected. The order comes from documented criteria and never from commission. Paid links are marked “Ad” and carry a disclosure; agents get the same labels and are asked to show them.

Limits

What the quick check measures

Every requested destination is resolved and all returned addresses must be public. We connect to the verified numeric address while checking the HTTPS certificate for the hostname. Up to five safe HTTPS redirects are followed within one ten-second measurement budget. HEAD requests can fall back to GET response headers for 403, 404, 405 or 501 responses. The quick response check does not inspect page contents. Sensitive query strings are omitted from exposed URLs.

Public evidence, private analytics

Utility observations, the score method and the error library are public. Agent access never returns visitor analytics, session recordings, input history or access data. The project cockpit and improvement proposals are not available on this public service. See the privacy details.

Helping agents use your own website

A public response, readable content and callable browser actions are separate questions. Practical steps for website owners cover clear text and controls, search versus training bots, hosting protection, optional llms.txt and experimental WebMCP. None of this is a promise of search inclusion or improved ranking.