Why it matters
Most sites that score badly here do it by accident. A security setting meant for scrapers also turns away ChatGPT, Claude and Perplexity when a customer sends them to you.
AI crawlers allowed in robots.txt
10 points
What we check. Checks all 12 major AI crawlers. Blocking a live browsing agent such as ChatGPT-User or Claude-User is scored as critical, because that is a customer being turned away, not a training scraper.
How to fix it. Remove the Disallow rules for live browsing agents (ChatGPT-User, Claude-User, Perplexity-User, OAI-SearchBot) in robots.txt. They are customers mid-task, not training scrapers. Whether to allow training crawlers is your business decision.
Site serves a request from an agent
7 points
What we check. Every request we make says who it is (AgentReadyBot), the way ChatGPT, Claude and Perplexity agents do. A 403, 401 or challenge page means your bot protection turns those agents away. We never retry in disguise.
How to fix it. In Cloudflare, or whatever firewall sits in front of the site, let verified AI agents through to your public pages, or lower the security level on them. Nothing else matters while agents are refused at the door.
Secure and predictable address
4 points
What we check. HTTPS, and whether the final URL matches what was asked for.
How to fix it. Serve the site over HTTPS and redirect plain HTTP to it.
Sitemap an agent can follow
4 points
What we check. A sitemap.xml, and whether robots.txt points to it.
How to fix it. Generate a sitemap.xml and add a "Sitemap: https://yoursite.com/sitemap.xml" line to robots.txt.
See how your site does.
A free scan takes about a minute and shows the evidence for every check.
Other areas: Comprehend · Act · Complete · Delegate · Accurate · Locations · Commerce · How the score works