Free tool · live probe
Is your website readable for AI crawlers?
We analyze robots.txt AI bot rules, llms.txt, sitemaps, structured data, meta tags, content quality, and AI accessibility to return a crawlability score and recommendations.
Public sites only. 20 free scans / IP / hour
What we audit
A live diagnostic probe of AI search readiness: robots.txt bot access, llms.txt summaries, sitemap discovery, structured data, meta tags, content quality, and AI reader proxy accessibility. You get a crawlability score out of 100 and actionable fixes.
Ensures AI bots like GPTBot, PerplexityBot, & ClaudeBot aren't blocked.
Verifies existence of structured markdown summary for LLM context.
Probes sitemap existence to ensure AI bots can discover all URLs.
Checks for machine-readable entity schemas & structured data.
Evaluates title, description, and OpenGraph metadata presence.
Validates text length and H1/H2/H3 structure for AI parsing.
Tests if headless AI crawlers can extract text (Jina proxy).
Other free tools: llms.txt generator · Agent Readiness Scanner
Frequently asked questions
How AI crawlability works, what the score means, and how to improve your visibility.