Your robots.txt may say yes while your firewall says no. We request your page as ChatGPT, Perplexity and Claude, and show you exactly what they get.
Try ··
Most checkers only read robots.txt. We also send live requests, with control groups so a bot-verification rule isn't mistaken for a block.
Per-crawler verdict for 20+ AI agents, with search and training crawlers kept separate, so you can block training without vanishing from AI answers.
Live requests as OAI-SearchBot, PerplexityBot, Claude-SearchBot and more. We flag 403s and challenge pages from Cloudflare, CloudFront, Akamai and others.
noindex and nosnippet directives, and whether your content exists without JavaScript, which most AI crawlers don't run.
Our November 2026 sample of popular sites
reachable sites silently block AI search crawlers at the firewall, while robots.txt allows them and Googlebot gets through.
362 reachable sites from a random 600 of the Tranco top 20k. Requests sent from a data-center IP with crawler user agents and control groups.
Nothing in your CMS, sitemap or Search Console tells you. A default bot rule or a security plugin returns 403 to AI crawlers, and your pages quietly drop out of ChatGPT and Perplexity answers.
Daily re-checks and alerts when a CDN update, plugin or robots.txt edit cuts off AI crawlers. Multi-site reports for agencies.