If your site ranks well on Google but never shows up in ChatGPT or Perplexity answers, the most common cause is technical, not content-related: your robots.txt is blocking AI bots.
Why this happens
Many WordPress themes, SEO plugins and CDN configurations add "cautious" robots.txt rules that block unknown crawlers by default, or were written before GPTBot, ClaudeBot and PerplexityBot existed. The result: Googlebot gets through, AI bots don't, and nobody notices until they check explicitly.
Bots worth checking
- GPTBot, OAI-SearchBot — OpenAI
- ClaudeBot, anthropic-ai — Anthropic
- PerplexityBot — Perplexity
- Google-Extended — Gemini/AI Overviews training (separate from Googlebot)
- Applebot-Extended — Apple Intelligence
Check it in 30 seconds
Use our free AI Crawler Access Checker: paste your domain, get the list of who's allowed and who's blocked, without having to read robots.txt by hand.
How to fix it
If you find blocked bots, a full AIVIS report generates a corrected robots.txt ready to publish, and verifies the fix worked with a re-scan.