AEO/GEO reference
Quick-reference resources on AI bots, robots.txt and llms.txt — generated directly from our scanning engine, always up to date.
AI bots: the complete user-agent list
Every known AI crawler, what it's for, and how to manage it.
AI-friendly robots.txt template
A commented robots.txt file that explicitly allows every known AI bot.
Annotated llms.txt example
A real llms.txt file, section by section, with an explanation of every line.