Let the right AI bots crawl you
Pick which AI crawlers can read your site and we’ll build a clean robots.txt. Block GPTBot and you can’t be quoted in ChatGPT — so choose carefully.
Powers ChatGPT’s live answers
OpenAI crawling & training
On-demand browsing in ChatGPT
Indexes pages for Perplexity
Live fetches for answers
Anthropic (Claude) crawler
Claude live browsing
Gemini & AI Overviews
Apple Intelligence
Amazon / Alexa AI
Feeds many open models
ByteDance / TikTok AI
Block all other crawlers
Sets User-agent: * to Disallow
# robots.txt — generated by Aeoix (aeoix.com/tools/ai-robots-generator)
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Perplexity-User
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: Claude-Web
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: Amazonbot
Allow: /
User-agent: CCBot
Disallow: /
User-agent: Bytespider
Disallow: /
# Standard search crawlers
User-agent: *
Allow: /
Save as robots.txt at your site root. Blocking a bot means that engine can’t quote you.
Why AI crawler access matters
Before you can rank in AI answers, the answer engines have to be allowed to read you.
Blocked means uncited
If GPTBot or ClaudeBot can’t crawl you, those engines literally cannot quote your pages. Access is step zero for AEO.
Citation bots vs. trainers
Some crawlers power live answers (ChatGPT Search, PerplexityBot); others only train models. We flag which are which.
One file, total control
Allow the engines that send you buyers, block the ones you don’t want, and ship a single clean robots.txt.
Find out if AI is recommending you
One domain, five minutes, and you'll know exactly where you stand. *No card required.
Free report · No credit card · Results in ~5 minutes
Free tools
More free AEO tools
No sign-up, no catch. Built to help you get found, quoted and recommended by AI.