Free AI Bot Checker Which AI crawlers can read your site?
Enter your site. We read your robots.txt the way the crawlers of ChatGPT, Gemini, Perplexity and about forty others do, then show you who gets in, who is kept out, and the exact line that decides.
Why check which AI crawlers your robots.txt lets in
Before an assistant like ChatGPT, Gemini or Perplexity can quote your pages, its crawlers have to be allowed to read them. Your robots.txt decides that, one crawler at a time, and a single line written years ago for another purpose can keep an AI search crawler out without anyone noticing.
This checker reads your file the way the standard (RFC 9309) says crawlers must: it finds the group that names each crawler or falls back to User-agent: *, picks the longest matching rule, and lets Allow win a tie. For each of the 40+ AI crawlers we track, you see the verdict, the role of the crawler and the exact line that decides. When something is blocked, you get lines to add at the end of your file, without touching the rest.
Frequently asked questions
Does blocking GPTBot remove my site from ChatGPT?
Not on its own. GPTBot collects pages to train OpenAI's models. ChatGPT's search uses OAI-SearchBot, and ChatGPT-User opens a page live when someone asks about it. You can block training and keep search: that's what the "Block training, keep search" option writes for you.
Why does Google-Extended never appear in my logs?
Because it's only a name for robots.txt, never a visitor. It tells Google whether Gemini may use your pages for training and to back up its answers. The pages themselves are still fetched by Googlebot. Blocking Google-Extended does not affect Google Search.
Which rule wins when several lines match?
The one with the longest path pattern. If an Allow and a Disallow are just as long, Allow wins. A crawler named in its own group ignores the User-agent: * group entirely. That's the behavior the robots.txt standard (RFC 9309) asks for, and the one this checker follows.
My robots.txt allows everything. Why can't AI crawlers read my site?
robots.txt is only a request. A firewall, a CDN rule or an anti-bot protection can still turn crawlers away. A robots.txt that answers with a server error also makes crawlers treat the whole site as off limits.
Is it free, and do you keep anything?
Yes, it's free with no signup. We read your public robots.txt when you ask, show you the result and keep nothing.