Guides to AI visibility

The questions site owners ask about AI answers and the bots behind them, answered from each vendor's own docs, published studies and our study of the sites AI answers cite.

The guides

Pick the question you have.

Our data

What the sites AI cites do.

Each guide draws on our study of the 1,605 sites that ChatGPT, Claude, Perplexity and Google's AI cited in answers we recorded.

3.1%

of cited sites block GPTBot, OpenAI's training crawler; 0.9% block OAI-SearchBot, which ChatGPT search reads through.

37.6%

of their robots.txt files name an AI crawler, most of them to let it in.

62 of 1,605

sites refused our crawler's request for robots.txt, the one file written for crawlers.

87.0%

of the cited pages we measured show at least 99.5% of their main text in the HTML, readable without JavaScript.

  • What we read: The 1,605 sites cited in 1,033 answers from ChatGPT, Claude, Perplexity, Gemini and Google's AI Overviews and AI Mode, for 54 questions about AI visibility tools and web analytics, recorded on October 6 and 7, 2026.
  • How: Each site's robots.txt, read as GEOFastBot with the X-ray's own rules on October 8, 2026. Shares are of the 1,529 of 1,605 sites whose robots.txt we could read, or that have none; 62 refused our request. We also rendered up to 3 cited pages from 1,505 of the sites in Chrome; 2,022 pages rendered cleanly.
  • Limits: AI answers already cite these sites, so sites that shut AI search crawlers out may be rarer here than on the web at large. A site can also serve other crawlers other rules than it served us.
Free X-ray

Find out what AI says about your site.

Paste your URL and leave with a ranked list of what to fix, with steps and code.

  • Free
  • No account