Why doesn't ChatGPT show your website?
Usually one of six things, and the first four take minutes to check. Can OAI-SearchBot get in, through robots.txt and past your firewall? Are your pages indexable, and is their text in the HTML? Bing's index is worth a look, though OpenAI doesn't require it. If ChatGPT can read you and still leaves you out, the pages it reads name others.
Checked against each vendor's own docs on October 20, 2026.
1 and 2: can ChatGPT's crawler get in?
OpenAI gives site owners one step for ChatGPT search: let OAI-SearchBot crawl the site, and let its traffic through your host or CDN.[1] Check that first, since nothing after it matters until the crawler gets in.
1. robots.txt shuts OAI-SearchBot out
Sites opted out of OAI-SearchBot aren't shown in ChatGPT's search answers, though they can still appear as navigational links.[2] A group that names OAI-SearchBot with
Disallow: /does it, and so does a wildcard group that shuts every crawler out. The X-ray's check “AI search crawlers may crawl your site” reads your robots.txt for 13 AI crawlers on your homepage and up to 11 more pages; the robots.txt checker shows the lines that decide it.2. A firewall or CDN turns it away
OpenAI asks you to confirm that your host or CDN lets traffic from its published IP addresses through, and lists OAI-SearchBot's at openai.com/searchbot.json.[1, 2] A bot setting can refuse the crawler even when robots.txt lets it in. The check “AI crawlers get past your firewall” fetches your homepage as four AI crawlers, OAI-SearchBot and ChatGPT-User among them, and compares what they get with what a browser gets. Behind a firewall that verifies crawlers by IP, a refusal of our requests is a prompt to check its AI setting rather than a verdict. Try it with the AI crawler checker; the firewall guide says where each provider keeps the setting.
3 to 5: can it index and read your pages?
Past the door, two things keep a page out of what ChatGPT reads, and a third is worth a look.
3. noindex on the pages that matter
OpenAI's crawler docs don't mention noindex.[2] Bing says to use it when a URL should not appear in Bing search, Copilot or grounding results, and Google drops a noindexed page from its results.[3, 4] ChatGPT search sends its queries to partner search providers, so a page they leave out can't come back from them.[1] A setting left on from a staging site can add it to every page. The checks “Your homepage can be indexed” and “Important pages aren't hidden from search” read the robots meta tag and the X-Robots-Tag header, in the free GEO audit.
4. Text that only JavaScript writes
Vercel's 2024 study of crawler traffic found none of the major AI crawlers rendering JavaScript, OAI-SearchBot and ChatGPT-User among them.[5] A page whose text appears only after its scripts run is close to empty for them. The check “AI crawlers can read your pages without JavaScript” compares each page's HTML with a rendered copy, across up to 300 pages; the JavaScript rendering checker gives each page a verdict, and the rendering guide covers the fixes.
5. Missing from Bing's index
OpenAI's help page says ChatGPT search sometimes partners with other search providers and points to Microsoft's privacy statement; its page for business workspaces says their queries may go to Bing.[1, 6] No OpenAI page says a site must be in Bing's index, and OpenAI keeps an index of its own.[7] So it's worth a check, not a requirement. Bing's
url:keyword shows whether an address is in its index, and Bing asks for sitemaps and IndexNow to find new pages.[3, 8] The X-ray checks that robots.txt lets Bingbot in; it can't see Bing's index. Who ChatGPT draws on is its own guide.
What the sites AI answers cite get right.
Among sites AI answers already cite, these faults are rare. Finding one on your own site makes it a likely cause.
cited sites close their homepage to OAI-SearchBot in robots.txt (0.9%).
close it to ChatGPT-User, the agent that fetches a page someone asks ChatGPT about.
pages ChatGPT cited show over 40% of their main text only after JavaScript runs.
cited sites refused our crawler's request for robots.txt, so even sites AI cites turn some bots away.
We took the 1,605 sites cited in 1,033 answers from ChatGPT, Claude, Perplexity, Gemini and Google's AI Overviews and AI Mode, recorded on October 6 and 7, 2026 for 54 questions about AI visibility tools and web analytics, and read each site's robots.txt as GEOFastBot with the X-ray's own rules on October 8, 2026. These are sites AI answers already cite, so sites that shut AI search crawlers out may be rarer here than on the web at large, and a site can serve other crawlers other rules than it served us.
Shares are of the 1,529 of 1,605 sites whose robots.txt we could read, or that have none; 62 refused our request. The JavaScript figure counts the pages ChatGPT cited among the 2,022 that rendered cleanly; a page other engines cited too counts for each.
6: does anything it reads name you?
If ChatGPT can reach and read you, the last cause is the slowest to change.
6. The pages it reads name others
An answer recommends what its sources recommend. In a 2026 audit of product advice, 56.7% of the domains ChatGPT showed as sources were editorial and review sites, by labels its authors call exploratory.[9] In Ahrefs' study of 75,000 brands, how often the web mentions a brand tracked how often ChatGPT named it at 0.664, against 0.266 for Domain Rating; Ahrefs adds that correlation isn't causation.[10]
Or it answered without searching
ChatGPT searches when a question would benefit from current information.[1] Without a search, it names what its model learned in training, which a new site may not be part of.
How to check
When the X-ray's site checks end, it asks ChatGPT, Perplexity and Gemini three of your buyers' questions once each and lists who they named under “Named instead of you”: try the AI visibility checker. A handful of answers is a first look, and the guide to finding who ChatGPT recommends shows how to count enough of them.
Read the symptom, then run the check.
What you see when you ask ChatGPT yourself narrows the list. These point to a cause; the checks above confirm it.
Your site appears only as a bare link
That is how OpenAI says opted-out sites can still appear.[2] Start at cause 1.
It can't open a page you paste
ChatGPT fetches that page as ChatGPT-User, and robots.txt rules may not apply to those visits.[2] A refusal points to cause 2.
It describes you only from other sites
When answers quote reviews and directories about you but never your own pages, your pages may be closed or empty to its crawler: causes 1 to 4.
It knows you by name, never on buyer questions
ChatGPT can read you; the pages behind its recommendations name others. That is cause 6.
One answer left you out
Ask again first. In SparkToro's study the chance that ChatGPT gave the same list of brands twice was under 1 in 100.[11] The guide to checking ChatGPT by hand has the steps.
Questions about ChatGPT and your site.
Does ChatGPT need my website to be in Bing?
OpenAI doesn't say so. Its one stated step is to let OAI-SearchBot crawl your site and let its traffic through your host or CDN. OpenAI names Bing as a search provider for business workspaces and keeps an index of its own, so being in Bing's index is worth checking, not a documented requirement. Does ChatGPT use Google or Bing?
Does noindex keep a page out of ChatGPT?
OpenAI's crawler docs don't say. Bing says noindex keeps a URL out of its search, Copilot and grounding results, and Google drops a noindexed page from its results. Where ChatGPT search draws on a search partner, a noindexed page is missing from what comes back.
Why does ChatGPT show my site only as a link?
OpenAI says sites that opt out of OAI-SearchBot aren't shown in ChatGPT's search answers but can still appear as navigational links. Look in robots.txt for a group that names OAI-SearchBot, or a wildcard group that shuts every crawler out.
Why can't ChatGPT open my website when I paste the link?
ChatGPT fetches a page someone asks about as ChatGPT-User, and OpenAI says robots.txt rules may not apply to those visits. So a refusal points past robots.txt, to a firewall, a CDN's bot setting or a security plugin. Firewalls that block AI crawlers
Is my website too new for ChatGPT?
It can be. When ChatGPT answers without searching, it names what its model learned in training, which a new site may not be part of. It searches when a question would benefit from current information, and that is how a newer site reaches an answer.
Where each fact comes from.
Each numbered source is the page a fact comes from, most of them the vendor's own documentation, read on October 20, 2026. The answers above repeat facts marked in the guide. Vendors change these pages, so check them before you act.
- 1OpenAI: Searching the web with ChatGPThelp.openai.com/en/articles/9237897-searching-the-web-with-chatgpt
- 2OpenAI: Overview of OpenAI Crawlersdevelopers.openai.com/api/docs/bots
- 3Microsoft: Bing Webmaster Guidelinesbing.com/webmasters/help/webmaster-guidelines-30fba23a
- 4Google: Robots Meta Tags Specificationsdevelopers.google.com/search/docs/crawling-indexing/robots-meta-tag
- 5Vercel: The rise of the AI crawlervercel.com/blog/the-rise-of-the-ai-crawler
- 6OpenAI: ChatGPT search for Enterprise and Eduhelp.openai.com/en/articles/10093903-chatgpt-search-for-enterprise-and-edu
- 7OpenAI: Offline web search for ChatGPT workspaceshelp.openai.com/en/articles/20001203-offline-web-search-for-chatgpt-workspaces
- 8Microsoft: Advanced search keywordssupport.microsoft.com/en-us/bing/advanced-search-keywords
- 9Uberti-Bona Marin et al., arXiv: “If I Had to Buy Just ONE: Galaxy S26 Ultra”: Auditing AI-Generated Product Recommendationsarxiv.org/abs/2609.18729
- 10Ahrefs: Top Brand Visibility Factors in ChatGPT, AI Mode, and AI Overviews (75k Brands Studied)ahrefs.com/blog/ai-brand-visibility-correlations/
- 11SparkToro: NEW Research: AIs are highly inconsistent when recommending brands or products; marketers should take care when tracking AI visibilitysparktoro.com/blog/new-research-ais-are-highly-inconsistent-when-recommending-brands-or-products-marketers-should-take-care-when-tracking-ai-visibility/
Keep reading
- The free GEO auditA free GEO audit of up to 300 pages: AI crawler access, JavaScript rendering, 21 checks and fixes ranked by impact, with code. No sign-up.
- The free AI crawler checkerTest 13 AI crawlers, from GPTBot to PerplexityBot, against your robots.txt, and fetch your homepage as 4 of them. Free, no sign-up.
- The free JavaScript rendering checkerSee how much of your pages AI crawlers miss because it only appears after JavaScript runs. GEOFast renders your key pages and compares. Free, no sign-up.
- The free AI visibility checkerCheck your AI visibility for free: what AI crawlers can read on your site, then who ChatGPT, Perplexity and Gemini name for three of your buyers' questions.
- Actions that earn AI mentionsA ranked list of what gets you named: site fixes, pages to create, answers to improve and sites to get onto, then a check of whether each one moved the answers.
- AI visibility tools, comparedEvery tool we have checked, by starting price.
- What is GEOGEO, generative engine optimization, in plain words: how AI answers pick their sources, what research shows helps, and how to measure your AI visibility.
- AEO vs GEO vs SEOAEO vs GEO vs SEO: where each name came from, what Google and Microsoft say, the work all three share, and the four things that change with AI answers.
- Check if ChatGPT mentions your brandCheck by hand in five steps, see why one chat can't measure it (answers change, memory, location, leading questions), and where ChatGPT and Claude do help.
- Find the competitors ChatGPT recommendsA method for finding who ChatGPT recommends instead of you: which questions to ask, how many times, how to count the names and work out your share of voice.
- Does ChatGPT give everyone the same answer?No. ChatGPT writes each answer afresh and can use your memory, location, a web search and your plan's model. OpenAI's own pages, and our repeat-run numbers.
- AI visibility metricsMention rate, first-pick rate, citation rate, share of voice and sentiment: what each one counts, how to work it out, and why each needs a range.
- Get ChatGPT to recommend your businessWhat gets a business named in ChatGPT's answers: what you control, whether you can submit your business, how long changes take, and how to measure it honestly.
- Get cited by ChatGPTHow ChatGPT search finds and picks the pages it links, from OpenAI's docs, what makes a page usable as a source, and which kinds of pages our data shows cited.
- Does ChatGPT use Google or Bing?What OpenAI documents about where ChatGPT search gets results (Bing, its own crawler and index), what is only reported about Google, and what it means for you.
- Rank in Google's AI OverviewsWhat Google says you need to show up in AI Overviews, how query fan-out and nosnippet work, and what studies show: 38% of cited pages rank in the top 10.
- Rank in Google AI ModeWhat sets Google's AI Mode apart from AI Overviews: query fan-out, follow-ups, a custom Gemini and Search Console's counts, and how often the two cite one page.
- Perplexity SEOHow Perplexity finds, reads and cites pages: its two crawlers, what its docs say and leave out, the Cloudflare dispute, studies, and what our runs show.
- Which AI crawlers to allowWhich AI crawlers feed ChatGPT, Claude, Perplexity and Google's answers, which only train models, and what the 1,529 of 1,605 cited sites we could read allow.
- robots.txt for AI crawlersCopy-ready robots.txt rules that keep AI search crawlers in and training crawlers out, how each crawler reads the file, and the mistakes that lock AI out.
- What is llms.txt?llms.txt is a Markdown file that tells AI agents what a site holds. Its format, a real example, who reads it today, and what studies show about its effect.
- Firewalls that block AI crawlersCloudflare, Vercel, AWS WAF and others can turn AI crawlers away even when robots.txt lets them in. Where each setting lives and how to let crawlers through.
- Do AI crawlers render JavaScript?Google's and Bing's crawlers render JavaScript. OpenAI, Anthropic and Perplexity don't say, and Vercel's 2024 study found theirs don't. Our data on cited pages.
Find out what AI says about your site.
Paste your URL and leave with a ranked list of what to fix, with steps and code.
- Free
- No account