How we measure AI answers
Last updated
AI answers change from one ask to the next. The same question rarely gets the same list of brands twice, so a single answer is an anecdote, not a measurement. This page says how we collect answers, how we count them, and how sure each number is.
Where the questions come from
- From your site. When you add a project we read your site (the X-ray) and draft the questions buyers ask AI assistants, grouped by topic and by kind: "best for", alternatives, comparisons, problems, learning, and a few about you by name.
- Edited by you. You switch off, rewrite or add questions before anything is asked, and any time after.
- Not a prompt volume. We don't claim how many people ask each question. Where we show search demand, it is Google search data, labeled as such.
How answers are collected
- ChatGPT and Gemini: captured from the apps themselves, signed out, in the project's country, through a data provider. Each answer comes with the pages it cited, and for ChatGPT the searches it ran.
- Microsoft Copilot: not asked. Its app has needed sign-in since August 2026, so nobody can read it as a signed-out visitor would. Where Copilot cited your site comes from Bing Webmaster Tools' AI Performance export, which you upload on the Search Console page.
- Google's AI Overviews and AI Mode: read from Google's results for the question, as a searcher in that country sees them, through the same provider. A search that shows no AI Overview is recorded as such: nobody is named in it.
- Perplexity and Claude: asked through their APIs with web search switched on, because our data provider doesn't capture their apps. API answers can differ from what the apps show, so this page, onboarding and shared reports say which method each engine uses. Both are asked once a week on every plan, Claude of a team's first 5 to 25 prompts, by plan. Between those runs, a run's figures include their latest answers from the past week, so runs compare on the same engines.
- The free X-ray's "Ask the AIs": a small sample (18 answers), shown as such: ChatGPT and Gemini read from their apps signed out, as tracking reads them, and Perplexity through its API. A capture that fails is asked through the engine's API instead. It's exploratory, not a measurement.
How each answer is read
A language model reads every answer once and lists the products it offers as options, in order; whether it names you; whether the company it names is you or another company with the same name; how it speaks of you (positive, neutral or negative); and how it describes you in a few words. Answers and those readings are stored, so when you add a competitor or another spelling of your name, the whole history is matched against it again.
How we count
- Mentioned: answers naming you ÷ answers that came back. Failed calls, and answers the model couldn't read, are left out of every rate and shown as "didn't answer", never as "not named".
- First pick: answers where you're the first option ÷ answers. We don't show an average rank: answers rarely order brands the same way twice.
- Share of voice: your mentions ÷ mentions of you and the competitors you track, with a range over those mentions.
- Cited: answers whose sources include a page on your site ÷ answers.
- AI visibility (0–100): the average of each answer's weight: 1 when you're the first option, 0.8 second or third, 0.6 further down, 0.5 when named outside the options, 0 when not named.
How sure each number is
- Mentions and first picks show how many answers they rest on and a 90% range (a Wilson interval): where the true rate likely sits. Citations show their count.
- A change between runs counts only when the two ranges don't overlap. Smaller moves are shown in gray as within the usual swing.
- An action you mark done "earns" only when the answers after it clear the range from before. Until then it reads "no change yet" or "measuring".
- By the IAB's 2026 grades, the free 18-answer sample is exploratory. Weekly runs with their counts and ranges are part of what its decision grade asks for. It also asks for 50 or more questions across four intent types and results weighted by each platform's share, so we don't claim that grade.
What we don't do
- No industry averages or benchmarks we can't source.
- No ranks, and no percentage changes against too little data.
- No logged-in or personalized answers: everyone sees them differently, and we can't speak for that. You can still check whether ChatGPT mentions your brand by hand.