GEOFast
Sign inFree X-rayRun a free X-ray

Applebot, Apple's crawler for Siri, Spotlight and Safari

Applebot is Apple's web crawler. Apple says its data powers search in Spotlight, Siri and Safari and may also help train Apple's foundation models. A second token, Applebot-Extended, opts a site out of that training without leaving search.

Check your site for Applebot

Checked against Apple's own documentation on October 13, 2026.

The crawler

What Applebot is for.

Apple says the data crawled by Applebot powers features such as the search technology in Spotlight, Siri and Safari, and may also be used to help train the Apple foundation models behind its generative AI features.[1]

  • Run by

    Apple

  • Its job

    Search index. It crawls on its own schedule, to find pages for search results.

  • Its name in robots.txt

    Applebot

  • Does it follow robots.txt?

    Yes, by its operator's account. Apple says Applebot respects standard robots.txt directives that are targeted at Applebot. If a file doesn't mention Applebot but mentions Googlebot, it follows the Googlebot instructions. It does not follow crawl-delay.[1]

  • Its IP ranges

    Apple publishes Applebot's IP ranges as a JSON file of CIDR prefixes, and says its traffic is generally identified by reverse DNS in the applebot.apple.com domain.[1] search.developer.apple.com/applebot.json

  • Applebot-Extended covers training

    Apple says Applebot-Extended does not crawl webpages. It only decides how Apple may use the data Applebot crawled: disallowing it opts a site out of training Apple's foundation models.[1]

  • nosnippet covers AI answers

    Apple says publishers can opt out of their content being used in broad world knowledge answers in Siri and Search by applying the nosnippet meta tag.[1]

  • It may render pages

    Apple says Applebot may render a site's content within a browser, and may not render it properly if robots.txt blocks the JavaScript, CSS and other resources the page needs.[1]

User agent

The Applebot user agent strings.

Apple gives a general format and these two examples, and says Applebot occasionally updates the browser version it advertises while staying in that format.[1]

  • Apple's desktop example

    Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)

  • Apple's mobile example

    Mozilla/5.0 (iPhone; CPU iPhone OS 17_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Mobile/15E148 Safari/604.1 (Applebot/0.1; +http://www.apple.com/go/applebot)
robots.txt

The lines that allow and block Applebot.

Each pair of lines is a group of its own in robots.txt. The robots.txt generator writes the whole file, and the robots.txt checker reads what yours says to Applebot now.

  • Allow Applebot

    With no robots.txt, or one that doesn't disallow it, Applebot is allowed already. These lines say so by name: a crawler that finds a group with its own name follows that group, and follows the User-agent: * group only when it finds none.[2]

    User-agent: Applebot
Allow: /

  • Block Applebot

    Apple says enabling Applebot in robots.txt allows website content to appear in search results for Apple users in Spotlight, Siri and Safari, so a block gives that up. To opt out of training alone, Apple gives Applebot-Extended, and says pages that disallow it can still be included in search results.[1]

    User-agent: Applebot
Disallow: /
Our data

How many cited sites block Applebot.

We read the robots.txt of the 1,605 sites our recorded AI answers cited, for the 13 crawlers the X-ray reads for.

0.7%

of cited sites close their homepage to Applebot in robots.txt: 10 of 1,529.

Cited sites whose robots.txt closes the homepage to each crawler
CCBot4.2%
GPTBot3.1%
ClaudeBot2.6%
Google-Extended2.0%
ChatGPT-User1.3%
PerplexityBot1.2%
Perplexity-User1.0%
Claude-User1.0%
OAI-SearchBot0.9%
Claude-SearchBot0.8%
Applebot0.7%
Googlebot0.1%
Bingbot0.1%

Blue: Applebot. Gray: the other 12 crawlers the X-ray reads for.

We took the 1,605 sites cited in 1,033 answers from ChatGPT, Claude, Perplexity, Gemini and Google's AI Overviews and AI Mode, recorded on October 6 and 7, 2026 for 54 questions about AI visibility tools and web analytics, and read each site's robots.txt as GEOFastBot with the X-ray's own rules on October 8, 2026. These are sites AI answers already cite, so sites that shut AI search crawlers out may be rarer here than on the web at large, and a site can serve other crawlers other rules than it served us. Shares are of the 1,529 of 1,605 sites whose robots.txt we could read, or that have none; 62 refused our request.
FAQ

Questions about Applebot.

  • What is Applebot?

    Applebot is Apple's web crawler. Apple says its data powers the search technology in Spotlight, Siri and Safari, and may also be used to help train Apple's foundation models.

  • What is the difference between Applebot and Applebot-Extended?

    Applebot crawls. Applebot-Extended is a second robots.txt name that, by Apple's account, does not crawl webpages: it only decides whether the data Applebot crawled may train Apple's foundation models. Apple says pages that disallow Applebot-Extended can still be included in search results.

  • What is the Applebot user agent string?

    Apple's desktop example is: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot). Apple says the browser version it advertises changes from time to time.

  • Does Applebot follow rules written for Googlebot?

    Apple says that if robots.txt doesn't mention Applebot but mentions Googlebot, Applebot follows the Googlebot instructions. It respects standard robots.txt directives targeted at Applebot, and does not follow crawl-delay.

  • How many sites block Applebot?

    In our study, 10 of 1,529 sites (0.7%) close their homepage to Applebot in robots.txt. The base is the sites cited in AI answers about AI visibility tools and web analytics whose robots.txt we could read, or that have none, so other fields may differ. What the cited sites allow

Sources

Where each fact comes from.

Each numbered source is the page a fact comes from, most of them Apple's own documentation, read on October 13, 2026. The answers above repeat facts marked on the page. Vendors change these pages, so check them before you act.

  1. 1Apple: About Applebotsupport.apple.com/en-us/119829
  2. 2IETF: RFC 9309: Robots Exclusion Protocolrfc-editor.org/rfc/rfc9309.html

Keep reading

Free X-ray

Find out what AI says about your site.

Paste your URL and leave with a ranked list of what to fix, with steps and code.

  • Free
  • No account