Published 2026-09-09 · Last updated 2026-09-09
Which AI is reading your website right now? Almost always the same one. Promptwatch, which counts crawler visits from raw server and CDN logs and keeps only the requests it can verify by IP, measured OpenAI's crawlers at 79.8% of all AI crawler requests in the week of August 31 to September 6, 2026. That is down from 94.8% in early June, so the field is opening up, but one company still does most of the reading. The practical point is not the number. It is that your website now has a second reader, a machine, and its job is to be found and quoted by that machine, not only to look good and rank on Google.
What is an AI crawler, and why does it matter who is doing the reading?
An AI crawler is a bot that reads web pages so an AI can answer a question using them. OpenAI runs three. GPTBot reads pages to train the model. OAI-SearchBot reads pages so ChatGPT can cite them in a search answer. ChatGPT-User fetches a page live when someone's question sends ChatGPT to it. Whichever company's crawlers do most of the reading is the company whose answers decide whether your business gets named. When one crawler family reads roughly four out of five pages, being readable to that one is most of the game.
How much of the reading does one company do?
A lot, and it is measured, not guessed. Promptwatch tracks OpenAI, Anthropic, Google, Perplexity, and Mistral, counting only requests whose source IP matches each crawler operator's published ranges, so spoofed bots are thrown out. It reports the result as a share of verified requests, which means a rising slice is a larger portion of the reading, not always more visits. OpenAI moving from 94.8% of that mix in June to 79.8% in September tells you the others are starting to read too. It does not tell you OpenAI stopped mattering. It tells you the one crawler you cannot afford to be invisible to is still OpenAI's.
Other measures point the same way. Cloudflare, which sees a large share of web traffic, reported in 2025 that OpenAI's live-fetch bot, ChatGPT-User, was responsible for nearly three quarters of user-triggered crawler requests. Promptwatch's advice to site owners is blunt: a block that felt harmless a year ago now removes you from the providers that dominate the mix. The reading moved. A lot of websites did not.
| OpenAI crawler | What it reads for | What it decides for your business |
|---|---|---|
| GPTBot | Training the model on web pages | Whether your words help shape what the model knows. A separate setting you are allowed to decline. |
| OAI-SearchBot | Finding pages to cite in ChatGPT search | Whether ChatGPT can show your business in a search answer at all. |
| ChatGPT-User | Fetching a page live when someone asks | Whether ChatGPT can read your page in the moment a customer is deciding. |
Why does this change what a website is for?
Because the search box is being replaced by an answer. Your next customer is not always scrolling ten blue links. They ask an AI, and the AI decides whether your business is part of the reply. The audience is already there: the Pew Research Center surveyed 5,119 US adults in February 2026 and found 44% use ChatGPT and about one in four use an AI chatbot every day. For fifteen years the job of a business website was to look good and rank on Google. The new job is to be found, understood, and quoted by AI. Those are not the same skill.
A site can look beautiful to a person and be almost invisible to the model that is now doing the recommending. The design speaks to the human who has to trust you in five seconds. The plain text on the page speaks to the crawler that has to quote you. A website now has to do both, and most were built for only the first.
What does it take to be found and quoted by an AI?
Three things, and none of them is expensive. Be reachable, be clear, and be consistent. The model can only quote a business it is allowed to read, can understand, and can trust the facts of.
- Be reachable. Check robots.txt and your CDN or firewall so the answer crawler, OAI-SearchBot, is allowed in, even if you block the training crawler. OpenAI's own documentation says a site opted out of OAI-SearchBot will not be shown in ChatGPT search answers, and a robots change takes about 24 hours to apply.
- Be clear. Give every service its own page and answer the real question in plain sentences, because that sentence is the text an AI repeats. A model cannot quote a slogan it cannot parse.
- Be consistent. Put the business name, the service, the towns you cover, and the phone number in the words on the page, not only in a logo or an image, and match what your profile and reviews say.
- Then check your work. Ask ChatGPT for your trade in your town and see whether it names you, and what it says. That is the same audit a customer runs before they call.
Does Google and your reviews still matter?
Yes. This sits on top of what already works, it does not replace it. Most searches now end without a click and AI summaries answer more of them every month, both covered elsewhere in this field guide, and yet the local map results still send most trade jobs and your reviews still feed the answers the AI writes. Being quoted by AI is a new lane next to the old ones, not a swap. The businesses that win it are the ones already keeping their profile, reviews, and service pages honest and current.
So the takeaway for a small business is not more traffic. It is to be the answer when one customer asks. We build for both readers at once: the person who has to trust you in five seconds, and the AI deciding whether to mention you at all.
Common questions
How do I know if AI can read my website?
Check your robots.txt, CDN, and firewall rules. The facts also have to be in the text of the page, not only inside an image or a logo, because that is what a crawler reads. If OAI-SearchBot is allowed and your service, town, and phone are written in plain words, the answer crawler can find and quote you.
I blocked AI crawlers to protect my content. Did I disappear from ChatGPT?
If you blocked OAI-SearchBot, OpenAI's documentation says your site will not be shown in ChatGPT search answers. Blocking GPTBot, the training crawler, does not have that effect on its own, because it is a separate setting. You can decline training and still allow the search crawler. A change takes about 24 hours to apply.
Is 79.8% the share of the whole internet?
No. It is OpenAI's share of verified AI crawler requests across the server logs Promptwatch tracks, for the week of August 31 to September 6, 2026. It shows OpenAI is the single largest reader by a wide margin. It does not mean OpenAI is 80% of all web traffic.
Does being quoted by AI replace SEO and Google reviews?
No. It is a new lane on top of them. An accurate profile and real reviews still feed the answers, and local map results still send most trade jobs. Treat it as one more place to be found, not a reason to drop the others.
What is the fastest thing I can do this week?
Allow OAI-SearchBot, give each service its own page written in plain sentences, and search your own trade and town in ChatGPT to see whether it names you. Those three steps take an afternoon and cost nothing.
