Skip to main content
Strategy & Growth

What AI crawlers actually do on medical practice and MedSpa websites

We counted every AI crawler request on a group of the practice websites we run. Here is who came, why, and what assistants read when a patient asks.

At a glance

Across a group of the medical practice and medical spa websites we run, AI crawlers made 24,100 requests in seven weeks. Nearly half came to collect training data, a third to index pages for AI search, and about one in seven was an assistant reading a page live because a person had just asked it something. Those live reads went mostly to condition and treatment pages. One request in seven claimed an AI company’s name from an address that company does not publish, and nine in ten of those asked for pages that do not exist.

Where does this data come from?

Every practice website built on the Urge Foundation™ platform keeps a daily tally of AI crawler visits: which crawler, which company runs it, why it came (training, search, or a live read for a person), which page it asked for, and whether the request really came from that company. It stores counts only. No visitor’s address, no cookie, nothing about a patient. The practice sees it on one screen in its dashboard, and we see the same totals across every site we run.

That makes this rare data. Most practices have no idea which AI companies read their website or what they read, and neither do most agencies. For this first look we took the client websites with the longest-running logs, added them together and named none of them. The window is 2026-09-28 to 2026-11-15, and some sites joined partway through, so this is an early sample. We will update it as it grows.

How much AI crawler traffic does a practice website get?

More than most physicians, practice managers and med spa owners expect. Together, these sites drew 24,100 AI crawler requests in about seven weeks. The first weeks on each site ran heavier, as crawlers made their first full pass.

These are floor counts. A page served from a content delivery network’s cache never reaches the website, so it never reaches the tally. Google’s own search crawler is also outside these numbers, because Google uses one crawler for both its search results and its AI answers.

Why are AI crawlers on a practice website?

They come for three different reasons, and only one of them is a patient asking a question right now.

  • Training: 11,493 requests (48%). Crawlers collecting pages that may be used to train future AI models. Meta’s crawler made the largest share.
  • AI search: 8,420 requests (35%). Crawlers building the index an AI search engine draws on when it answers, the way a search engine indexes pages before anyone searches. Perplexity, Amazon and OpenAI led here.
  • Live reads for a person: 3,685 requests (15%). An assistant opening a page at the moment someone asked it a question, so it can answer from the page itself. [1–3]
  • Ad checks: 502 requests (2%). Requests carrying the name of OpenAI’s ad-review crawler. More on these below, because they are not what they appear to be.
Why AI crawlers visited24,100AI crawler requests in seven weeks.requests, 2026-09-28 to 2026-11-15
Training11,493
AI search8,420
Live reads for a person3,685
Claimed ad checks502

Which AI companies crawl practice websites the most?

Meta made the most requests (5,599), almost all for training. Anthropic followed (4,264), also mostly for training. OpenAI (4,042) was the most balanced: 1,495 search requests, 1,427 live reads for its users, and 618 for training. Perplexity (3,973) and Amazon (2,642) came almost entirely to index pages for search. Common Crawl, DuckDuckGo’s assistant, You.com, Cohere and Mistral made up most of the rest.

AI crawler requests by company10AI companies sent crawlers. Meta sent the most, almost all for training.2026-09-28 to 2026-11-15
Meta5,599
Anthropic4,264
OpenAI4,042
Perplexity3,973
Amazon2,642
Common Crawl1,232
DuckDuckGo904
You.com823
Cohere535
Mistral70

The useful split for a practice is search and live reads versus training. Search and live reads are how a practice gets named in an answer. Training shapes what a model knows in general, months from now.

What do AI assistants read when a patient asks about a practice?

Condition pages, then treatment pages. When an assistant opened a page live for a person, it went to a condition page 314 times, a treatment page 266 times, the home page 189 times and a skin-concern page 110 times. Provider pages (45) and location pages (28) came next, along with an insurance-coverage page for skin cancer screening and a page of instructions for before and after Mohs surgery.

Pages assistants opened live for a person314live reads went to condition pages, more than any other kind of page.requests that reached a real page
Condition pages314
Treatment pages266
Home page189
Skin-concern pages110
Provider pages45
Location pages28

That is the closest honest signal we have of what patients ask AI about a practice: what is this condition, how is it treated, who is the doctor, and where are they. AI search crawlers showed the same priorities when they indexed these sites, starting with treatments (1,414), concerns (841) and conditions (673), then before-and-after galleries (421), providers (195) and locations (192).

For a practice, that sets the order of work. A condition or treatment page that answers the question plainly, names the provider and states the location is the page an assistant is reading on a patient’s behalf. For a MedSpa, that means a page for each treatment you offer, from injectables to laser, written the way a patient would ask about it.

Are all AI crawlers who they say they are?

No, and the difference is easy to measure. OpenAI, Anthropic, Perplexity and Google publish the internet addresses their crawlers use. [1–3] Urge Foundation downloads those lists every day and checks each request against them.

  • Requests from a company’s published addresses almost never asked for a missing page: 175 of 8,989, about 2%.
  • Requests that used an AI crawler’s name from an address the company does not publish asked for missing pages 89.8% of the time (2,968 of 3,306).
Share of requests that asked for a page that does not exist89.8%of requests from addresses an AI company does not publish asked for pages that do not exist.percent
From a company’s published addresses1.9%
From unpublished addresses89.8%
Using OpenAI’s ad-checker name95.4%

The clearest case is the ad checker. OpenAI says its ad-review crawler, OAI-AdsBot, only visits pages submitted as ads in ChatGPT. [1] None of these practices runs ChatGPT ads. Even so, 502 requests arrived under that name, every one from an address outside OpenAI’s published list, and 95% of them asked for pages that do not exist. We cannot say who sent them. We can say they could not be confirmed as OpenAI, and they behaved like scanners looking for weak spots.

The practical point: blocking or allowing crawlers by name alone is guesswork, because a name costs nothing to copy. Checking the address is what separates the crawler that might name your practice from the one probing it.

Do AI answers send patients to the website?

Some, and this is the number to watch next. When an AI answer links to a practice and a person clicks through, the visit arrives with the assistant named as the referring site. Urge Foundation began counting those visits in late October on some of these sites, and three have arrived so far. Three is small, and it is early. Live reads tell us assistants are already answering patients from these pages. Click-throughs will show how often a patient then visits the site.

What else did the crawlers use?

Urge Foundation publishes a plain-text Markdown copy of each page beside the normal web page, written for software rather than browsers. AI crawlers fetched those copies 1,437 times, 906 for training and 531 for search. The sites also offer assistant tools that let an AI agent look up treatments, providers, practice details and open appointment times directly, and Urge Foundation logs every call. Most calls so far came from our own testing, so we will report real use once there is enough of it to count.

What should a practice do with this?

  • Find out who reads your site. If your website cannot tell you which AI companies visit and what they read, you are guessing about the fastest-growing way patients research care.
  • Write condition and treatment pages for the question a patient asks. That is where live reads land. Name the provider and the location on the page, in plain text.
  • Keep search and live-read crawlers allowed. OpenAI states that a site blocking its search crawler will not appear in ChatGPT search answers. [1] Training crawlers are a separate choice you can make on its own.
  • Verify by address, not by name. A crawler’s name can be copied by anyone. Its published address cannot.

AI companies are already reading medical practice websites every day, and when a patient asks an assistant about a condition or a treatment, the assistant reads the practice’s own page to answer. Practices that can see this traffic can write for it. Every site built on Urge Foundation counts it from day one, with no personal data, and we will keep publishing what it shows.

About this data: the client websites that have kept the Urge Foundation platform’s AI Activity log the longest (dermatology, medical spa and aesthetics, pain medicine), 2026-09-28 to 2026-11-15; some joined in October. Combined, anonymous daily counts; no visitor data. Requests served from a CDN cache are not counted. Google’s shared search crawler is not counted. Assistant tool calls are reported without a count because most were internal tests.

Want to see which AI companies read your website?

Every site we build on the Urge Foundation platform counts AI crawler visits from day one. We can show you what yours would see.

Sources

Keep reading

More insights

Strategy & Growth

How to choose a med spa marketing agency: questions, red flags and the numbers to ask for

4 min read Read
Advertising

What a cataract or LASIK lead should cost in 2026, with real numbers

3 min read Read
Advertising

What a dermatology lead should cost in 2026, with real numbers

3 min read Read
Let's grow your business

Want this kind of thinking on your account?

Tell us where you're stuck and a strategist will map the fastest path forward.

Book a free consult