August 4, 2026 · 20 Boulder buyer queries, 3 runs each, 60 live SERP calls · measured, not researched

---

The answer in one table

Your hypothesis was that Google's AI answer at the top of the page is what matters now, so we should test that and skip the rest.

For restaurants and med spas, an AI Overview did not appear once in 27 attempts. The local pack appeared in all 27. Those are the two verticals you just decided to lead with for consulting revenue.

So for the businesses you're about to email, testing AI Overview presence would return "absent" every time and tell you nothing. The map pack is the entire game there.

The pattern, and it's a clean one

AI Overviews show up on research queries. The local pack shows up on buying queries.

• "how much does it cost to live in boulder" → AI Overview, no map pack
• "what is the elevation of boulder colorado" → AI Overview, no map pack
• "things to do in boulder colorado" → AI Overview, no map pack
• "best med spa boulder co" → no AI Overview, map pack
• "botox boulder colorado" → no AI Overview, map pack
• "best pizza boulder co" → no AI Overview, map pack

Somebody deciding whether to do a thing gets an AI answer. Somebody deciding who to pay gets three map results.

The other thing this exposed

Look at the "first organic result" column from the run: on almost every commercial query it was position 4.

That means the top three things on the page are all local pack. A business's actual website is the fourth item down, below three competitors' map listings. For a local service business, ranking your website is a consolation prize; the map pack is the prize.

This is the strongest possible argument for the audit change the models recommended. All four ranked GBP health and review velocity as the top missing checks, and this measurement is why they were right.

Where your hypothesis IS correct

Things to do: 7 of 9 runs had an AI Overview, and the local pack showed in only 3. "moving to boulder colorado neighborhoods" returned an AI Overview 3 out of 3 times with no map pack at all.

That is boulderthingstodo.com, outdoorboulder.com and newtoboulder.com territory. So AI visibility matters a great deal, for your own directory sites, and barely at all for your restaurant and med spa clients. That split is worth building around:

• Client audits (restaurants, med spas, movers): GBP, reviews, map pack. Skip AI.
• Your own sites (things to do, relocation, outdoors): AI Overview citation is a primary channel, possibly the primary one.

Your instinct to run it three times was right

Of the five queries that produced an AI Overview, two rotated their citations across three identical back-to-back runs:

• "moving to boulder colorado neighborhoods" gave 3 distinct citation sets in 3 runs
• "how much does it cost to live in boulder" gave 3 distinct citation sets in 3 runs

Three returned the same citations every time. So a single check is a coin flip on the rotating ones: you could conclude a site is absent when it appears two times in three. Any AI visibility check needs at least three runs, and that's now built into the script.

Caveats worth stating

• Desktop, Boulder-localized, one point in time. Mobile can differ, and Google changes this constantly. Re-run before betting anything big on it.
• Absence of an AI Overview in the API response is strong evidence but not proof; these render asynchronously and the test explicitly requested them, which is why the informational controls correctly returned 6 out of 6.
• 20 queries is enough to see a pattern this stark. It is not enough to slice finely within a vertical.

What I'd change because of this

1. Drop AI Overview from the client audit for restaurants, med spas and movers. It would have been a check that always returns the same answer.
2. Add "position in the map pack for your money query" as the headline finding instead. It's the thing that decides their revenue and it's screenshot-able.
3. Keep AI visibility, but point it at your own sites. Track whether boulderthingstodo.com and newtoboulder.com get cited for the queries above, three runs each, monthly.

Reproduce it

py -3.13 scripts/test_ai_overview.py --runs 3

60 DataForSEO calls, a few cents, no LLM involved. Output at data/ai_overview_test.json with every run's surface list and citations.