tasteboulder.com is a directory of every restaurant in Boulder, Colorado. 747 pages, 644 venues, and the part no competing guide has: 31,313 real menu prices, searchable, with a page per dish listing every restaurant that serves it, cheapest first.

Total data cost to build it: $7.63 in metered API spend, plus one paid scraper run (below). Every figure in this piece is a single snapshot taken on 2026-08-02.

This is the honest version, written for developers. It includes the mistakes, because the mistakes are most of what is worth knowing.

Start here: is the city worth building?

The build is cheap. That is exactly why the discipline has to come first. You can produce 747 pages for under ten dollars, which means the constraint is never "can we build it", it is "will anyone ever find it, and will it pay for itself".

Answer both before writing a line of code.

Check the keyword competition first, city by city

Every city looks like a good idea until you look at who already ranks. Pull real search volume and difficulty for the head terms and, more importantly, look at who actually holds page one. Boulder's numbers:

The volume was not the reason to build. The reason was the shape of page one for best restaurants boulder: a travel blog, the tourism board, a Reddit thread, and two individual restaurants. No dedicated Boulder restaurant guide ranked at all. That absence is the opening. If a well-funded local guide already owns those results, pick a different city, because you are not going to outrank an established directory with a new domain.

Two more checks that changed the plan:

• Authority gap. The site had 12 referring domains, all nofollow, so zero dofollow. The weakest competitor on page one had 134. That says: concede the head term for year one, target the softer "best" cluster, and plan for 25 to 40 real linking domains rather than pretending the content alone will do it.
• AI answers. Seven query variants were tested to see whether an AI Overview appeared. It appeared on exactly one, so the strategy was not built on being cited by an AI engine. Test it rather than assuming it, in either direction.

The general rule: a city qualifies when the search demand is real, the intent is commercial, and nobody has built the obvious thing yet. Two out of three is not enough.

How a site like this makes money

Traffic is not a business model, so decide the model before the build, because it changes what you collect. All four of these are open here:

1. Featured placement. A restaurant pays to sit at the top of a category, neighborhood or dish page. This is why the venue schema carries a slot component from day one rather than being retrofitted. Unsold slots must render an honest "this slot is open" notice, never scaffolding text.
2. Selling SEO services to the restaurants themselves. This is the strongest one, and it comes free with the data. The crawl scored all 769 businesses on what their own web presence is missing, and the counts are the pitch:

That scorecard is a ranked prospect list with a documented, specific reason to call, and it cost nothing extra to produce: it is derived from the same site crawl that found the menus.

One thing it does not contain, despite being the obvious thing to want: a mobile performance or mobile-friendliness verdict. No Lighthouse or PageSpeed run was ever done across the venues. The nearest signal held is whether a site declares a viewport meta tag (149 do not), which is a prerequisite for a usable mobile page rather than proof of a broken one. If you are going to sell on "your site is bad on phones", measure it properly first, because a viewport tag is not evidence. 3. Advertising against high-value adjacent terms. Restaurant terms are cheap per click, but catering carries an $8.69 CPC in Boulder, roughly ten times the restaurant terms, and page one for it is all local operators. That is a far better advertiser pool than the restaurants themselves. 4. Affiliate and booking revenue, which is the weakest of the four for an independent local guide and should not carry the plan.

The practical consequence: the SEO gap scorecard and the catering research are outputs of the same pipeline that builds the site. If you are going to monetize, collect for that from the first crawl rather than going back for it.

What the data actually costs

$7.63 in metered API spend for the whole city, broken down by stage further down, plus a paid scraper run that bills on its own credits. Hosting is a static site on Cloudflare Pages. Keeping a city current costs roughly fifty cents a month for a listings refresh, which is also how newly opened restaurants get found.

What it actually is

A static site. Astro 5, Cloudflare Pages, two npm dependencies (astro and @astrojs/sitemap). No React, no Tailwind, no CMS, no server. One piece of client-side JavaScript: the dish search box.

Behind it sits a SQLite database and about 5,800 lines of Python across 23 scripts that scrape, classify, verify and export. The site build reads three JSON files. That separation matters: the scraping is slow, expensive and failure-prone, and the site build is fast, free and deterministic.

Google business listings ─┐
restaurant websites ──────┼──► SQLite ──► export scripts ──► JSON ──► Astro ──► Cloudflare Pages
menus (HTML/PDF/image) ───┘                                    │
                                                      photos ──┘

The data pipeline

Every stage is checkpointed in a scrape_log(stage, key, status, detail, cost_usd) table and skips work already marked done. This is not polish. These runs take hours, sessions die, credits run out mid-run, and a pipeline that cannot resume means paying twice for everything.

Data comes from DataForSEO for anything Google-shaped, and OpenRouter for the LLM work (deepseek-v4-flash for text, a cheap Qwen vision model for images).

There is a live balance guard with a hard floor that stops the pipeline, checked against the account balance rather than our own bookkeeping, because our own bookkeeping drifts.

The eight things worth knowing

1. The menu was already in the listing, and we never read it

This is the big one.

The Google business listing returned by the listings endpoint contains a services block. That block is the merchant's own menu, with prices. The first version of the pipeline stored it, because it stored the whole record, and then never looked at it.

For weeks the entire site was built from menus crawled off restaurant websites, which is the hard way, while a second copy sat unread in the database:

The gap fell hardest on exactly the restaurants worth having, because a good restaurant is more likely to publish a PDF menu or render its menu in JavaScript, and a crawler gets a navigation shell from both:

The dish pages topped out at $99 while the real top of Boulder's steak market was $175. Reading a field we already had took the site from 12,271 dishes at 220 venues to 29,327 at 391.

Check what your data source already gives you before you build a crawler.

2. A missing category does not make a listing thin, it deletes the business

Discovery filters by Google category. The first list had 70 category slugs, cuisine by cuisine, and looked thorough. It had no spanish_restaurant.

Corrida is a Spanish-inspired steakhouse in the middle of Boulder, 754 reviews, a $150 wagyu New York strip on the menu. It was not in the dataset. Not incomplete: absent. So was Rincon Argentino, 2,509 reviews.

Nobody noticed. The site listed 630 restaurants and looked complete. It surfaced only when the owner said "I know there are expensive steaks in this town".

A read-only probe with the excluded categories cost 5.6 cents and found six missing venues. Widening the list permanently cost 10.5 cents.

One trap in the fix: discovery is batched ten categories per request and checkpointed by batch index. Appending is safe; inserting or reordering changes what batch03 contains while the log still says batch03 is done, which silently re-pays for old work or skips the new categories. The list is marked append-only in a comment for that reason.

3. Restaurants name the cut, not the category

The dish pages match menu item names to dish types with word-bounded patterns. The steak page matched the literal word "steak".

A steakhouse never writes that word on its most expensive beef. It writes ribeye, New York strip, filet mignon, porterhouse, tomahawk. So the page filled up with steak tacos and steak frites while a $95 ribeye four blocks away was invisible.

Broadening to the named cuts took the page from 49 restaurants to 106 and the ceiling from about $40 to $175.

The traps in broadening are local. In Boulder, "Flatiron" is a mountain: the data contains a Flatiron Spritz, a Flatirons Salmon and a VEGAN FLATIRONS, so the flat iron steak pattern had to be spelled out rather than matched loosely. "Filet" is a beef cut and also a salmon filet. Most wagyu on Boulder menus is sashimi, bao and rolls rather than steak. Every city will have its own version of the Flatiron problem.

The method that works: query your own data for expensive items in a category that your pattern does not match. That query tells you what menus actually say. Guessing at synonyms does not.

4. Unfiltered menu data publishes absurd numbers

An extractor stores the first money-looking number it finds, and on plenty of menus that number is not a price. Each of these came from a real row that would have shipped:

Two rules came out of this. If a number would look absurd to a local, it is wrong. And when the true value is a guess, suppress the row rather than inventing a price: the lost-decimal rows are dropped, not divided by ten.

Exemptions have to be surgical. Wine really does cost $480 a bottle and a tomahawk for two really does cost $175, so a flat implausibility cap deletes real data. The cap exempts wine varietals, vintages and named steakhouse cuts by name instead of being raised.

Before publishing the $175 tomahawk, it was checked against the restaurant's own menu PDF. Cowboy rib-eye $99, porterhouse $140, tomahawk $175. Exact.

5. The identity check is the difference between a directory and a liability

Image search returns photos of other businesses constantly, and it is worst for the venues that are hardest to photograph: similar names, shared buildings, food halls.

Two rules run before any photo is published:

1. Uniqueness. Any image claimed by two or more venues is dropped from all of them. In one pass this removed 1,863 images; in a later pass, 2,421 venue-photo pairs across 813 images.
2. A vision check on signage. Pearl Poké's hero showed a sign reading "AMIRA ORIG LION'S SHARE". The Gyro Stand's read "KARLS KAFE". La Colombe's read "PEAKS". 58 caught in the first pass, at roughly two hundredths of a cent per image.

The site ships with 69 venues showing no photo at all. That is the guard working, not failing. A venue with no photo is acceptable; a venue with someone else's photo is a factual error on a page about a named business.

One subtle bug: photo files were originally keyed by position (slug-0, slug-1), so when a photo was rejected the next one inherited its filename and the stale file kept being served, resurrecting images the check had already thrown out. Files are now keyed by a hash of the source URL.

6. Performance came from measuring, not from vibes

• Fonts loaded from Google reflowed every page. CLS was 0.231 against a 0.1 threshold. Self-hosting them with font-display: swap and metric-matched fallback faces (ascent-override, descent-override and size-adjust computed from the real font files with fontTools rather than guessed) took CLS to 0 and the performance score from 86 to 98.
• Thumbnails were 400px files rendered into 52px table cells, about 11 MB of waste on the directory page. Adding a 112px size took table thumbnails from 44 MB to 4.3 MB.
• Hero images capped width only. A portrait source became a 1336x2250 file weighing 726KB to fill a banner that renders at most 460px tall, so the browser threw away roughly 80% of every pixel via object-fit: cover. Centre-cropping before encoding gives the identical visible image at half the bytes: 32.9 MB down to 15.7 MB.
• The 1.3 MB dish search index is fetched on the first keystroke, never imported. A visitor who does not search downloads none of it.

7. Two bugs that wasted real money and time

A reasoning model spent its budget reasoning. Menu extraction used a model with reasoning enabled by default. Reasoning consumed 2,779 of a 3,000 token budget and the JSON came back truncated mid-object, so menus recorded as "empty" had simply been cut off. Re-running the 957 affected menus added 11,976 dishes. They were never empty.

An exception escaped a worker inside a thread pool. A non-dict entry raised out of the worker, escaped the ThreadPoolExecutor context, and made __exit__ block on every remaining future while the process stayed alive issuing paid API calls and recording zero completions. Every mysterious "stall" across two sessions was this one bug. Catch inside the worker, always.

8. When your own crawler loses, a paid scraper is cheaper than cleverness

246 menu URLs defeated both fetchers: plain HTTP got a 403, and a headless browser still could not read the JavaScript ordering platforms. The first plan was to reverse-engineer Toast's internal JSON API. Two independent reviewers killed that idea: their terms prohibit it, their schema changes without notice, and you end up maintaining an unofficial SDK for a company that does not want you to have one.

Running the same URLs through a commercial extraction service instead recovered 115 of 246, including the Toast pages, and 78 restaurants got their own menu read for the first time, among them three of the best restaurants in town. Extraction of the recovered pages cost nine cents.

Three things that made it work, all of which cost something to learn:

1. Do not set a relevance query. Asking the extractor for "menu items with prices" reranks the page and returns only the top-scoring chunks. One restaurant came back as 1,861 characters with 13 prices; with no query at all, 48,882 characters and 126 prices. A relevance filter is the wrong tool when the whole page is the thing you want. Measure this on one URL before running hundreds.
2. Check that a recovery can actually be saved. The menus table is unique on venue plus URL and the retry code used INSERT OR IGNORE, so a recovered menu would have been silently discarded and the failed row left untouched. The run would have reported success and changed nothing. Update the existing row; do not insert a new one.
3. Know the ceiling before you start. 67 of the 131 remaining failures are plain 404s: the menu page no longer exists at that address. No scraper fixes a dead URL. That is a discovery problem, and counting it as a scraping failure makes the job look bigger and more winnable than it is.

The honest caveat: this points a commercial scraper at ordering platforms whose terms restrict automated access. It is a lower-risk path than reverse engineering their APIs, but it is not zero risk, and it is a decision to make deliberately rather than by drifting into it.

Things that are deliberately not done

• No aggregateRating markup. Google's structured data policy forbids marking up ratings you did not collect yourself. Ratings appear on the page, attributed, linked to the source listing, and outside the schema block.
• Neighborhoods are assigned from coordinates, and left null when uncertain. An earlier version keyed on street names and put 115 venues on University Hill because the rule matched "Broadway", which runs the length of the city. A wrong neighborhood on a published page is a factual error, so only 327 of 645 venues carry one.
• The homepage hero is a plain gradient. Every photo in the dataset belongs to one specific restaurant, so using one sitewide would both imply a featured venue nobody chose and raise a licensing question.
• The site has not been submitted to any search engine. "The site is live" and "the site should be indexed" are two separate decisions. The IndexNow script refuses to run without an explicit owner-approval flag.

The failure that reached real visitors

Worth including because it is the most instructive one.

Three roundup pages shipped with 63 [VERIFY: ...] placeholder strings and 7 [CLIENT SLOT: ...] strings, plus a visible TODO: real hero photo on the homepage. They were live to real visitors for four days.

A handoff note had claimed every placeholder was gone. That was true of the homepage only. The roundups were built earlier, the editorial policy landed later, and nobody went back.

A placeholder that can deploy will deploy. Bracket text no longer goes in an array that renders; an unsold ad slot goes through a component that renders an honest "this slot is open" notice when its data is empty.

The same category of mistake happened again later, differently: the site exporter rebuilds the venue JSON from the database, and the database holds no photos (they are written in by two later scripts). Running the exporter last silently stripped every photo from the live site. The build succeeded. The pages simply had no images, and nothing caught it except loading a page and looking. The run order is now documented at the top of the exporter with a warning.

Reusing this

The work is packaged as five composable skills plus an orchestrator, so the same build can be run for another city:

The numbers