# Modern Web Guidance: the seven-line loop that replaced a 23 MB transformer

> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.satyajit.si/articles/modern-web-guidance-static-embeddings
> date: 2026-11-13
> tags: retrieval, search, distillation, quantization, agent-skills, small-models, on-device, javascript

On 12 November the Chrome for Developers account posted a for loop with a claim on top of it: "From 507 MB and C++ dependencies to this 7-line loop: how Modern Web Guidance cut query embed time to 30 µs." The loop is this, verbatim from the post and from [`src/rag/static-embedder.ts`](https://github.com/GoogleChrome/modern-web-guidance-src/blob/main/src/rag/static-embedder.ts) in the repo:

```js
for (const [i, f] of enc) {
  const wt = scales[i] * f;
  const offset = i * 256;
  for (let k = 0; k < 256; k++) {
    out[k] += wt * table[offset + k];
  }
}
```

That is a whole sentence embedder. No attention, no layers, no tokenizer model. Each known word in the query contributes one row of a table, scaled, and the rows are added. Paul Irish's [technical write-up](https://developer.chrome.com/blog/mwg-technical-deep-dive) explains how they got there and why it works for this job. I read it, the [source repo](https://github.com/GoogleChrome/modern-web-guidance-src) and the published [npm tarball](https://www.npmjs.com/package/modern-web-guidance), re-implemented the embedder and the search in my own JavaScript from the shipped data files, and timed it. Then I ran the real package on a second machine, an Intel i7-14700, to check my copy against it. Most of the claim holds. A couple of the numbers around it need a footnote, and one part of the search does not work the way the post implies.

<Figure
  src="https://media.thesatyajit.com/ai/articles/modern-web-guidance-static-embeddings/post.jpg?v=6440b900a678f47d"
  alt="A line chart titled 'Modern Web Guidance: Search hit rate (Hit@K) across milestones', subtitled 'Held-out benchmark of 5,029 real coding-agent searches'. Points: Whole doc 78.2%, Chunked 84.0%, 0.38 gate 80.6% (in red), + BM25 86.1%, Top-10 compact 93.5%, Static Int8 95.4% (highlighted, '7-line loop (30 µs)'), and a dashed line to Gemini API 96.9% as the cloud reference."
  caption="The image on the post: Google's hit rate at each milestone, ending at 95.4% for the static Int8 embedder against 96.9% for calling the Gemini API (image from @ChromiumDev's post on X; Google's measurements on its internal telemetry set)."
/>

## What Modern Web Guidance is, and why its search box is odd

[Modern Web Guidance](https://developer.chrome.com/blog/modern-web-guidance-v1) 1.0 is an open-source agent skill: about 200 curated guides (the shipped index has 208) on doing things the modern way in HTML, CSS and web APIs, so that a coding agent writes `<dialog closedby="any">` instead of `window.confirm()`. The guides total 1.7 MB of Markdown, far too much to put in context. So the skill exposes a CLI. At rest the agent sees only the skill's description, about 205 tokens. On a frontend task it runs `modern-web-guidance search "animate height auto"` and gets ten `{id, description}` results, about 530 tokens; then it runs `retrieve` on one id to pull that guide, about 800 tokens. All of it runs locally through `npx`, offline, in whatever sandbox the agent lives in.

That makes the search workload unlike most retrieval:

- **The queries are tiny.** Google's telemetry says 86.7% of live agent searches are two to five words, median three.
- **The corpus is small and frozen.** About 200 guides, chunked into 2,016 pieces in the 1.0.0 tarball, fixed between npm releases.
- **Every search is a new process.** The agent shells out, Node starts, the index loads, one query runs, the process exits. Cold start is the latency the agent sees.
- **Install has to just work.** `npx` in a cloud sandbox, no compiler, no native binaries.

The first generation was a standard local RAG stack: LanceDB, Apache Arrow and Transformers.js on `onnxruntime-node`. The blog puts its extracted `node_modules` at 507 MB, with architecture-specific `.node` binaries and `node-gyp` builds that failed in sandboxes. The second generation ran `all-MiniLM-L6-v2` on pure-JS TensorFlow.js, pruned to the 26 kernels MiniLM uses, at 23 MB. It shipped and, by the blog's count, served about 750,000 searches. Then it failed in an instructive way.

<Figure
  src="https://media.thesatyajit.com/ai/articles/modern-web-guidance-static-embeddings/fig1.png?v=9566401c24f08e3b"
  alt="Two panels. Top: 'Outdated 2018 vocabulary', the word fetchpriority split by 2018 WordPiece into fetch, ##pr, ##ior, ##ity. Bottom: 'Strict 0.38 cutoff drops valid matches (returns [])': three queries with their best cosine scores, 'confirm destructive action' 0.349 to the dialog guide, 'popover anchor' 0.362 to popover, 'autofill address form' 0.354 to autofill, each crossed out at a dashed 0.38 cutoff line."
  caption="Why MiniLM missed: a 2018 WordPiece vocabulary shreds post-2020 API names, and a 0.38 cosine floor threw away right answers that scored 0.349 to 0.362 (Modern Web Guidance deep dive, Figure 1)."
/>

An agent asked to build a delete confirmation searched `confirm destructive action`, got `[]`, and wrote `window.confirm()`. Two causes, both in the figure: MiniLM's tokenizer predates `fetchpriority`, `popover` and `startViewTransition`, and a 0.38 similarity floor added to reject off-topic queries rejected that one too, at 0.349. Google fixed the floor and added BM25 first (80.6% to 94.7% Hit@K on held-out telemetry, by their numbers), then went after the embedder.

## A sentence embedding is a forward pass

A transformer sentence embedder like MiniLM turns the query into WordPiece tokens, looks up a 384-d vector for each, and then runs six layers in which every token attends to every other. Each layer rewrites each token's vector in light of its neighbours, so "light" next to "dismiss" ends up somewhere different from "light" next to "theme". The output vectors are mean-pooled into one.

The cost is in those layers. From MiniLM's `config.json` (hidden 384, six layers, FFN width 1536), one token through one layer costs $4 \cdot 384^2$ multiply-adds for the query, key, value and output projections plus $2 \cdot 384 \cdot 1536$ for the feed-forward block: 1,769,472. Six layers make 10,616,832 multiply-adds per token, before attention scores, biases and LayerNorm. A three-word query that becomes seven tokens with `[CLS]` and `[SEP]` costs about 74.5 million. That is what TensorFlow.js was grinding through, in JavaScript, at a warm 227 ms per query by Google's measurement, after a 770 ms cold start that parsed a 22.6 MB network.

## A static embedding is a lookup and a sum

Drop the layers and keep only the lookup. Give every word one fixed vector $E[i]$ and one weight, and embed a text as the weighted sum of its words' vectors, normalised:

$$
\mathbf{q} \;=\; \frac{\sum_{i \in \text{query}} s_i\, f_i\, E[i]}{\left\| \sum_{i} s_i\, f_i\, E[i] \right\|}
$$

Here $E[i]$ is word $i$'s row of the table, $s_i$ its weight, and $f_i$ a fraction that is 1 for a word found whole (more on it below). This is old. Averaging word2vec or GloVe vectors was the standard sentence baseline in the mid-2010s, and the SIF paper (Arora et al., 2017) showed that down-weighting frequent words and removing a shared direction made the average competitive with trained encoders on similarity tasks. What it cannot do is context: a word has one vector whatever surrounds it, and since addition commutes, word order is gone. "dog bites man" and "man bites dog" are the same point.

For a search box fed by coding agents, those losses cost little. A three-word query has almost no syntax to lose. "light dismiss" has no order that matters, and the guides being searched are about named features, so the words carry most of the meaning on their own. Context, the one thing the transformer adds, matters most for long, ambiguous text, which is exactly what agents do not send.

The demo below is a static embedder in two dimensions with a hand-made 34-word table, so the sum can be drawn. The arithmetic is the real one: each word's row is stored as Int8, its scale is its weight times the row's largest component over 127, and the contributions add head to tail before the sum is normalised onto the circle where six toy guides sit.

<StaticEmbedDemo />

<Video
  src="https://ai.satyajit.si/articles/modern-web-guidance-static-embeddings/demo-typing"
  poster="https://media.thesatyajit.com/ai/articles/modern-web-guidance-static-embeddings/demo-typing-poster.jpg?v=c1f41013111fd189"
  alt="A screen recording of the static-embedding demo above. A query is typed into the input one letter at a time: first 'light', whose single arrow points at the light-dark color-scheme guide, then ' dismiss', whose longer arrow drags the sum round to dialog. The word table lists each word's Int8 row, scale and contribution, and the cosine bars re-rank on every keystroke. Then the presets 'confirm destructive action' and 'fetchpriority hero image' are clicked."
  caption="The demo above being typed into. 'light' alone points at color-scheme; adding the heavier 'dismiss' moves the sum to dialog. Toy table, real arithmetic."
/>

Type "light" and its single row points straight at the color-scheme guide. Add "dismiss": its row is weighted 2.4 against light's 1.1, its contribution is [295.3, 47.2] against light's [-54.8, 114.5], and the normalised sum swings to dialog with a cosine of 0.964. Swap the two words and nothing changes. Type a word that is not in the table and it is struck out and adds nothing; type "the" and the stopword list drops it before lookup, just as `bm25.ts` does in the real code.

That is the whole mechanism. The rest is where the table comes from.

## How Google built this table

The recipe is [Model2Vec](https://github.com/MinishLab/model2vec) plus its fine-tuning step, Tokenlearn. Model2Vec's idea is to distil a static table from a real sentence model: push every vocabulary item through the teacher on its own, keep the output vector as that item's row, reduce the dimension, and weight rows by how rare the item is. MinishLab's off-the-shelf `potion-base-8M` is a useful reference point: its config says it was distilled from `bge-base-en-v1.5` with PCA to 256 and Zipf weighting, and its safetensors header holds one `[29528, 256]` F32 tensor, 30,236,672 bytes.

Modern Web Guidance's version, read from [`train-query-words.ts`](https://github.com/GoogleChrome/modern-web-guidance-src/blob/main/src/rag/train-query-words.ts):

1. **Whole words, not subwords.** The vocabulary is 29,165 words built with the same tokenizer BM25 uses (lowercase runs of letters, digits and inner hyphens, minus 31 stopwords): every token in the guides, plus search aliases, plus general English. `fetchpriority`, `popover` and `scrollytelling` are single rows, which fixes the WordPiece problem by construction.
2. **The teacher is Gemini.** Each word is embedded on its own by `gemini-embedding-001` with task type `RETRIEVAL_QUERY`, 768-d, then truncated to its first 256 dimensions and renormalised. Gemini's embeddings are trained Matryoshka-style, so the prefix is a usable smaller embedding; no PCA is needed.
3. **IDF weights.** Each word starts with the BM25-style weight $\log\!\big(1 + \tfrac{N - df + 0.5}{df + 0.5}\big)$ over the guide chunks, where $df$ is the number of chunks containing it. Words that never appear in a guide get the median.
4. **Tokenlearn.** The rows and weights are then fitted so that the weighted sum of a query's rows matches Gemini's embedding of the whole query. The README gives the training set: 32,915 queries, 2,999 synthetic ones from guide titles and headings plus 29,916 unique historical CLI searches from March to October, with the benchmark queries removed. The loss is one minus cosine, the optimiser Adagrad at learning rate 0.01, with an L2 pull of 0.05 back toward the teacher's original rows, for up to 12 epochs with early stopping on a validation split. This is what teaches "light" plus "dismiss" to land on `<dialog closedby="any">` rather than on lighting.
5. **Int8 with the weight folded in.** Each 256-d row is quantised symmetrically to Int8 with its own scale (largest component over 127), and that scale is multiplied into the word's weight. So `scales[i]` in the loop is weight times quantisation scale, and the loop never sees either separately. The script checks that the Int8 sum stays above 0.999 cosine of the float sum.

<Figure
  src="https://media.thesatyajit.com/ai/articles/modern-web-guidance-static-embeddings/fig2.png?v=43765317d2d7a406"
  alt="Two panels. Top, 'Tuned word vectors': an arrow for 'light' alone points to illumination / color-scheme; adding 'dismiss' bends the sum to dialog closedby any. Bottom left, 'Asymmetric (Gemini guides)': guides spread over a sphere, the dialog guide well separated and 'best pizza' at 0.09, rejected. Bottom right, 'Symmetric (300-word static avg)': all guides crowded into one narrow cone, 'best pizza' at 0.899, higher than the 'dialog' query at 0.844."
  caption="Google's picture of the two design decisions: tuned word vectors that bend multi-word sums toward the intended feature, and Gemini-embedded guides that stay spread out where averaged static guides collapse into one cone (Modern Web Guidance deep dive, Figure 2)."
/>

And `enc`, the thing the loop iterates? It is a list of `[row, fraction]` pairs, one per query word found in the table. The fraction is 1 for a word found whole. A hyphenated token that is not in the table, say `foo-dialog`, is split on hyphens, and each part that is in the table gets $1/m$ where $m$ is the number of parts found. Unknown words contribute nothing. There are no per-query weights to compute at runtime; the IDF lives in `scales`.

The shipped table, from the 1.0.0 tarball:

| file | bytes | what it is |
|---|---|---|
| `query-words.json` | 316,653 | the 29,165 words, sorted |
| `query-words.i8` | 7,466,240 | 29,165 × 256 Int8, exactly |
| `query-words.scale.f32` | 116,660 | 29,165 × 4-byte scales |
| `guide-chunks.i8` | 516,096 | 2,016 chunks × 256 Int8 |

The table compresses to 5,538,988 bytes with `gzip -9` on my Mac, which is 5.3 MiB, matching the blog's "5.3 MB gzipped".

## Why the guides are still embedded by Gemini

The obvious next step would be to embed the guides with the same table and drop Gemini entirely. Google tried it, and it failed in the way static averages always fail on long text. Average 300 words and the frequent, generic ones dominate; every guide drifts toward the same "average English" direction, and the guides collapse into a narrow cone. The repo's README says the averaged chunks sat inside a 0.970 cosine cone, and the blog's example is that "best pizza in new york" scored 0.899 against the dialog guide, above the query "dialog" at 0.844.

So the design is asymmetric. Guides are embedded once per release, in the cloud, by `gemini-embedding-001` with task type `RETRIEVAL_DOCUMENT`, truncated to 256-d and shipped as Int8 (`guide-chunks.i8`). Queries are embedded locally by the static table, which was trained to land in that same Gemini space. Ranking takes each guide's best-matching chunk by cosine, ranks guides by BM25 over the same chunks, and fuses the two lists with reciprocal rank fusion at $k = 60$. A query returns `[]` only if the dense top-1 cosine is under 0.30 and BM25 matches nothing.

## What the loop costs, measured

The repo has no harness for the 30 µs. `benchmark-latency.ts` times the old TensorFlow.js embedder, and the README repeats "~30 µs" without saying how it was taken. So I measured it, in three ways, with my own code: a re-implementation of `embedSync` (tokenize, look up, loop, normalise, convert to a plain array) over a synthetic Int8 table of the real shape, and the same over the real shipped table with the 1,029 public agent queries. Each number is a median: batches of 1,000 calls for the hot loop on the synthetic table, batches of 100 over the real table, 2,000 single calls otherwise, with a 64 MB sweep through memory before each "cold" call to push the table, the word map and the heap out of cache.

| what | M1 Max, Node 26.5 (mine) | i7-14700, Node 22.22 (mine, box under load) |
|---|---|---|
| loop only, 3 words, rows hot | 1.17 µs | 1.66 µs |
| loop only, 10 words, rows hot | 2.87 µs | 4.90 µs |
| full embed, real table, agent queries (median 4 known words) | 2.05 µs, or 6.81 µs with `Array.from` | 2.8 µs, or 9.31 µs with `Array.from` |
| full embed, 3 words, cold caches | 24.96 µs | 20.79 µs |

The loop is the cheapest part. Three rows of 256 bytes is 768 multiply-adds, about a microsecond. On the M1 Max, the single most expensive line in a warm `embedSync` is the last one: `Array.from` turning the 256-float `Float32Array` into a JavaScript array costs 4.3 µs on its own, more than the loop. Cold, with the 64 MB sweep before every call, the breakdown on the real table is 3.7 µs for the loop, 9.4 µs for tokenising and looking words up in a cold `Map`, and 11.8 µs for `Array.from`, about 24 µs together. That is close enough to 30 µs that I believe Google's number is a single call on a cold-ish process, and the "seven lines" are the smallest part of it.

Then the real package. I installed `modern-web-guidance@1.0.0` on the i7-14700 box (28 cores, Node 22.22.1, install scripts disabled, a training job keeping the load average between 12 and 16, everything niced) and ran it. Its own `searchUseCases` took a median 1.375 ms per warm search over 3,000 runs, and 1.252 ms with the embedder swapped for a stub that returns a fixed vector. The difference, about 0.12 ms, is an upper bound on the embed inside a real search rather than a measurement of it: my stub's vector scores at most 0.044 against any chunk, under the 0.30 floor, so the stubbed path also skips fusing the dense list. Subtracting two medians on a loaded shared CPU is noisy too. The first search in a fresh process took 228.5 ms, and the cold CLI, which is what an agent pays per search, took a median 260 ms over 20 runs (197 to 291). Google reports 185 ms cold. On my Mac a bare `node -e 0` takes 40 ms and my own loader (read the table, parse the words, build the map, gunzip and dequantise the chunks, build BM25) takes about 130 ms end to end.

So the 30 µs is a fair description of the embed and a small share of a search. The project's own dashboard data lists the static-embedder milestone at 240 ms cold and an 8.3 MB tarball, closer to what I see than the post's 185 ms and 7.1 MB. The headline "~7,500× faster" is 227 ms over 30 µs, against the pure-JavaScript TensorFlow.js engine. Against Gen 1's native ONNX figure in the same table, 34 ms, the ratio is about 1,100×, by my arithmetic. Either way the user-visible cold start went from 770 ms to 185 ms by Google's numbers, and almost none of what remains is the embedder.

The counter below makes the cost model concrete: multiply-adds for the table against MiniLM's forward pass, from MiniLM's config.

<EmbedCost />

At its defaults, three known words and seven MiniLM tokens, the table does 1,024 multiply-adds including the norm and MiniLM does 74,543,616, a ratio of about 72,797. The table side touches 780 bytes of a 7.5 MB file. Nothing else in the file is read.

## Does the search still find the right guide?

Google's numbers, from the blog and the dashboard data in [`src/dashboard/search/search.js`](https://github.com/GoogleChrome/modern-web-guidance-src/tree/main/src/dashboard/search): on held-out telemetry queries the static embedder with BM25 reaches 95.4% Hit@K, against 96.9% when the same pipeline calls the Gemini API for each query and 80.6% for the MiniLM release with the 0.38 gate. The post's image says 5,029 searches; the dashboard labels that series "Wild-v2 test", 865 answerable queries of 2,000. On the agent-eval queries, Hit@10 is 99.4%, which the dashboard labels "Agent test", 494 of 514 held-out queries. The telemetry set is internal (the benchmark README points at a `go/` link), so the 95.4% cannot be checked from outside.

<Figure
  src="https://media.thesatyajit.com/ai/articles/modern-web-guidance-static-embeddings/fig3.png?v=e14c87a98d241816"
  alt="Two charts. Left: 'Search hit rate (Hit@K) across milestones', the same line as the post image from 78.2% to 95.4%, with Gemini API at 96.9%. Right: 'Cold latency vs Hit@K (bubble = RAM)': MiniLM with the 0.38 gate at 685 ms and 342 MB, MiniLM + BM25 at 770 ms and 346 MB, Static Int8 at 185 ms and 145 MB, labelled '4.2x faster, -58% RAM'."
  caption="Google's scoreboard: hit rate by milestone, and cold latency against hit rate with RAM as bubble size. All of it is Google's measurement (Modern Web Guidance deep dive, Figure 3)."
/>

The agent set is public: [`src/rag/benchmarks/data`](https://github.com/GoogleChrome/modern-web-guidance-src/tree/main/src/rag/benchmarks/data) has 1,029 queries mined from nightly runs of Antigravity CLI, Claude Code, Codex CLI and Jetski CLI, with relevance grades from 0 to 2 per guide. So I replayed it with my re-implementation of the full search (dense, BM25 and fusion) over the shipped vectors, scoring a hit when a grade-2 guide that is not one of the 24 broad "discipline" guides appears in the top results. 988 of the 1,029 queries have such a guide.

| retriever (my re-implementation, all 1,029 queries) | Hit@1 | Hit@5 | Hit@10 |
|---|---|---|---|
| static dense only | 74.1% | 97.7% | 99.0% |
| BM25 only | 75.5% | 95.0% | 97.9% |
| hybrid, as shipped | 78.8% | 97.8% | 99.4% |

On the test split, the one Google reports, I also get 494 answerable queries, and a hybrid Hit@10 of 99.2% against the dashboard's 99.4% (with a 98.7 to 100 interval). That is one query of 494, inside their interval; the exact rule for which guides count as hits may differ by that much. Dev comes out at 99.6%, and all 1,029 together at 99.4%. Dense alone is within half a point of hybrid at ten results; BM25 earns its place at the top of the list, where hybrid's Hit@1 beats either alone by three to five points. The real package, run on the i7 box, returned the same guides in the same order as my re-implementation for every query I compared, with the same cosines to three decimals, so the table above is the package's behaviour.

Two results do not match the post.

**The confirm query.** The blog ends its story with `npx modern-web-guidance@latest search "confirm destructive action"` and "The dialog guide comes right back at the top of the list." In 1.0.0 it comes back second. The real package's list starts `agentic-forms` (cosine 0.765), then `dialog` (0.7534). BM25 ranks dialog first; the dense side ranks it sixth. Second of ten is fine for an agent that reads all ten descriptions, but it is not the top.

**The out-of-scope gate is gone in practice.** In the shipped files, everything is close to everything. The 2,016 guide-chunk vectors have a mean cosine of 0.907 with their own centroid, so they occupy a cone, not the whole sphere Figure 2a draws. Query vectors from the static table sit at a median 0.79 to that centroid, and two agent queries picked from the set have a mean cosine of 0.705 with each other over the 25,382 pairs I sampled (the README quotes 0.28 for query pairs). So every query has a high cosine with every guide. Across all 1,029 agent queries the median top-1 cosine is 0.845; across the repo's own 90 curated out-of-scope queries ("unit test mocking with jest", "react native flatlist performance", Kubernetes, Tailwind) it is 0.780. The floor is 0.30. My re-implementation returned the full ten guides for every one of the 90, and the real package gave "best pizza in new york" ten guides with cosines between 0.79 and 0.82. Worse, the inversion that Figure 2 attributes to the rejected symmetric design shows up here too, smaller: "best pizza in new york" scores 0.800 against the dialog guide's best chunk, and "dialog" scores 0.755. The ranking still puts dialog first for "dialog", because rank is relative. But absolute cosine no longer means anything, so a threshold on it cannot separate in-scope from off-topic. The code comment on `DEFAULT_MIN_SIMILARITY` says as much: single in-vocabulary words never score below 0.475, and the 0.30 floor only catches queries with no known word at all. The cost is the one the team designed their scoring to punish: ten irrelevant results, about 530 tokens, for every off-topic search. Agents mostly search on-topic, so this is a small tax rather than a bug, but the "best pizza (0.09), unrelated query rejected" panel in Figure 2 does not reproduce with the shipped table and chunk vectors.

## The sizes, checked against the tarball

Most of the post's sizes are right; a few have moved since it was measured. The 1.0.0 tarball is 8,794,985 bytes, 8.4 MiB, against the table's 7.1 MB. Of its 16.5 MB unpacked, 4,992,779 bytes is `legacy-scan.mjs`, the bundle behind a separate `legacy-scan` command that the search path never loads. `search.mjs` is 6,890 bytes against the post's 5.7 KB. The guide vectors gzip to 362,574 bytes against "335 KB", consistent with the corpus growing from the 1,859 chunks the README lists to the 2,016 in the tarball. None of this changes the argument. The dashboard's own 8.3 MB for the static milestone suggests the post's 7.1 MB came from an earlier build.

## How it compares with what this site covers

Yesterday's [MetaEncoder](/articles/meta-encoder) is the opposite end: a 28B decoder turned into an embedder for text, images and video, where one query embed is a forward pass through billions of weights. [EmbeddingGemma 2](/articles/embeddinggemma-2) at 740M parameters and [UltraBERT-Embed](/articles/ultrabert-embed) at 150M are the middle, still transformers, still a forward pass per query. Modern Web Guidance gets within 1.5 points of a Gemini API call on its own benchmark with 768 multiply-adds, because it took the teacher's work offline twice: once for every word, once for every guide.

Static tables are already on the site, unnamed. [Desert Ant Labs](/articles/desert-ant-labs)' `emo` builds on `potion-multilingual-128M`, a Model2Vec table distilled from `bge-m3`. [TurboVec](/articles/turbovec) is about the other half of the problem, searching many vectors fast; here there are only 2,016, and a brute-force scan is under a millisecond. [jev-semgrep](/articles/search-by-meaning) argues that cosine has no complement, so "about X" cannot be thresholded cleanly. The dead out-of-scope gate above is that argument showing up in production. And [BM25](/articles/bm25) is still doing real work: it is why "confirm destructive action" finds dialog at all.

## Would a static embedding help this site's search?

This site's `/api/search` is BM25 over contextualised chunks, in [`lib/search.ts`](/blog/site-search-contextual-bm25): every chunk carries its article's title, description and tags at half weight, the index is built in memory on the first request, and ranking is pure lexical. No model, no build step. It is good at what people and agents actually type here, which is names: `bm25`, `turbovec`, a model id, a paper title.

A static query embedder would help one kind of query: the paraphrase with no shared word, "make inference cheaper" against an article titled "speculative decoding". Concretely it would need three things. A table: `potion-base-8M` is 30 MB as F32 and would be about 7.6 MB as Int8, well inside the serverless function budget. Document vectors: the guides lesson says these must come from a real encoder, not the table, so a build step would embed every chunk of 730-odd articles with a teacher (`bge-base-en-v1.5`, the one potion was distilled from, so the spaces match) and commit the vectors, as Modern Web Guidance commits its chunk cache. And a reason to believe it helps: the site logs no search queries, so there is nothing to tune on and nothing to measure against, and Google's whole post is an argument that the measuring is the work.

So not yet. The part worth copying now is the fusion, which is cheap: reciprocal rank fusion of BM25 with any second ranker. The part worth copying first is the query log, opt-out and words-only, the way theirs is. Without it a static embedder here would be a guess.

## Where I land

The seven-line loop is real, and it is the right design for this workload. Short queries, a frozen corpus, a process per search and an install that must never compile anything: under those constraints, a distilled lookup table matched against teacher-embedded documents gives up 1.5 points of hit rate for removing the neural network from the runtime entirely. The 30 µs is believable as a cold single call, and the arithmetic inside it is closer to one microsecond. What I would fix is the gate. A cosine floor of 0.30 on a space where off-topic queries score 0.78 does nothing. The benchmark harness already computes an out-of-scope admission rate from the 90-query set; neither the post nor the dashboard publishes it, and on the shipped files I get 90 of 90 admitted. A floor on the BM25 score, or on the gap between the top guide and the median guide, would at least be measuring something.

## Runs and reads

Ran, my own code: a JavaScript re-implementation of `embedSync` timed over a synthetic 29,165 × 256 Int8 table and over the shipped table on an Apple M1 Max (Node 26.5.0); the same scripts on an Intel i7-14700 (Node 22.22.1, shared machine under a training job); a re-implementation of the hybrid search over the shipped chunk vectors, scored on the 1,029 public agent queries and the 90 out-of-scope queries. Ran, third-party code, on the i7-14700 only: the real `modern-web-guidance@1.0.0`, installed with install scripts disabled in a throwaway directory that was removed afterwards, for warm and cold search timings and result lists. Read: the deep-dive post, the repo at commit `3bb177a`, the 1.0.0 tarball's files, the dashboard data, MiniLM's and potion's configs. Not checked: the 4,000-query telemetry set and its 95.4%, which are internal; the training run itself, which needs a Gemini API key.