# EmbeddingGemma 2: the encoders are optional, the text model is not > Satyajit Ghana — Head of Engineering @ Inkers Technology > canonical: https://ai.satyajit.si/articles/embeddinggemma-2 > date: 2026-10-06 > tags: retrieval, multimodal, on-device, benchmarks, small-models Sundar Pichai's post called EmbeddingGemma 2 a "lightweight, modular 740M parameter form factor" that handles text, code, images, video and audio. The word I didn't understand was *modular*. A multimodal embedding model is usually either one big network that reads everything, or a set of separate towers glued together by a contrastive loss. "Modular" could mean either, and the two behave very differently when you try to put one on a phone. So I read the files. The [model card](https://huggingface.co/google/embeddinggemma-2), the safetensors header (fetched with two HTTP range requests, no weights downloaded), the `embedding_gemma2` model code in transformers, the processors that turn a JPEG or a WAV into tokens, and Google's three launch posts. The short version: every input, whatever it started as, ends up as a sequence of 512-wide vectors fed through one small text encoder, and that encoder is the only part you can't remove. The vision and audio towers are front ends. And the audio front end turned out to be something I didn't expect: it is Gemma 4's audio encoder, unchanged, down to the byte. | | | |---|---| | Weights | [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2), one 1.49 GB bf16 `model.safetensors` | | Licence | Apache 2.0 (EmbeddingGemma 1 shipped under the Gemma terms) | | Output | 768 dimensions, Matryoshka cuts at 512, 256, 128 | | Context | 8,192 tokens shared by every modality | | Launch posts | [Google blog](https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2), [developer guide](https://developers.googleblog.com/en/embeddinggemma-2-the-developer-guide/), [AI Edge post](https://developers.googleblog.com/google-ai-edge-with-embeddinggemma-2) | | Paper | none yet; the closest is [EmbeddingGemma 1's report](https://arxiv.org/abs/2509.20354) | ## What is in the 744 million The safetensors header is a JSON index of every tensor with its shape. Summing them gives 744,371,992 parameters, all bf16. The Hub's own counter says 744,371,512. The 480 missing are scalars: the audio tower stores a minimum and maximum for the input and output of every clipped linear layer, 40 per conformer layer across 12 layers, each a zero-dimensional tensor that the Hub's counter skips. Either way, 740M is a fair rounding. Grouped by which module the tensor belongs to: | Module | Parameters | What it is | |---|---:|---| | `language_model.embed_tokens` | 134,217,728 | the vocabulary table, 262,144 tokens x 512 | | `language_model.ple`, `embedding_projection`, final norm | 6,685,696 | per-layer input projection, and the 512 to 768 output head | | `language_model.layers` (24) | 130,099,224 | the transformer | | `vision_tower` + `embed_vision` | 167,757,824 | 16-layer ViT, 768 wide, and a 768 to 512 projection | | `audio_tower` + `embed_audio` | 305,611,520 | 12-layer conformer, 1,024 wide, and a 1,536 to 512 projection | That matches the card's own split to within rounding: a 130M transformer, a 140M "embedder", a 170M vision encoder and a 300M audio encoder. Two things stood out. Half of the text model is one lookup table. The 262,144-token vocabulary times a 512-wide embedding is 134.2M of the 271.0M. The transformer that does the actual reading is 24 layers of about 5.25M each (6.3M for the four global-attention layers, which have wider heads). This is the usual shape for a small Gemma: a huge multilingual vocabulary on a narrow model. And the audio tower is the largest single module in the file, bigger than the whole text model. If you want audio, you are carrying 306M parameters for it. ## An image is 280 words the vocabulary does not have The mechanism is easiest to see in `EmbeddingGemma2Model.forward`. The processor writes the text with a placeholder for every piece of media: `<|image|>`, `<|video|>`, `<|audio|>`. It then expands each placeholder into as many placeholder tokens as the media will produce, wrapped in begin and end markers. The model embeds the text tokens normally, runs each tower, projects its output to 512 dimensions, and overwrites the placeholder slots with those vectors: ```python # transformers/models/embedding_gemma2/modeling_embedding_gemma2.py:797-799 inputs_embeds = inputs_embeds.masked_scatter( image_mask.to(inputs_embeds.device), image_features.to(inputs_embeds.device) ) ``` After that line, the text encoder can't tell a word from a patch of a flamingo. It sees one sequence of 512-wide vectors and reads all of it with the same 24 layers. Google's own diagram draws it that way, with the encoders feeding a single box labelled 270M:
How many slots each piece of media takes is decided by the towers. Images go through a 16-layer ViT with 16-pixel patches. Its output is pooled three by three, so each output token stands for nine patches. At the default budget of 280 tokens an image gets up to 2,520 patches, roughly an 800 x 800 picture at its own aspect ratio. The budget can be set anywhere from 70 to 1,120, the same ladder Gemma 4 uses, which I covered in [the Gemma 4 piece](/articles/gemma-4). Video is the same tower run on frames, sampled at one per second, at 140 tokens a frame. Audio is resampled to 16 kHz mono and turned into 128-bin log-mel frames every 10 ms. Two stride-2 convolutions shrink that by four, so one audio token covers 40 ms: 25 tokens a second. The conformer's attention is chunked and local (`attention_chunk_size: 12`, `attention_context_left: 13`, `attention_context_right: 0` in the config): a token attends within its own 12-token chunk, 480 ms, plus about half a second of the past, and never past the end of its chunk. It was built to stream. Any understanding of a whole clip has to come from the text encoder, which sees every audio token at once. The text encoder itself is a Gemma 4-style stack turned bidirectional. The attention class sets `self.is_causal = False` (line 331) and the masks come from `create_bidirectional_mask`. Five sliding-window layers alternate with one global layer, 24 in all. The config says `sliding_window: 512` while the card says 1,024, and both are right: the bidirectional window lets a token see 512 positions on each side, about 1,024 in total. The global layers use one key-value head (multi-query attention) with 512-wide heads; the local ones use two key-value heads with 256-wide heads. The last step is the one that makes it an embedding model. The final hidden states are normalised and projected from 512 to 768 per token, then `sentence-transformers` takes the mean over every position, prompt included (`include_prompt: true` in `1_Pooling/config.json`), and L2-normalises it. The code comments why projecting before pooling is allowed: ```python # modeling_embedding_gemma2.py:504-505 # Projecting per token is equivalent to projecting after mean pooling self.embedding_projection = nn.Linear(config.hidden_size, config.embedding_dim, bias=False) ``` A linear map commutes with an average, so it costs more compute but gives the same vector. That head is one 512 x 768 matrix with no bias. EmbeddingGemma 1 used two dense layers after pooling, 768 up to 3,072 and back down to 768, according to [its report](https://arxiv.org/abs/2509.20354). The new model is narrower inside (512 against 768) and has a simpler head. Mean pooling has a consequence worth stating plainly. The embedding is an average over positions, and a default image is 280 positions while a caption is twenty. Attention mixes them all before the average, so a 280-to-20 split is not a 93% share of the meaning, but the media does dominate the arithmetic. I come back to this when budgeting the context window below. ## Per-layer embeddings without the lookup table One detail in the text encoder took me a while, and it is the most interesting design choice in the file. Gemma 3n and the small Gemma 4 models use per-layer embeddings (PLE): besides the normal embedding table, every token id also looks up a small extra vector for every layer, from a second, very large table, and each layer mixes its slice into the residual stream. It is how Gemma 4 E2B keeps 5B parameters on disk and runs like a 2.3B model, as I explained in [the Gemma 4 article](/articles/gemma-4). The catch, for an embedding model fed images and audio, is that a soft token has no token id. There is nothing to look up for a patch of a flamingo. EmbeddingGemma 2 keeps the per-layer signal and drops the table. The per-layer inputs are computed from the input embeddings themselves, through one linear layer: ```python # modeling_embedding_gemma2.py:193-200 (EmbeddingGemma2TextPLE) def forward(self, inputs_embeds): per_layer_projection = self.per_layer_model_projection(inputs_embeds) * self.per_layer_model_projection_scale per_layer_projection = per_layer_projection.reshape( *inputs_embeds.shape[:-1], self.num_hidden_layers, self.hidden_size_per_layer_input, ) return self.per_layer_projection_norm(per_layer_projection) ``` The class docstring calls it "projection-only PLE: the signal is derived from `inputs_embeds` alone, with no token-identity lookup". The projection is 512 to 24 x 512, which is the 6.29M `per_layer_model_projection` in the header. Each layer then gates its slice in with a small bottleneck after attention and the MLP (`hidden_states * per_layer_input` at line 224). This works the same for a word and an image patch, because both arrive as a 512-wide vector before the projection. It also explains part of why the text model is 271M and not several billion: there is no per-layer table to carry. I'd read it as a requirement of native multimodality more than a cost trick, though Google doesn't say which. ## Modular means the front end, not the model Now the word from the post. In `EmbeddingGemma2Model.__init__`, each tower is built only if its config exists: ```python # modeling_embedding_gemma2.py:646 self.vision_tower = AutoModel.from_config(config.vision_config) if config.vision_config is not None else None ``` Pass `config_kwargs={"vision_config": None, "audio_config": None}` to `SentenceTransformer` and neither tower nor its projection is constructed; the loader is told to ignore their weights (`_keys_to_ignore_on_load_unexpected`, line 637). What is left is the 271M text encoder. Add vision for 440M, audio for 570M, both for 740M. Those are the card's four configurations, and the header sums agree with them. The part that matters for an index: a text input never touches either tower. The placeholder masks are empty, the `masked_scatter` calls are skipped, and the same 24 layers run on the same token embeddings. So a text embedding from the 270M configuration is the same vector as one from the full model, and the developer guide says exactly that: "Embeddings you have already computed do not need to be re-computed." You can index a document store with the small configuration on a laptop today and add photos to the same index later with the full one. What you can't do is drop the text encoder. An image embedding is the mean of the text encoder's output over the image's soft tokens. There is no vision-only embedding path, and the vision tower alone produces nothing you can compare. "Modular" here means a model with swappable front ends and one shared reader, not a set of independent towers. ## The audio encoder is Gemma 4's, byte for byte The launch blog says EmbeddingGemma 2 "shares its text tokenizer and audio encoder" with Gemma 4, so that the two can run together "with a lower combined total memory footprint". The developer guide softens that to sharing the "audio encoder architecture". Those are different claims: one means the same code, the other means the same weights you could load once. I checked the weights. Safetensors stores each tensor as a contiguous byte range listed in the header, so I fetched the same tensors from `google/embeddinggemma-2` and `google/gemma-4-E2B` with range requests and hashed them. All 36 audio tensors I sampled, three from each of the 12 conformer layers (attention projections, convolution input projections and the clipping bounds), plus the tower's output projection, were byte-identical. They are also identical in `gemma-4-E2B-it` and `gemma-4-E4B`. The tokenizer vocabulary is the same 262,144 entries in the same order. The vision tower is a different story. Same shapes, different bytes: none of the eight vision tensors I sampled matched Gemma 4 E2B's. So the vision encoder was trained for embeddings, and the audio encoder was either frozen throughout or never trained here at all, with only the 1,536 to 512 projection (`embed_audio`) and the text encoder learning to read it. Two practical consequences. If you run Gemma 4 E2B for generation and EmbeddingGemma 2 for retrieval on the same device, the 305M audio tower can, in principle, be loaded once; whether a runtime does that is up to the runtime, and I couldn't check what LiteRT does. And EmbeddingGemma 2's audio understanding is bounded by what Gemma 4's speech encoder already captures, which is mostly speech. That fits the benchmark numbers below. ## How it was trained, as far as the files say There is no EmbeddingGemma 2 paper. The model card covers data (web text in over 140 languages, code, images, video, speech and sounds, paired cross-modal samples, a January 2025 cutoff) and nothing about the loss or the stages. So what follows is partly inference, and I'll mark it. EmbeddingGemma 1's report describes the recipe that this almost certainly builds on: a noise-contrastive loss over in-batch negatives with harder negatives weighted up, a spread-out regulariser that penalises squared dot products between unrelated embeddings in a batch, a loss that matches the student's embeddings to Gemini Embedding's, Matryoshka versions of those losses applied to nested prefixes of the vector, a large pre-finetuning stage on unlabelled pairs, a finetuning stage with hard negatives on several mixtures, and a final model that averages the checkpoints from those mixtures. The card's task prefixes (`task: search result | query: `) are the same family as EmbeddingGemma 1's, and the blog says EmbeddingGemma 2 is "built from the same technology as Gemini Embedding models". For the multimodal part, the contrastive loss itself doesn't care what the positives are. A batch can pair a spoken question with a passage, a caption with a photo, or a text query with a video, and InfoNCE pulls each matched pair together and pushes the rest apart. I walked through that loss, and the two failure modes a shared index has (the right concept in the wrong modality, and a modality gap that clusters captions with captions), in [the Ovis-Embedding article](/articles/ovis-embedding). What the files do tell me: the audio tower didn't move, the vision tower did, the text encoder is a new 512-wide network rather than EmbeddingGemma 1's 768-wide one, and MRL is trained in: the card says so, and its truncation table degrades gently down to 256, which plain truncation of an untrained vector would not. The AI Edge post adds that quantization-aware training produced the int4 and int8 versions. Everything else about the recipe, including whether a Gemini Embedding teacher was used for images and audio, I couldn't check. ## Spending the window The card advertises an 8,192-token context "capable of processing minutes of audio or video", and a table: about 29 images, 58 video frames or 327 seconds of audio. Those are 8,192 divided by 280, 140 and 25. They describe the window, not what the shipped preprocessing will actually feed it. The video processor in transformers has `max_frames = 32` and `overflow_strategy = "uniform"` (`video_processing_embedding_gemma2.py:192-193`). Hand it two minutes of video at one frame per second and it keeps 32 frames spread evenly, one every 3.75 seconds. The audio side goes through Gemma 4's feature extractor, whose `__call__` defaults to `max_length=480_000` samples with truncation on, which is 30 seconds at 16 kHz. A five-minute recording becomes its first 30 seconds. Both caps can be overridden, and the LiteRT runtime streams audio in its own way, but the default `model.encode({"audio": ...})` path is what most people will run first. I couldn't run it to confirm the end-to-end behaviour, so treat this as a reading of the code with a test to do, not a measured result. There is also a third number in the repo's own `processor_config.json`, `audio_seq_length: 280`, which caps the token count reported by `_compute_audio_num_tokens` (the helper serving engines call to reserve placeholder slots) at 280 tokens, 11.2 seconds. The class default is 750. I don't know which value vLLM ends up using; if long audio fails to embed there, this is where I'd look. None of this is a defect. A single vector is a poor summary of five minutes of anything, and Google's own Video Moments Finder demo indexes chunks of a video, not the whole file. The practical rule is to chunk long media yourself, a few seconds of audio or a handful of frames per embedding, and let the index hold many vectors per file. The planner below adds up a single input. It also shows the share of positions each modality contributes to the final mean, which is where interleaved inputs get surprising. In numbers: text costs one token per subword, an image its budget plus two marker tokens, a video frame 142, and an audio clip 25 per second plus two. A product listing with a 40-token description and two photos at the default budget is 604 tokens, about 7% of them text. Add 20 seconds of video and it is 3,444 tokens and the text is about 1%. If the words matter, I'd embed the description on its own as well and keep both vectors, or lower the vision budget to 70 for the photos in an interleaved input. ## The benchmark claims The card's headline table, all at 768 dimensions and full precision: | Benchmark | Metric | EmbeddingGemma 2 | EmbeddingGemma 1 | |---|---|---:|---:| | MTEB multilingual v2 | mean over tasks | 61.36 | 61.15 | | MTEB code v1 | nDCG@10 | 78.68 | 68.76 | | MIEB lite | mean over task types | 64.64 | – | | MMEB v2 image | Hit@1 | 57.28 | – | | MMEB v2 visual documents | nDCG@5 | 67.84 | – | | MMEB v2 video | Hit@1 | 50.67 | – | | MSEB retrieval | MRR@10 | 69.54 | – | | MAEB | mean over tasks | 49.39 | – | Text is flat: 61.36 against 61.15 is a 0.21-point gain, which the blog fairly calls matching. Code is where the text model improved, by 9.92 points, the "14%" in the developer guide (9.92 / 68.76 is 14.4%). For scale on text, Qwen3-Embedding-0.6B's own card reports 64.33 on MTEB multilingual and 70.70 on MTEB English v2 against EmbeddingGemma 2's 61.36 and 68.46. Qwen's is a 600M text-only model against a 271M text path, so I don't read it as a loss, but EmbeddingGemma 2 isn't the strongest small text embedder; its case is that the same index takes pictures and sound. The "outperforms some specialist models more than twice its size" line comes from three charts in the launch blog, each scoring models against size on a log axis. I read every dot's value off its pixel position against the tick marks, which is good to about half a point.
On code the chart places EmbeddingGemma 2 at its text size, near 270M, and its nearest rivals are Qwen3-Embedding-0.6B (about 75.3) and inf-retriever-v1-1.5b (about 67.0). One oddity: the blue dot sits at about 76.7, not the card's 78.68, and EmbeddingGemma 1's dot sits at about 66.7 against its card value of 68.76. The gray dots line up with published values (Qwen3-Embedding-0.6B reads about 75.3 against the 75.41 in Qwen's own README), so the axis is fine. Both Gemma dots are about two points low, which looks like the chart and the table came from different evaluation runs. The table is the better number to quote, and either way the ranking on the chart doesn't change.
On images the claim holds cleanly. EmbeddingGemma 2 reads about 64.5, which matches the card's 64.64. jina-embeddings-v5-omni-small, 1.63B parameters by its own header count and so 2.2 times EmbeddingGemma 2, reads about 61.4. BidirLM-Omni-2.5B reads about 55.5, VLM2Vec-LoRA about 44.6. Only LCO-Embedding-Omni-3B, about four times the size, is higher, by about one point.
Audio is where I'd push back. EmbeddingGemma 2 reads about 48.6 (card: 49.39) and does beat e5-omni-3B (about 48.1), Qwen2-Audio-7B (about 34.8) and Qwen2.5-Omni-3B (about 23.6), all well over twice its size. But the blog also says it "achieves leading scores among sub-1B multimodal embedders" on MAEB, and its own chart shows jina-embeddings-v5-omni-nano above it at about 51. That model's header holds 985,984,512 parameters, under a billion. It is licensed CC-BY-NC-4.0, so if "leading" quietly means "leading among models you can ship commercially", the claim survives. As written, it doesn't, on Google's own figure. The pattern across the three charts is consistent with the audio tower being Gemma 4's speech encoder: strong on code and images, where the trainable parts did the work, and mid-pack on general audio, where the encoder was inherited. ## Shorter vectors Matryoshka training means the first 512, 256 or 128 numbers of the 768-dimensional vector are themselves a usable embedding, once renormalised. The card's truncation table shows how much each benchmark keeps. The cost is uneven. At 256 dimensions every benchmark keeps at least 95% of its full score, and storage drops threefold: a million vectors go from 3.07 GB to 1.02 GB in float32, or 1.54 GB to 512 MB in bfloat16. At 128 the text rows keep 94% to 96% of their scores (multilingual text goes from 61.36 at full width to 57.89 cut), but the multimodal average, MMEB v2 overall, falls from 59.01 at full width to 45.65 cut, about 77%, and MSEB spoken retrieval from 69.54 to 56.71. The card's own advice is "128d is best suited to text-only workloads", and the numbers back it. Two small inconsistencies in the launch material. The AI Edge post says MRL cuts index footprints "by up to 8x"; 768 to 128 is 6x, which is what the card and the main blog say. And the card's re-normalisation warning is worth taking seriously: a cut vector is no longer unit length, and cosine scores on unnormalised prefixes still look plausible while ranking worse. ## Ten million vectors in 0.4 GB, and what that leaves out *Added 7 October 2026.* The day after launch, Qdrant's Shivendu (@KShivendu_) posted the storage side of the story: "We pushed 10M of Google's EmbeddingGemma 2 vectors from ~30GB of RAM down to ~0.4GB", by quantizing each dimension to one bit with TurboQuant and then cutting the vector to 256 dimensions with the Matryoshka truncation above. "77x less RAM for a ~5% drop in quality." The [Qdrant blog post](https://qdrant.tech/blog/embeddinggemma-2) behind it has the numbers, and most of them hold up. Two things about how they're framed don't, so here is the arithmetic first. Ten million 768-dimensional float32 vectors are 10,000,000 x 768 x 4 bytes, which is 30.72 GB. The blog says 30.7, the post rounds to 30. At one bit a dimension, 768 dimensions are 96 bytes, but the blog gives 104 bytes a vector; at 512 dimensions it gives 72, where the bits alone are 64; at 256 it gives 40 against 32. So every quantized vector carries 8 bytes beyond its bits. Qdrant's [TurboQuant write-up](https://qdrant.tech/articles/turboquant-quantization/) says what goes in them: per-vector scalars, as far as I can tell a stored length and RaBitQ's correction for how much quantization shrank the vector, which the scoring multiplies back in. That accounts for the whole gap you'd otherwise puzzle over. Bits alone would give 96x at 256 dimensions; 3,072 bytes over 40 is 76.8x, the "77x". The post's own "30 GB to 0.4 GB" is 75x, but that is just the rounding of 30.72. At 768 dimensions the same overhead turns a 32x bit ratio into 29.5x, which the blog calls "30x".
The footnote on that chart is the first thing I'd hold onto. "Vector RAM excludes graph indexes, payloads, and original vectors stored for rescoring." The 94.5% belongs to the rescoring bar, and rescoring means Qdrant fetches 40 candidates with the 1-bit codes and ranks them again with the original vectors, which have to be stored somewhere. At 256 dimensions in float32 that is another 10.24 GB, normally on disk and read on every query. So "0.4 GB of RAM with 94.5% kept" is true of the RAM, and the 94.5% also needs ten gigabytes of originals the RAM figure doesn't count. Without rescoring the same 0.4 GB keeps 88.1%, which is a 12-point drop, not 5. The graph is the other thing left out, and at this compression it is the bigger one. Qdrant searches an HNSW graph, and with its usual m = 16 each vector keeps up to 32 neighbour ids of 4 bytes at the bottom layer. That is up to 128 bytes a vector, three times the 40-byte vector itself, and about 1.28 GB for ten million. That is my estimate of an upper bound, not a number Qdrant published, and real graphs fill fewer links than the maximum, but once vectors get this small the index structure around them is what you're actually paying for. None of that makes the result bad. The configuration the blog itself recommends starting with is the full 768 dimensions at 1 bit with rescoring off: 104 bytes a vector, about 1 GB for ten million, and 99.0% of the baseline's nDCG@10. On Quora's 523,000 documents Qdrant reports it took less than half the query time of float32 search. That is the number I'd take from the post, more than the 77x. Two caveats the blog states itself. The retention is an average over five small BEIR sets (SciFact, NFCorpus, ArguAna, SCIDOCS and FiQA), so the ten million is a byte calculation, not a measured run. And 99% of the nDCG is not 99% of the same results: without rescoring, about 74% of the top 10 matched exact search. The comparison I found most useful is one the blog makes in passing. At 72 bytes a vector you can have 512 dimensions at 1 bit, which kept 97.3%, or 128 dimensions at 4 bits, which kept 84.0%. Same memory, thirteen points apart. Cutting bits is cheaper than cutting dimensions, which fits the MRL table above: at 128 dimensions the model itself has already thrown away what the extra bits would have preserved. Qdrant's results are all text retrieval. Shivendu says in the replies that MRL and TurboQuant "work on any kind of embeddings", and mechanically they do, but the card's own table has the multimodal scores falling much faster under truncation than the text ones, so for an index of photos and recordings I'd keep 512 or 768 dimensions and take the savings from the bits. ### Whose 1-bit quantizer it is TurboQuant ([Zandieh et al.](https://arxiv.org/abs/2504.19874)) rotates each vector by a random orthogonal matrix so that every coordinate follows the same known distribution, then quantizes each coordinate with one fixed Lloyd-Max codebook. I walked through it for the KV cache in [the TurboQuant article](/articles/turboquant-kv-cache), and for vector search in [TurboVec](/articles/turbovec). At four or two bits the codebook is the interesting part. At one bit it isn't: the optimal one-bit quantizer for a symmetric distribution is the sign, so 1-bit TurboQuant stores the sign of each rotated coordinate. That is also exactly what [RaBitQ](https://arxiv.org/abs/2405.12497) stores. Qdrant's own write-up is candid about this. The 8 bytes of per-vector scalars come from RaBitQ's length renormalization, not from the TurboQuant paper, and "The 1-bit scoring path follows RaBitQ as-is, without specific modifications": the query is scalar-quantized into bit planes and scored against the stored signs with an AND and a popcount per plane. One reply to the post, from @gaoj0017, quoted that line and called the "TurboQuant" label misleading. Going by Qdrant's own documentation, I agree. Every configuration in the EmbeddingGemma 2 chart is a 1-bit one, so what was measured is a RaBitQ-style 1-bit code with a Hadamard-type rotation in front of it. The name is the only thing wrong. Where Qdrant's TurboQuant does add something is at two and four bits, and its own benchmark shows the 1-bit version beating its older binary quantization on every dataset it tried.
That chart is the honest summary. Plain binary quantization takes the sign of the raw coordinates; a random rotation first, plus a length correction and a multi-bit query, is worth 9 to 21 points of recall at the same 32x. Whoever's name goes on it, it is why 1-bit storage of EmbeddingGemma 2 vectors keeps 99% of the ranking quality. One other reply linked [arXiv 2608.30384](https://arxiv.org/abs/2608.30384); that is RSLM, a separate training-free rotate-and-Lloyd-Max codec, not the TurboQuant paper. ## On a phone The [LiteRT bundles](https://huggingface.co/litert-community/embeddinggemma-2-740m-litert-lm) use quantization-aware training: int4 per-channel for the transformer and the embedding table, int8 for the vision encoder, and a mix of int2, int4 and int8 for the audio encoder. The downloads are 165 MB for text only, 388 MB for text and vision, and 485 MB for everything. The AI Edge post gives per-image vision latency across devices. The text-only number on the LiteRT card is the one I find most useful: 8.3 ms on a Pixel 11 Pro's TPU and 41.8 ms on an iPhone 18 Pro's CPU, both for a 128-token input.
Note the 70-token budget in that table: a quarter of the default. The LiteRT bundles only expose 70 and 140 soft tokens per image, so the on-device model is running images at lower resolution than the benchmark table, which was measured at full precision with default settings. I couldn't find on-device quality numbers at 70 tokens. The memory figures don't line up across sources. The blog says "~191MB active RAM for text-only weights and ~567MB for the full multimodal model" on a Pixel 11 Pro. The LiteRT card's Pixel 11 Pro row reports 112 MB and 127.5 MB, but it measures CPU memory only and says accelerator memory is excluded. The S26 Ultra on CPU reports 334 MB and 811 MB. I'd budget from the CPU rows for your own platform, not the headline. ## Using it The card's calls, put together the way I'd start a text index. I haven't run this here; it is assembled from the card's and the developer guide's own examples, and needs `sentence-transformers` 6.1.0 or later. ```python import torch from sentence_transformers import SentenceTransformer # bf16 or fp32 only: the card says fp16 overflows and returns NaN or degraded vectors dtype = torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float32 # Text and code only: the vision and audio towers are never built (271M parameters) model = SentenceTransformer( "google/embeddinggemma-2", config_kwargs={"vision_config": None, "audio_config": None}, model_kwargs={"torch_dtype": dtype}, truncate_dim=256, ) files = {"pool.py": "def mean_pool(h, mask):\n return (h * mask[..., None]).sum(1) / mask.sum(1, keepdim=True)"} # Documents: format the title yourself; prompt_name="Document" would write "title: none" docs = [f"title: {name} | text: {code}" for name, code in files.items()] doc_emb = model.encode(docs, normalize_embeddings=True) # Queries: the task prefix matters, and queries and documents must share a width q_emb = model.encode("average token states ignoring padding", prompt_name="CodeRetrieval", normalize_embeddings=True) print(model.similarity(q_emb, doc_emb)) ``` Later, the same index takes photos and recordings from the full model, without re-embedding the text: ```python full = SentenceTransformer("google/embeddinggemma-2", model_kwargs={"torch_dtype": dtype}, truncate_dim=256) photo = full.encode({"image": "IMG_0412.jpg"}, normalize_embeddings=True) # no text prefix for media memo = full.encode({"audio": "memo_16k_mono.wav"}, normalize_embeddings=True) # chunk long audio first listing = full.encode({ "text": "Waterproof trail shoe. <|image|> Grip test on wet rock: <|video|>", "image": "shoe.jpg", "video": "grip.mp4", }, normalize_embeddings=True) ``` The prefixes are listed in `config_sentence_transformers.json`: `SearchQuery`, `QuestionAnswering`, `FactChecking`, `CodeRetrieval`, `Classification`, `Clustering`, `SentenceSimilarity` and `Document`. Retrieval is asymmetric (a query prefix on one side, `title: … | text: …` on the other); classification and similarity put the same prefix on both. Media go in without a prefix. Because the pooling includes the prompt, the prefix changes the vector; mixing prefixed and unprefixed text in one index is a quiet way to lose recall. ## Half a gigabyte, and which half A few hours after launch Unsloth posted that EmbeddingGemma 2 "runs locally on 0.5GB RAM", in the same breath as "740M parameter ... combines a 270M text model with vision (170M) + audio (300M)", and linked [its own GGUFs](https://huggingface.co/unsloth/embeddinggemma-2-GGUF) and [a guide](https://unsloth.ai/docs/models/embeddinggemma-2). Read together, that sounds like the whole multimodal model in half a gigabyte. I wanted to know which configuration the number belongs to, so I read the GGUF headers of both repos the same way I read the safetensors one: range requests for the first few megabytes, then a parse of every tensor's name, shape and type. The [ggml-org conversion](https://huggingface.co/ggml-org/embeddinggemma-2-GGUF), made by the same people who merged EmbeddingGemma 2 support into llama.cpp (pull request 30054, merged on launch day), has four files: a text model in BF16 and Q8_0, and an `mmproj` in BF16 and Q8_0. Unsloth's repo has those four with byte-identical tensors (same 413 and 963 tensor lists, same types, the last 4 MB of each pair hash the same; the files differ by 64 to 96 bytes of name metadata), plus three "UD" text quants, Q4_K_XL, Q5_K_XL and Q6_K_XL, and F16 copies of both files. That is the whole difference. | File | Size | What is inside | |---|---:|---| | text, UD-Q4_K_XL (Unsloth) | 175.7 MB | 271,002,648 params; Q4_K almost everywhere | | text, UD-Q5_K_XL (Unsloth) | 210.1 MB | Q5_K, with Q6_K on `attn_v` and `ffn_down` in 12 of the 24 layers | | text, UD-Q6_K_XL (Unsloth) | 248.8 MB | Q6_K throughout | | text, Q8_0 (both) | 309.9 MB | Q8_0, the per-layer projection kept in BF16 | | text, BF16 (both) | 558.0 MB | unquantised | | mmproj, Q8_0 (both) | 554.8 MB | 473,369,344 params: vision 167.4M, audio 304.8M, two projections 1.2M | | mmproj, BF16 (both) | 982.1 MB | unquantised | The text file is exactly the 271M path from the safetensors header: token table, 24 blocks, the projection-only PLE (`per_layer_model_proj`, 512 x 12,288) and the 512-to-768 output head, stored as `output.weight`. There is no per-layer token table in it, which confirms the reading above from a second, independent implementation; llama.cpp's graph comments the function "this model has no per-layer token embeddings, the per-layer inputs come only from the projection". The header also answers the window question from earlier: it stores `sliding_window = 1024`, and llama.cpp's symmetric mask divides it by two, masking anything more than 512 positions away on either side. Same window, written down two ways. Two things in the files surprised me. About 15.8 MB of every text GGUF, 9% of the Q4 file, is not weights at all: it is the header, holding the 262,144-token vocabulary and 514,906 BPE merges. And the Q4_K_XL is less exotic than the "UD" label suggests. Every large matrix, the embedding table included, is plain Q4_K; what gets more bits are the small pieces, the per-layer gate and projection in every block (Q8_0), the PLE projection (Q5_K) and the output head (Q6_K). For a model whose last step is one 512 x 768 matrix that every embedding passes through, protecting that matrix is the sensible choice. The quantisation was calibrated with an importance matrix over 230 chunks of a text file (`quantize.imatrix.dataset = data/calib.txt` in the metadata), which is fine for a text index and says nothing either way about how image or audio tokens fare, since images and audio pass through the same quantised blocks. On the multimodal side there is only one mmproj, and it holds both towers. I checked what llama.cpp does with that: `clip_init` builds a vision context if the file has a vision encoder and an audio context if it has an audio encoder, with no switch to skip either (the only exception in the code is Gemma 3n). So the four configurations from the model card collapse to two in this runtime, text only or everything. The 440M "text and vision" setup doesn't exist in GGUF form; you carry the 305M audio tower whether you use it or not. The smallest mmproj is Q8_0, and even that one is 70.5 MB of F32, most of it the vision tower's 768 x 10,240 x 2 position table (62.9 MB), upcast from the BF16 it was stored in. ### The arithmetic for 0.5 GB For an embedding model llama.cpp builds no KV cache (`build_attn_inp_no_cache` in the `gemma-embedding2` graph): every input is read once, in one batch. So the memory is the weights plus whatever the forward pass needs at once, and the forward pass has one large resident: the per-layer inputs. They are computed for every position before the first layer runs and kept until the last, 24 layers x 512 values x 4 bytes, which is 49,152 bytes per token. At the 2,048-token context Unsloth's guide uses, that tensor is 100.7 MB. Add the 175.7 MB Q4 file, and the scratch for one layer (the 2,048-wide feed-forward activations are about 34 MB; the global layers' attention scores, if flash attention is off, are 4 heads x 2,048 x 2,048 x 4 bytes, 67 MB), and the total comes out around 0.4 GB. So for text and code, at that context, "0.5 GB" is believable. At the full 8,192 tokens the per-layer inputs alone are 402.7 MB, and a text-only Q4 run is already past half a gigabyte before anything else. With pictures and sound it isn't close. The smallest files that can embed an image are the Q4 text model plus the Q8_0 mmproj, 730.5 MB on disk before a single activation. The figure that does fit the full model is Google's own: about 567 MB of active RAM for the full multimodal model on a Pixel 11 Pro, but that is the LiteRT bundle, whose audio encoder is partly int2, not a GGUF. So I'd read Unsloth's 0.5 GB as true of the 271M text configuration at a modest context, and of Google's LiteRT build for the whole model. Through Unsloth's own files, the whole model needs about 0.73 GB before it starts. These are sums from the headers and the graph code; I didn't run llama.cpp here, so the scratch sizes are an estimate of what its allocator holds, not a measurement. One more thing the repo contains that I wouldn't use: F16 copies of both files. The model card says EmbeddingGemma 2's activations exceed float16's range and come back as NaN or silently degraded, and Unsloth's own guide repeats "Do not use FP16". In ggml's CPU path an F16 weight matrix is multiplied against activations converted to F16, so the F16 GGUF is the one file where that warning plausibly applies. Take BF16 or Q8_0. ### Running it Unsloth's llama.cpp recipe serves text and code only. The part worth copying is the flags: ```bash # unsloth.ai/docs/models/embeddinggemma-2, "Run EmbeddingGemma 2 GGUFs in llama.cpp" ./llama.cpp/build/bin/llama-server \ --model "embeddinggemma2-gguf/embeddinggemma-2-UD-Q4_K_XL.gguf" \ --alias embeddinggemma2 \ --embeddings \ --pooling mean \ --ctx-size 2048 \ --batch-size 2048 \ --ubatch-size 2048 \ --parallel 1 \ --host 127.0.0.1 \ --port 8080 ``` `--pooling mean` matches the sentence-transformers pooling; the GGUF also carries `pooling_type = 1`, which is mean. The three sizes have to move together because a non-causal model can't split one input across batches: every token attends to every other, so an input has to fit in one physical batch. And the server does nothing about prompts. The guide's request writes them by hand, `task: search result | query: ...` for the query and `title: none | text: ...` for the document, and that is required: as noted above, the prompt is part of the mean. The guide's closing hint is honest about the rest: "For images, video or audio, use a runtime with explicit support for those encoders and their processor files." I didn't check whether llama-server's embeddings endpoint feeds an mmproj today. If you need media embeddings outside Python, LiteRT is the path Google tested. Unsloth also wired EmbeddingGemma 2 into Unsloth Desktop as the embedding model for its document chat. The settings screenshot in the guide labels it "Not downloaded · 135 MB". That matches none of the files: the smallest GGUF is 175.7 MB and the safetensors is 1.49 GB. I couldn't find which artefact it refers to.
That last line in the screenshot is the right instinct, and it leads into the fine-tuning part. ## Fine-tuning moves every modality at once Unsloth's guide trains the text model with LoRA. It saves the text-only configuration first, then loads it in 4-bit and attaches adapters to every attention and MLP projection: ```python # unsloth.ai/docs/models/embeddinggemma-2, "Prepare the model" model = FastSentenceTransformer.from_pretrained( model_name = "embeddinggemma2-text", max_seq_length = 1024, dtype = torch.bfloat16, load_in_4bit = True, load_in_16bit = False, full_finetuning = False, ) model = FastSentenceTransformer.get_peft_model( model, r = 16, target_modules = [ "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj", ], lora_alpha = 32, lora_dropout = 0, bias = "none", use_gradient_checkpointing = "unsloth", random_state = 3407, task_type = "FEATURE_EXTRACTION", ) ``` Training is sentence-transformers' `SentenceTransformerTrainer` with `MultipleNegativesRankingLoss`, which is the in-batch-negatives contrastive loss from the training section above, a no-duplicates batch sampler (so a repeated query never becomes its own negative), `bf16=True` and the query and document prefixes applied to the training columns. The guide's advice to hold out queries and compare before and after is the part people skip and shouldn't. Two remarks on the recipe. Loading a 271M model in 4-bit saves about 0.4 GB of weights, which is small next to the activations at a training batch; the text notebook's own log puts peak reserved memory at 11.3 GB on a T4. Plain 16-bit LoRA costs little more here and trains against the weights you will actually deploy. And the adapters don't reach the per-layer gates, the PLE projection or the 512-to-768 head, which is fine for adapting a domain. The bigger point follows from the architecture. An image embedding is the text encoder's average over the image's soft tokens, and so is an audio one. LoRA on `q_proj` through `down_proj` of the text model therefore changes the vector for every photo and recording too, even though the training data was only text. The "embeddings you have already computed do not need to be re-computed" property from the modular section holds for adding towers, not for training the reader: after a fine-tune, everything in the index has to be re-embedded, and the image-text alignment has been nudged by a loss that never saw an image. If the index is mixed, train on mixed pairs, or keep the fine-tuned model for text and test cross-modal recall before swapping it in. Unsloth does have mixed-pair notebooks. The image one trains on 3,000 Flickr8k images with five captions each and tests on the Flickr30k 1K split, adding `finetune_vision_layers = True` so the vision tower gets adapters too; the audio one trains on Clotho clips with `finetune_audio_layers = True`. Note what the second does to the claim about sharing an audio encoder with Gemma 4: once those adapters are merged, the tower is no longer byte-identical to Gemma 4 E2B's, and a runtime that loads it once for both models can no longer do so. None of the three notebooks shows an EmbeddingGemma 2 result yet. The image and audio ones have no saved outputs. The text one does, but they come from the earlier model: its log reads "Fast Gemma3 patching" with transformers 4.57.3, and "Trainable parameters = 13,074,432 of 315,937,536". Rank-32 LoRA on EmbeddingGemma 1's 24 layers (768 wide, 1,152 MLP) is 8,355,840 parameters, and its two dense head layers, 768 x 3,072 each, add 4,718,592; together that is exactly the 13,074,432. The NDCG@10 of 0.9267 before and 0.9351 after on the medical set are EmbeddingGemma 1 numbers, in a notebook titled for EmbeddingGemma 2. ## What else the launch blog says A few things in [Google's launch post](https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2/) I haven't covered. EmbeddingGemma 1 passed 20 million downloads, which is the context for a second version. The 8K window is four times EmbeddingGemma 1's. Besides LiteRT there is a MediaPipe Decision Task API for classification and routing on top of the embeddings, and a Foresight app that pairs EmbeddingGemma 2 retrieval with Gemma 4 generation, which is where the shared-audio-encoder argument would pay off. The blog lists day-one support in transformers, sentence-transformers, MLX, vLLM, llama.cpp, SGLang, Ollama, LM Studio, transformers.js and Qdrant, and points to Unsloth for fine-tuning. Omar Sanseviero's launch post summed up the release as "Modular, going from 270m to 740m parameters", which after reading the llama.cpp loader I'd amend to: in GGUF, 271M or 744M, nothing in between. ## Typed Decision: a decision head on the 271M *Added 12 October 2026.* Five days after launch, Manuel Romero (mrm8488) published [Typed Decision](https://huggingface.co/mrm8488/embeddinggemma-2-ft-typed-decision), the text encoder fine-tuned to answer typed questions about a piece of state: a choice among named options, a level on a rubric, or a yes/no probability, the three System One primitives this site has followed since [Jev](/articles/reproducing-jev). His [post](https://x.com/mrm8488/status/2109406679045423394) put it at "76.4% on typed-decisions, ~6 pts behind a 27B LLM with ~100× fewer params". One of the first replies asked the obvious question: the AI Edge post above already sells EmbeddingGemma 2 as a zero-shot "decision engine", so what does the fine-tune add? The answer is in the code, and it's a nice small design.
The state, each question and each option are embedded separately by the same 271M text encoder, with the `task: classification | query: ` prompt and mean pooling. Before embedding, the state is flattened from JSON into `key: value` lines, so `{"trace_summary": {"constraint_violations": 1}}` becomes `trace summary · constraint violations: 1`, which an embedding model reads far better than braces. Every option is embedded as a whole sentence, `question: | answer: : `; a score level becomes `level 2 of 3: significant risk`, and a yes/no question is just two options whose answers are "no" and "yes". Then a head of about a million parameters turns those vectors into one logit per option: ```python # modeling_gemmadecision.py:210-222 S, Q, O = self.state_proj(s), self.question_proj(q), self.option_proj(o) f = self.field_norm(self.field(torch.cat([S, Q, S * Q], -1)) + self.type_emb(kind)) On = self.option_norm(O) ... joint = self.log_joint_scale.clamp(max=cap).exp() * F.cosine_similarity(f, On, dim=-1) residual = self.residual(torch.cat([f, On, f * On, (f - On).abs()], -1)).squeeze(-1) anchor = F.normalize(s + q, dim=-1).unsqueeze(-1) prior = self.log_prior_scale.clamp(max=cap).exp() * (F.normalize(o, dim=-1) @ anchor).squeeze(-1) logits = prior + torch.sigmoid(self.gate) * (joint + residual) ``` If that formula looks familiar, it is Cloudflare's Clef joint-schema head, the one [Unsloth vendored](/articles/unsloth-train-a-decision-model#what-a-clef-head-is): a fixed `prior`, plus a gated sum of a learned cosine and a small MLP over the usual matching features. The difference is what feeds it. Clef reads an LLM's hidden states, where each option's tokens can attend to the whole prompt. Here every text is pooled into one 768-wide vector before the head sees it, so the option never reads the state; the only place they meet is a 256-wide layer in the head. And the `prior` term is exactly the reply's answer: the cosine between an option's embedding and the normalised sum of state and question is what an untrained embedding model does when you use it as a zero-shot classifier. The fine-tune keeps that and learns a correction on top. I read the trained scalars out of the safetensors file with range requests. The prior's scale is 20.5 and the joint scale 11.0, both close to where they were initialised (20 and 10). The gate went from -2.0 to -1.92, so the learned term is still multiplied by σ(-1.92), about 0.13. The residual MLP's output is unbounded, so I can't say from the weights alone how big the correction gets on a real input. But the card's own numbers suggest where the gain came from. Training the head alone on frozen embeddings scored 67.7%; unfreezing the encoder lifted it to 76.4%. Most of the work went into making the encoder's cosines sharper, with a head that barely opened its gate. The header totals 272,120,354 values: the 271.0M encoder (half of it, 134.6M, is the token embedding table, which phase 2 kept frozen), a head of 1,117,700 parameters, and the three temperatures. The card says the head is "~0.5M"; the file says about twice that. There's a deployment property here that the card doesn't mention. A case in the test split needs 22 to 26 texts embedded (one state, five questions, their options), and the test split has only four distinct schemas. Everything except the state is the same for every case of a workflow, so a server could embed the schema once and pay one 271M forward pass per new state. The shipped `decide()` doesn't cache across calls, it re-embeds the schema each time, but the design allows it, which an LLM-backed head never can. ### Checking the numbers The 76.4% is on the right split: `LocalLLaMA/typed-decisions`, subset `all`, the 400-case test set, argmax against the gold label. The per-type figures add up. Weighting 71.2% on 600 choice decisions, 74.0% on 800 score decisions and 84.8% on 600 yes/no decisions gives 1,528 of 2,000, which is 76.4%. Score questions are the weakest, and the card says why: numbers inside the state are hard for an embedding model. Two qualifiers belong next to it. It was trained on that benchmark's `train` split, so it belongs in the leaderboard's second table, the fitted models, not next to zero-shot rows like Jev's 0.727. And the card says plainly that "hyperparameters were chosen while looking at the test split, so treat these numbers as slightly optimistic". I respect that sentence; few cards carry it. The 27B LLM isn't named anywhere on the card. The only 27B on the benchmark's Hub leaderboard is [Darwin-27B-ZTC-v5](https://huggingface.co/FINAL-Bench/Darwin-27B-ZTC-v5), released on 11 October, the same day, at 0.812, and also fine-tuned on `train`. Against that the gap is 4.8 points, not six. The Hub counts 25.6B parameters in it, about 94 times this model's 272M, so "~100×" is fair. Its zero-shot sibling, Darwin-27B-ZTC, scores 0.743, which this model beats; the comparison only flatters the 27B when you pick the fine-tuned one, which is the right one to pick. Then there's the Laya row this site has quoted since [the relational-choice piece](/articles/jev-scores-zero): "Laya (fine-tuned), 2,000 typed decisions, 0.766". Same benchmark, same test split, same `train`-fitted table, same accuracy definition, both self-reported. So the two are directly comparable, and Typed Decision is 0.764 to Laya's 0.766, four decisions out of 2,000. It isn't the same number travelling under a new name; Laya is a 421M ModernBERT-large, this is EmbeddingGemma 2. It's a coincidence, and a gap that size is noise. In the same fitted table soft-decider-421m reports 0.774 and OpenDecider-nano, a 400M encoder, 0.796. Typed Decision is a respectable mid-table entry for its size, not a new high. ### What "calibrated" rests on The card reports a soft cross-entropy of 0.887 and a Brier score of 0.065, against 0.968 and 0.113 for the frozen-encoder phase. It reports no ECE and no KL. Calibration in the training recipe is one temperature per question type, fitted on held-out training rows, and the three stored in the file are 1.005 for choice, 1.144 for score and 1.056 for yes/no: the fitted model was already close to calibrated on held-out rows and needed only a slight softening of its score answers. The benchmark's leaderboard ranks KL from gold, not cross-entropy, but one converts to the other. KL is cross-entropy minus the gold distribution's own entropy, and I computed the mean entropy of the 2,000 test gold distributions: 0.767 nats. If the card's 0.887 is a mean over the same 2,000 decisions in nats, the KL is about 0.12. That puts it level with Aplomb 1 (KL 0.123, Brier 0.065 on its own submission) and behind the best fitted rows, which sit near 0.08 KL and 0.043 Brier. The Brier looks like the leaderboard's definition, squared error summed over options: the code's loss sums it that way, and the same definition reproduces the leaderboard's uniform reference of 0.238 on the test gold. I couldn't check the evaluation script itself; it isn't in the repo. The card's limitations section makes the point I'd want every decision-model card to make: on this benchmark "calibration means 'matches annotator disagreement'". The gold is the mean of three samples from a teacher, so a 43% answer where the teacher split 43/28/25 is correct, and the model is being calibrated against a roughly 4B-class teacher's uncertainty, not against the world. The benchmark's own README adds that its teacher only agrees with itself 73.5% of the time, which 76.4% is already above. Would I use it? For a fixed set of workflows on a phone or a CPU, yes, ahead of a prompted LLM: one forward pass per state, typed output that can't fail to parse, and Apache 2.0 like the base. The multilingual claim is backed by one probe, cosine between English/Spanish translation pairs moving from 0.869 to 0.901, which shows the encoder didn't forget Spanish, not that the decisions transfer; the card itself says the head was trained on English schemas only. I would not expect it to answer a question where one option's meaning depends on another; with every option pooled on its own, it has the same blind spot [Jev showed on relational choice](/articles/jev-scores-zero). ## An embedding model that transcribes, and what it actually runs *Added 13 October 2026.* On 12 October Shahir (@TslShahir), an on-device ML engineer, [posted](https://x.com/TslShahir/status/2109660368381395414) that he had "turned EmbeddingGemma 2, an embedding model, into a speech-to-text model", and that on a base M3 MacBook Air it "transcribes an hour of audio in about 13 seconds: 288× real time". The pitch is the one Google makes for the model: a spoken word and a written word land next to each other in the shared space, so you ought to be able to read a transcript straight out of it. His follow-up post gives the recipe in one line. The model stays untouched, and "a small reader on top takes each slice of audio, finds the nearest of ~229,000 words in the model's own text space, and strings them together." My first reaction was that this can't work as described. Embedding a 20-second clip with EmbeddingGemma 2 gives you one pooled vector, and one vector can't hold a sentence. So the reader has to be getting at something before the pooling. The code isn't public yet ("Soon", he replied when asked), but the free Mac app downloads its models from a public manifest, and Core ML packages describe themselves. I read those instead.