# Alpie Seed 9B: what a fine-tune of Qwen3.5-9B changed, one bf16 step at a time
> Satyajit Ghana — Head of Engineering @ Inkers Technology
> canonical: https://ai.satyajit.si/articles/alpie-seed
> date: 2026-11-09
> tags: qwen, vision-language-models, fine-tuning, grpo, lora, gguf, on-device, small-models
On November 8, 169Pi, a research company in India, posted benchmark results for Alpie Seed, "our 9B multimodal reasoning model": 82.54% on MMLU-Pro, 84.32% on GPQA Diamond, 72.99% on LiveCodeBench v6 and 61.20% on SWE-Bench Verified. The weights are on Hugging Face under Apache 2.0, with GGUF files and an Ollama package. The post is careful about its own comparison: "These evaluations were not rerun under a common harness, so differences in methodology and configuration should be considered when interpreting the numbers."
Alpie Seed is not a new architecture. The model card says it is built on Qwen3.5-9B and "post-trained to strengthen reasoning, instruction following, coding, and visual understanding." That makes it checkable in a way most launches are not. The base model is public, so the question "what did 169Pi change?" has an exact answer sitting in two safetensors files.
So I compared them. I read the safetensors header of `169Pi/Alpie_seed` (revision `021da65`) and the four shard headers of `Qwen/Qwen3.5-9B` by HTTP range request, compared all 760 Alpie tensors against the base on sampled spans, hashed six tensors in full and decoded six more value by value from both files. I read the 19-page technical report in the repository, parsed the headers of all three GGUF files, pulled the Ollama manifests, and rendered every chat template with the released tokenizer. I did not run the model: the smallest useful file is 3.83 GB and the machine this was written on is shared and has no llama.cpp build.
Every claim carries a label. **Measured** means I counted it myself from the files. **Read** means it is in 169Pi's report, card or config. **Reported** means 169Pi or Qwen says it and I did not reproduce it. **Reasoned** means I derived it from those numbers.
## What is in the box
| | Alpie Seed | Qwen3.5-9B |
|---|---:|---:|
| parameters in the safetensors (measured) | 9,409,813,744 | 9,653,104,368 |
| language model, embeddings and head (measured) | 8,953,803,264 | 8,953,803,264 |
| vision tower (measured) | 456,010,480 | 456,010,480 |
| multi-token prediction head (measured) | none | 243,290,624 |
| dtype (measured) | all bf16 | bf16, plus 3,840 values in f32 |
| file | one 18.82 GB file | four shards |
The architecture is Qwen3.5-9B's, unchanged (measured: every shared tensor has the same name and shape, and `config.json` matches). That architecture is worth a paragraph, because it decides most of what follows. The language model has 32 layers in eight groups of four. Three layers in each group are Gated DeltaNet, a linear-attention layer that keeps a fixed-size recurrent state instead of a growing cache. The fourth is ordinary gated attention with 16 query heads, 4 key/value heads and a head size of 256 (read, from `config.json`: `full_attention_interval: 4`). Every layer has the same SwiGLU MLP, 4,096 wide with a 12,288 hidden size. The vocabulary is 248,320 tokens, so the input embedding and the untied output head are each 1,017,118,720 parameters: 21.6% of the checkpoint between them (measured).
The 8,953,803,264 language-model parameters are exactly the parameter count in the header of all three GGUF files (measured), so the GGUFs are text-only; the vision tower ships separately as a 921,704,384-byte F16 `mmproj` file.
## How 169Pi trained it
The technical report describes two stages, both as adapters on a frozen base (read):
1. **SFT.** A DoRA adapter, rank 64, alpha 128, on `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj` and `down_proj`. One epoch at 8,192 tokens, learning rate 2e-5, effective batch 32.
2. **GRPO.** A second DoRA adapter with RSLoRA, rank 64, alpha 32, described as covering "all linear layers", trained against the frozen SFT adapter as the reference policy, with the base loaded in 4-bit NF4. Eight completions per prompt, a 1,024-token prompt and 8,192-token completion limit, KL coefficient 0.12, 3,034 steps across five phases on 8 H100s: 434 wall-clock hours, about 3,472 GPU-hours. Phase 4 was released.
The reward is a weighted sum the report writes as R = 2.0C − 1.1H + 0.9S + 0.6E + 0.6F − 0.6Rep − 0.5Len: correctness, a hallucination penalty, self-consistency, "education quality", structure, repetition and length (read). The report says plainly that the training data is confidential and that the reward implementations are not published, so none of this can be reproduced.
DoRA is the detail that matters for reading the weights. It splits each weight matrix into a per-row magnitude and a direction, adds a low-rank update to the direction, and renormalises. Merged back, it is still one dense matrix of the same shape. A merged DoRA model therefore looks, on disk, exactly like a fully fine-tuned one. The only way to see what it touched is to diff it against the base.
## The diff
Comparing two 18 GB files does not need either file. Safetensors puts a JSON header at the front that gives every tensor's byte range, so any slice of any tensor is one HTTP range request away. I took three random 16 KiB spans of every Alpie tensor (the whole tensor when it was smaller than 64 KiB), fetched the same bytes from the base, and compared them as bf16 values. That is about 37 MB a side instead of 19 GB. Sampling can miss a change, so I then hashed six tensors in full and decoded six more in full from both checkpoints (measured).
Three things come out of it.
**Only the adapter's named targets moved.** Every MLP matrix in all 32 layers changed, and so did `q_proj`, `k_proj`, `v_proj` and `o_proj` in the eight full-attention layers (measured). Nothing else did. The Gated DeltaNet layers were bit-identical in every sample of all 120 of their projection matrices, and the three I hashed in full (`in_proj_qkv` of layer 0, `out_proj` of layer 17, `in_proj_z` of layer 30) have the same SHA-256 as the base (measured). So do the embeddings and the output head in 150 of 150 sampled spans each, all 333 vision-tower tensors in every sample, and two vision matrices hashed in full (measured).
The reason is a naming one (reasoned). A PEFT adapter attaches to modules whose names match its target list. In Qwen3.5 the DeltaNet layers' matrices are called `in_proj_qkv`, `in_proj_z`, `in_proj_a`, `in_proj_b` and `out_proj`, so a list of `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` (the SFT list, read) reaches only the 8 attention layers and the 32 MLPs. That is 5,301,600,256 parameters, 56.3% of the checkpoint (measured). The GRPO stage is described as targeting "all linear layers", which in PEFT's usual sense would include the DeltaNet projections; in the released weights they did not move (measured). Either the GRPO adapter in practice had the same target list, or its effect on those layers was not merged. The report does not say which.
The practical consequence: three quarters of the model's sequence mixing, the part that carries long-range context in this architecture, is exactly Qwen's. The vision tower is exactly Qwen's too, which the report states ("no vision-specific post-training in this release") and the weights confirm.
**Each changed value moved by exactly one bf16 step.** In the six matrices I decoded in full, between 4.4% and 20.6% of the values differ from the base, and every one of those differs by exactly one unit in the last place of bf16. Not one value, in about 159 million compared, moved by two steps (measured). The relative size of the whole change, the Frobenius norm of the difference over the norm of the base matrix, is between 0.09% and 0.22% (measured).
This is what you would expect from a small update merged into bf16 (reasoned). Bf16 keeps 8 significant bits, so neighbouring values are 0.39% to 0.78% apart. The base weights are already bf16, so each one sits exactly on a grid point; adding a small update and rounding back leaves it where it was unless the update is more than half a step, about 0.2% of the value. Most of the merged update is smaller than that, so most weights round back to themselves. The ones that cross round to the next step and no further, because no individual update reached one and a half steps. Across the whole checkpoint, scaling the sampled shares by tensor size, about 4.9% of all values differ from Qwen3.5-9B (measured, from the samples).
It also means the stored difference is mostly a rounding pattern. I took the singular values of the difference for all six full matrices, hoping to see the rank-64 adapters. The spectrum is flat: the layer-19 `o_proj` needs 962 of its 4,096 directions to hold 90% of the difference's energy (measured). Whatever low-rank structure the adapters had, the bf16 merge spread it into something that looks like noise, so the weights alone cannot tell you the adapters' rank.
Per output row, the DoRA magnitudes moved by at most about half a percent: the ratio of each row's norm to the base's runs from 0.994 to 1.006 in all six matrices (measured). Between 57% and 78% of rows have no changed value at all (measured).
**One small cast.** The base stores the 24 DeltaNet layers' `A_log` decay parameters and their output-norm weights in f32, 3,840 values. Alpie Seed stores them in bf16 (measured). The 768 `A_log` values happen to be exactly representable in bf16 and come through unchanged; the 3,072 norm weights are rounded, a relative change of about 0.15% (measured). It is harmless, but it is the only change anywhere in the layers the adapters never touched.
## What is missing: the multi-token prediction head
Qwen3.5-9B ships a one-layer multi-token prediction head, 15 tensors named `mtp.*` with 243,290,624 parameters (measured). Runtimes that support it use it to draft extra tokens for speculative decoding. Alpie Seed's `model.safetensors` contains none of those tensors (measured). Its `config.json` still says `mtp_num_hidden_layers: 1` (read), the model card's overview says "MTP: trained with multi-steps", and the technical report's architecture table lists "One-layer multi-token prediction (MTP) head, relevant to runtimes supporting speculative decoding" (read).
The likely cause is mundane (reasoned): the MTP head is not part of the module tree that `transformers` builds for inference, so a save from a loaded model drops it. The effect is real, though. A server configured to use Qwen3.5's MTP head for speculative decoding has nothing to load for this checkpoint, and the base head would not match the fine-tuned trunk exactly anyway. The Q2_K GGUF's metadata shows the same confusion: it declares 33 blocks and `nextn_predict_layers: 1`, while its tensors stop at block 31, the same 8,953,803,264 parameters as the other two files (measured).
## Four chat templates, and a system prompt in every one
The report says "Base architecture retained; original chat template unchanged" (read). The architecture part holds. The template part does not, and it differs by file.
I rendered each template with a single user message, "Hi", and counted the tokens with the released tokenizer (measured; the tokenizer itself is Qwen3.5's, with the same vocabulary and merges, plus seven audio and TTS special tokens the base's tokenizer file does not declare).
| template | rendered prompt for one "Hi", in tokens | what it injects |
|---|---:|---|
| Qwen3.5-9B, base | 11 | nothing |
| Alpie Seed, Hugging Face | 106 | an identity paragraph in a system turn |
| Alpie Seed GGUF Q4_K_M | 135 | two identity paragraphs, one of them outside any turn |
| Alpie Seed GGUF Q3_K_M | 134 | the same, in a stripped-down ChatML template with no tools and no thinking switch |
| Alpie Seed GGUF Q2_K | 2,161 | the identity paragraph and a long policy |
All measured. The Hugging Face template always opens a system turn with "You are Alpie Seed 9B, a model built by 169Pi, an Indian research company" and instructions on how to answer questions about its identity, followed by any system message you supply. It also refers to a variable named `india_position_policy` that it never defines, which Jinja renders as an empty string.
The Q2_K GGUF defines it. Its template, 19,513 characters, prepends a section headed `INDIA-ONLY POSITION POLICY` to every conversation, with or without a system message (measured, from the rendered prompt). It tells the model to state India's documented official position on territory, borders and diplomacy "first and only", not to give other governments' positions unless a comparison is asked for, and it lists required positions on named places and disputes. It also contains accuracy instructions that read like fixes for specific past errors, for example about the Indus Waters Treaty's western rivers and the UN sanctions committee's consensus rule. Whatever one thinks of its content, it is a 2,000-token instruction applied to every request, including ones that have nothing to do with India, and it costs the same prompt-processing time as a short document on every turn.
The Q4_K_M and Q3_K_M templates do something different and smaller: both put an identity sentence as raw text in front of the first `<|im_start|>`, outside any role, and the Q3 template drops Qwen's tool-calling and thinking-mode logic entirely (measured). They name the model "Alpie 9B", and the Ollama packages then add a system message telling it to "Always identify yourself as Alpie Seed 9B, never as Alpie 9B" (read, from the Ollama manifest blobs). The Ollama packages carry no template of their own, so which template Ollama ends up applying depends on how it handles the one embedded in the GGUF; I did not run Ollama, so I cannot say which prompt it actually sends.
If you deploy this model, render the prompt your runtime builds and read it. Three files from one release give three different first turns.
## Running it locally: what the memory budget looks like
169Pi's deployment story is local inference, and the hybrid architecture is a real help there. Only the 8 full-attention layers keep a KV cache. At 4 KV heads of 256 dimensions, that is 32,768 bytes a token at 16 bits (reasoned, from `config.json`). The 24 DeltaNet layers keep a fixed state of about 53 MB however long the conversation runs (reasoned). If all 32 layers kept a cache, a token would cost four times as much.
The file sizes are measured: 5,629,109,184 bytes for Q4_K_M (5.03 bits a weight), 4,623,517,376 for Q3_K_M and 3,827,273,728 for Q2_K. The SHA-256 prefixes of those three files on Hugging Face match the Ollama blobs and the hashes printed in the report (measured).
Two things in the budget are worth knowing before you pull the Ollama package. Its parameters blob sets `num_ctx` to 262,144 and `temperature` to 0 (read). At the full context, the KV cache alone is 8.59 GB at 16 bits (reasoned), more than the Q4 weights; whether Ollama allocates all of it up front depends on the version and the device. And the report's own long-context profile grows faster than the cache does: peak GPU memory goes from 17.865 GB at 1K tokens to 26.252 GB at 32K (reported), about 264 KB a token, roughly eight times the KV cache (reasoned). That points to prefill activations in their harness, not to the cache, and it means the profile measures that harness rather than the model's floor.
The device table in the report gives prompt and generation rates for a Mac M2, an NVIDIA RTX PRO 4500, an iPhone 17 Pro, a Samsung Galaxy Z Flip8 and a Windows CPU-only machine, from 1.95 to 110.66 generated tokens a second (reported). The report is candid that these are "reported configuration measurements, not a controlled cross-device benchmark." One row needs more care than that: the RTX PRO 4500 is listed with a 5.24 GB artifact and 1,148 MB of GPU memory (reported). A 5.24 GB file cannot sit in 1.1 GB of VRAM, so either most layers ran elsewhere or the figure counts something other than the weights (reasoned).
## The benchmarks against the base
The comparison that matters for a fine-tune is against its own base, not against Gemma or GPT-OSS. Both columns below are reported: Alpie Seed's by 169Pi (FP16 checkpoint, thinking on, temperature 0, up to 65,536 output tokens), Qwen3.5-9B's from the Qwen model card.
Read with the band set to two points, the text results split three ways. MMLU-Pro is flat: 82.54 against 82.5. GPQA Diamond is up 2.62 points. LiveCodeBench v6 is up 7.39, the largest text gain, and the one most consistent with a GRPO stage that rewards correct answers. On SWE-Bench Verified, Alpie's card lists Qwen3.5-9B at 60.0 next to its own 61.20, but that figure is not on the Qwen3.5-9B card, and 169Pi's own report lists Qwen's SWE-Bench as "Not reported" (read). The same goes for the AIME 2026 base figure of 69.4. Treat both base numbers as unsourced.
The vision results are the interesting ones, because the vision tower is bit-identical to the base (measured). Five of six move by less than 2.2 points in either direction. HallusionBench is up 10.58 points, which would have to come from the language model's answers changing, not from what it sees. That is possible: HallusionBench checks whether a model asserts things an image does not support, and the reward has a hallucination penalty. It is also the kind of gap a different prompt or parser produces. Without a same-harness run of both models, there is no way to tell which.
The report itself says the comparison is "descriptive, not same-harness controlled," and it lists the experiments that would settle it: a benchmark manifest with prompts and raw outputs, and matched FP16 and quantized runs (read). The quantized files have no benchmark scores at all; the headline numbers are for the FP16 weights only (read).
## What else the report covers
The 19-page report goes further than most small-lab releases, and some of it is useful on its own terms (all reported):
- **NF4 retention.** On a 295,893-token text corpus, NF4 raised perplexity from 12.7178 to 13.2436 (+4.13%), with 88.40% top-1 agreement and a mean KL of 0.062325 against BF16.
- **Indic languages.** MMLU Indic 73.67% and MILU 73.01% overall across ten languages, Hindi highest on MMLU Indic at 77.55% and Tamil lowest at 68.10%.
- **OCRBench by category.** 100.0% on regular text down to 66.0% on handwritten mathematical expressions.
- **Safety.** StrongREJECT on 96 prompts across six attack types, refusal rates of 99-100%, with an explicit note that this does not establish jailbreak immunity and did not measure over-refusal.
- **Inference profile.** 36.66 ms to first token, 41.37 tokens a second and 9,796.22 prompt tokens a second, on a setup the report does not describe.
The report also states its limits more clearly than the launch post does. It says the 1.01M-token YaRN context is "support claim only, not validated", that the vision encoder was not post-trained, and that the agent demos "are not all direct tests of the released checkpoint" (read).
## What holds
- **Holds:** a 9B open-weight vision-language model under Apache 2.0, with 4-, 3- and 2-bit GGUFs whose hashes match the report (measured). The architecture and context window are Qwen3.5-9B's, unchanged (measured).
- **Holds, with a qualifier:** "post-trained with SFT and GRPO". The weights show a light merged update to the MLPs and the eight attention layers, about 4.9% of values moved by one bf16 step (measured). The DeltaNet layers, the embeddings and the vision tower are Qwen's, byte for byte.
- **Does not hold:** "original chat template unchanged". Every file injects an identity prompt, and the Q2_K file injects a 2,161-token policy (measured).
- **Does not hold for the released file:** the MTP head the config and report describe is not in the checkpoint (measured).
- **Unverified:** every benchmark score. The gains over the base are 169Pi's numbers against Qwen's, from different harnesses, and two of the base numbers in the card have no source I could find.
If you want a Qwen3.5-9B with a GRPO pass on top for code and science questions, Alpie Seed is a reasonable thing to try, and the Q4_K_M file fits an 8 GB GPU with room for a moderate context. Before you deploy it, render the prompt your runtime builds, decide whether you want the identity and policy text in it, and do not count on MTP speculative decoding. If you want to know whether the GRPO stage helped, the experiment is cheap: run Alpie Seed and Qwen3.5-9B through the same harness on LiveCodeBench and HallusionBench. I found no published run of that.
For other pages here that read a fine-tune against its base by range request, see [Drex Max](/articles/drex-max-v1), which measured how far a Qwen fine-tune moved from its base, and [WeVisOne-VL-4B](/articles/wevisone-vl-4b). [Bonsai 2](/articles/bonsai-2-small) covers where the bytes go in small Qwen3.5 models, including the same 248,320-token vocabulary.