Edge0-145B: a Qwen MoE with its experts squeezed, and a 51B table left on the SSD

On 12 November Samuel Zeng announced Edge0-145B and Edge0-120B as "the first device-native 100B-class foundation models with open weights." The thread makes four claims in its first post. It beats Claude Opus 4.6 on Terminal-Bench 2.1, 86.5% to 78.2%. It needs 29 GB of memory, a third of a standard 100 GB setup. It decodes at 75+ tok/s and prefills at 1050+ tok/s "on RTX Spark and Mac". It holds 128K to 256K tokens of context without slowing down. A Japanese post by @_ryu15_ read the model cards and added what the thread leaves out: the base is Qwen3.8-Flash-Next, the weights ship as NVFP4 and Q3_K, the design assumes a 24 GB card with the rest offloaded, and about 40% of the model is "PLE".

This is the same company whose 35B release I took apart in September (Edge0 streams a 35B MoE off SSD). That release was a trained prerouter on Apple's MLX that read experts off the SSD a layer ahead. This one is a different idea on a different engine, and the claims are checkable the same way: four Hugging Face repos, one GitHub repo, six videos. I read the GGUF headers of every model file by HTTP range request without downloading a weight, compared tensors against the base model byte for byte, read the engine at commit da9cc52 (one day after the v0.2.0 tag the thread links), and watched the videos frame by frame.

What 145 billion parameters are made of

The repos do not ship safetensors. Each folder is a pack for a fork of the Strata engine: a model-skeleton.gguf with attention, routers, shared experts, embeddings and output head; an experts.bin of fixed-size expert records; two files of latent projections; a ple-fp8.gguf; an MTP draft layer; and a profile.bin that ranks experts for the cache. The two GGUFs have ordinary headers, so 11 MB of range requests gives every tensor name, type and shape. native_experts.txt gives the record size per layer. Summed:

The card's "145B" and "120B" are honest totals. They count the n-gram table, which is 35.5% of the 145B's parameters and 43.0% of the 120B's. By bytes the share is larger, because the table is FP8 and the experts are about 4 bits: ple-fp8.gguf is 51,200,246,112 bytes, 45.4% of the 145B NVFP4 repo's 112.87 GB and 55.5% of the 120B Q3_K repo's 92.28 GB. The Japanese post's "about 40%" is right on parameters and low on bytes.

The Hub's card for the repo shows 4.95B parameters, because the only file it can parse is the skeleton GGUF. That 4,947,698,560 is the first row of the table above, and nothing else.

The base, byte for byte

Qwen's card for Qwen3.8-Flash-Next lists 125B parameters plus 51B of N-gram embeddings, 512 experts, 10 active, moe_intermediate_size 640 and a hidden size of 2,560 (the site covered the architecture in Qwen3.8-Flash-Next: four changes and an honest report). Edge0's skeleton header has the same 48 layers, the same hidden size, the same rhythm of three DeltaNet layers and one attention layer, and an n-gram table with the same 320,001,536 rows as Qwen's 128 shards. So I compared tensors.

The bf16 router weights blk.N.ffn_gate_inp.weight for layers 0, 23 and 47 are byte-identical to model.language_model.layers.N.mlp.gate.weight in Qwen's own safetensors, all 2,621,440 bytes of each. So are the PLE key projection of layer 1 (52,428,800 bytes) and a hyper-connection matrix in layer 10. The n-gram table matches NVIDIA's FP8 copy in nvidia/Qwen3.8-Flash-Next-NVFP4: two 64 KiB windows, one at the start and one inside shard 77, and the scale, 0.00019931793212890625, to the last digit. The skeleton GGUF's own general.name is "Nvfp4 Src".

That is a sample, not a full diff; I did not compare the q8 attention weights, which are requantized. But the routers decide which experts fire, and they are Qwen's untouched. The 145B and 120B repos also share byte-identical model-skeleton.gguf, ple-fp8.gguf, mtp/, labels.bin and profile.bin by their LFS hashes. The two models differ only in experts.bin and the D/U files.

What Edge0 changed is the expert pool. Qwen's experts are 3 × 2560 × 640, 120.8B parameters across 24,576 of them. Edge0's are 3 × 1792 × 640 for the 145B and 3 × 1280 × 640 for the 120B: exactly 70% and 50% of the base's pool. On the convention the card uses, Qwen3.8-Flash-Next is about 176.9B, so Edge0-145B is its base minus 32.7B. The thread's chart of on-device model sizes over time puts Edge0 alone above 40B. It is the largest model on that chart because it is a compressed copy of a larger one, which is fair to say and is not what the chart says.

expand

A latent in front of the experts

Start from why an MoE is attractive on a small machine at all. A dense 145B model reads all 145B weights for every token. An MoE with 512 experts and 10 active reads 10 of them per layer, so compute and bandwidth scale with the active set while storage scales with the total. The storage is still there. What the engine can do is keep the experts that fire most where they are cheap to read and the rest somewhere bigger.

The cost of a miss is the size of one expert, and the size of an expert is 3 × d × f weights for a SwiGLU with input width d and hidden width f. Qwen's d is the model width, 2,560. Edge0's change, which the card calls a latent MoE, inserts a shared down-projection D in front of the experts and an up-projection U behind them, so the experts work at a narrower width ℓ:

Each layer has eight (D, U) pairs, and labels.bin assigns every expert to one of the eight groups. With ℓ = 1792 an expert is 3,440,640 weights instead of 4,915,200. The router still reads the full 2,560-wide hidden state; its weights are Qwen's.

expand

The reference kernel in edge0-strata/src/latent/latent_moe.cpp makes the price visible. D is one GEMM over all eight groups for every token; U runs once per group that any of the window's tokens routed into:

// D: [T,H] Q8 x [G*ELL,H]^T -> [T,G*ELL] (Q4DU: fmt 8 Q8_0 / 4 Q4_0, from Inputs)
quantize_q8_1_rows(in.x, in.T, (int) g.H, b.xq, stream);
iq_mmvq(strata::kernels::lat_dense_fmt_to_type(in.fmt), in.d_q8, b.xq, b.xl, (int) g.H, (int) (g.G * g.ELL), in.T, stream);

So the latent moves weight from the part of the model that is sparse to a part that is dense. D and U together are 2 × 2560 × 1792 × 8 = 73.4M parameters a layer, 3.52B over 48 layers, and nearly all of it is read for every token. The ten routed experts a layer reads are 34.4M. Per layer, the dense projections in front of and behind the experts are twice the size of the experts that fire.

This is also where two active-parameter counts come from. The thread and model.json say 6.6B active per token; the README and the model page say 10.12B. On the headers, the skeleton (4.948B) plus ten routed experts per layer (1.652B) is 6.600B, and adding the D/U projections (3.523B) gives 10.122B. The 120B works the same way: 6.13B and 8.64B. Both numbers are exact. The thread quotes the one that leaves out 3.5B of weights every token multiplies.

What is 4-bit

The README says both models are "trained natively at 4 bits". The record sizes say what ships. An NVFP4 expert record for the 145B is 1,935,376 bytes for 3,440,640 weights, 4.5 bits a weight (FP4 values with an E4M3 scale per 16, plus a 16-byte tail). A Q3_K record is 1,630,720 bytes, 3.79 bits a weight: native_experts.txt names the types as Q3_K for gate and up and IQ4_NL for down, and 985,600 + 645,120 bytes is exactly that record. The D/U projections are Q8_0 in the 145B's default and Q4_0 in the 120B's. The skeleton is Q8_0 and bf16. The n-gram table is FP8. "Weights 47.79 GiB" on the NVFP4 card is experts.bin plus latent_dense_q8.bin and nothing else: 51,307,216,896 bytes. The skeleton, the MTP layer and the 51.2 GB table are outside it.

Whether the latent experts were trained at 4 bits or quantized after training is not something the files can show. The release contains no training code, recipe or report for these models, and release-scope.json says in its own words that "No training-source commit or independent ownership audit is asserted." I have no reason to doubt it and no way to check it.

The 51.2B the GPU never holds

The "PLE" is Qwen's N-gram embedding, not a per-layer embedding in the sense Raon-1 nano or Gemma use the term. The skeleton header spells out the mechanism: ple.ngram_size 3, ple.heads_per_ngram 8, sixteen hash heads with vocabularies of about 20M rows each (20,000,003 for the first), embedding_length_per_layer_input 160, and ple.layers [1]. Each token hashes its last two and three tokens into sixteen rows of 160 FP8 values, and one layer near the bottom of the stack mixes them in through a key and value projection.

Sixteen rows of 160 bytes is 2,560 bytes a token. The engine reads them with direct I/O (--ple-io direct in every model.json), which works in 4 KiB pages, so the device sees about 64 KiB a token; at 75 tok/s that is under 5 MB/s. The table needs no memory at all, and the addresses are known as soon as the tokens are, so the read can start before the layer that needs it. This is the one part of the release where "it doesn't have to be in memory" is plainly true, and it was already true of the base model: SGLang moved the same table off the GPU on day zero.

It is also why "145B on a laptop" is less of a jump than the chart suggests. 51.2B of the 145B never occupy RAM or VRAM. The model that has to be resident is the other 93B, and the part of that which is read per token is about a tenth of it.

The engine computes the misses where they lie

The thread says the engine keeps "a few thousand frequently used experts on the GPU and stream[s] the rest from system RAM as needed." The engine does something better than that, and it is upstream Strata's design, which the site read in detail in Strata in the source. By default every expert the GPU does not hold sits pinned in host RAM, and the CPU computes those experts in place, in the same window the GPU computes its cached ones. Only a share of the misses crosses PCIe: --pcie-frac 0.1 in every Edge0 model.json, so 90% of the RAM-side experts never travel. The SSD holds the n-gram table. Experts come off the SSD only when you ask for it with --mmap-experts or --resident-budget-gib N, which the README offers as the low-RAM mode.

Two things decide the speed, then. How many bytes a verify window reads, and which lane is slowest. With --spec 4 the MTP layer drafts up to four tokens and the full model checks them in one pass. Four tokens route to about 3.05 times one token's distinct experts (Strata's own constant in verify.hpp), and the upstream measurements put the kept tokens at about 3.2 a window. For the 145B Q3_K that is:

• experts: 480 records × 1,630,720 B = 0.78 GB a token, about 2.39 GB of distinct experts a window;
• dense: 5.31 GB of skeleton (everything but the embedding table, which is one row's lookup) plus 3.74 GB of D/U, read once a window.

The four residency readings in the README make a useful cross-check on that inventory. They report VRAM and host RAM for each build with every expert resident, "disjoint expert sets, nothing counted twice". Subtract every weight byte except the n-gram table from VRAM plus RAM and what is left is 7.26 GiB for the 145B Q3_K (23.14 + 31.28 GiB), 7.39 for the 145B NVFP4, 7.00 and 7.11 for the 120B builds. The same 7 GiB of runtime in all four, which is what a correct inventory and a correct measurement should give. It also means the card's own measured footprint for the 145B is 54.4 GiB in Q3_K and 61.5 GiB in NVFP4.

The widget is the strata-engine article's lane model with the bytes swapped for Edge0's and an SSD lane added. Its CPU and PCIe speeds (25 GB/s each), the 6.3 ms of drafting and commit per window and the 50% share of bus bandwidth the GPU reaches come from Strata's paper as read there. The cache sizes are my reconstruction from those residency readings: about 11.9 GiB of experts fit beside 9.8 GiB of fixed weights and 1.5 GiB of buffers on a 24 GB card, 32% of the Q3_K pool. The one free parameter is the skew of expert use, and the release does not publish it: profile.bin is a ranking of all 24,576 pairs with no counts. At a Zipf exponent of 0.8 the model gives 107 tok/s on the RTX 5090 D with every expert resident (the README reports 99.7 chat and 121.2 agent), 43 under the 16 GiB + 20 GiB cap (36.8 and 44.9), and 39 on an RTX Spark (37.7 and 36.7). It overshoots the 120B on discrete cards by 20 to 45% and the M2 Ultra by about half, so treat it as a model of the 145B on NVIDIA hardware.

Now the two questions the claims raise.

A 24 GB card and an NVMe drive, no RAM tier. At 3.2 tokens a window, 75 tok/s leaves 36.4 ms of lane time a window. A 7 GB/s drive reads 0.25 GB in that, about 10.7% of the 2.39 GB of distinct experts a window needs. So 89% of the experts a window touches must already be in VRAM, from a cache that holds 32% of the pool. Under the skew that reproduces the published rows, a 32% cache catches about 73%. Gen4 NVMe random reads of 1.6 MB records will not run at the sequential rate either. The widget lands near 32 tok/s for this machine. This is the mode the JP post describes and the thread implies, and nothing in the release measures it.

With the experts in RAM, the CPU is the lane to beat. If every miss is in RAM, the CPU must compute 90% of them at 25 GB/s inside the same 36.4 ms, which needs a VRAM hit rate of about 58%. A 24 GB card with 64 GB of RAM clears that, and the README's 5090 D and 4090 rows (99.7 and 69.9 chat, 121.2 and 91.7 agent) agree. That is the real result of this release: 145B parameters at around 100 tok/s on one consumer card, with 54 GiB of VRAM and RAM between them. It just isn't 29 GB, and it isn't the SSD doing it.

Big MoE on one RTX 3090 found the same shape for DeepSeek-V4.1-Flash on a 3090, and runtime dynamic compression is the opposite bet: shrink the experts to 1.5 bits and swap the resident set every 256 tokens. Edge0 shrinks them by width instead of by bits and keeps all of them.

75+ tok/s, and on which model

The thread's speed line is "decodes 75+ toks/s, prefills 1050+ toks/s on RTX Spark and Mac." The README's speed tables give per-machine numbers for each build. For the 145B on an RTX Spark N1X: 36.7 to 46.0 tok/s decode and 674.0 to 922.4 prefill. On an Apple M2 Ultra: 60.9 chat and 57.0 agent decode, 755.6 and 800.3 prefill. No 145B row on either machine reaches 75 or 1050. The 120B's Spark rows reach 68.4 and 68.9 agent decode and 1081.7 and 1066.9 prefill at 16K, the closest any Spark or Mac row comes.

The videos show where the numbers came from. In the launch video and in the six-step setup video, the status page of Edge0 Chat reads edge0_145b_l1280 on CUDA · NVIDIA RTX Spark N1X, with 1061.4 tok/s prefill and 75.9 decode in one, 1056.8 and 76.9 in the other. l1280 is a latent width of 1280. The release's own model page says the 145B uses 1792 and the 120B uses 1280; the 145B NVFP4 repo's model.json names its model edge0-145b-fp4 and the 120B's edge0-120b-q3k. The engine's source settles what the suffix means. src/kernels/iq_multi_parity.cpp calls 1280 "the v7l1280 latent pack's ELL", ELL being the latent width, and src/core/verify.cpp records a hang on Windows on Arm "with the q3k l1280 pack, every expert resident", which is the RTX Spark N1X configuration. A build named 145B at the 120B's latent width is not one of the four released folders. It may be an internal name for what shipped as the 120B, or an earlier 145B. Either way the 75+ was measured on a 1280-wide model.

The prefill number has a second problem. The two prompts are 39 and 69 tokens. 69 tokens at 1056.8 tok/s is 65 ms, and the same panel says the first token took 1.2 s. Edge0's own benchmarking guide, written for the 35B, has the right sentence for this: "Prefill speed is not meaningful for a 19-token prompt, where fixed costs dominate." Prefill rates on a 16K prompt are in the README, and they are the ones to quote. To be fair to the build in the video, its longer run holds up: by the end of the launch video's Godot session the status page reads 20 requests, 712,000 input tokens, 1831.9 tok/s average prefill and 82.3 average decode. That is consistent with the README's 120B Spark rows. It is not a 145B number.

The bandwidth agrees with the README rather than with the thread. On a unified-memory machine there is no cache and no miss: a window reads 9.05 GB of dense weights and 2.39 GB of experts over one bus. The N1X's bandwidth is not published by Edge0; reporting puts it near 300 GB/s. At the 50% of bus bandwidth that reproduces the README's other rows, that gives 39 tok/s for the 145B and 51 for the 120B. At 100% of a 301 GB/s bus with the same drafting overhead, the 145B tops out near 72. The 120B's smaller D/U (Q4, 1.42 GB) and narrower experts are what make 76 possible on that box.

The Mac half of the claim has a different problem. The repository's platform table lists macOS as unsupported for both models, and the README says the M2 Ultra rows "do not establish macOS release availability." EDGE0.md records that the fork removed the upstream metal/ engine. A Mac user who downloads the release today has no engine for it.

"Steady at 128K to 256K" has no measurement in the release that I can find. Every model.json launches with --max-context 131072. The architecture makes long context cheap (three of four layers are DeltaNet with a fixed state, and the attention layers use a top-2048 indexer), so the claim is plausible for the base model's reasons. The speed tables stop at 16K.

What "29 GB" can contain

The thread's memory chart plots Edge0-145B and Edge0-120B both at 29 GB, against about 70 GB for Qwen3.8-Flash-Next and about 87 GB for DeepSeek-V4-Flash. Its footnote reads: "Baseline memory estimated for standard 4-bit deployment (to be measured); Edge0 is measured internal peak." The number appears nowhere in the four model cards, the model page, the engine docs or the repository. What the release does measure is 54.4 GiB for the 145B Q3_K with every expert resident, and a documented minimum of "16 GiB of GPU memory and 20 GiB of system memory", 36 GiB, under which the 145B Q3_K decodes at 36.8 and 44.9 tok/s.

expand

So 29 GB is a different accounting, and the arithmetic says which. The weights that have to be next to the GPU (skeleton, D/U, MTP) are 10.5 GB. Add the 7 GiB of runtime the residency readings show and about 11 GB is left for experts, under 30% of the Q3_K pool. The other 70% has to be somewhere that a "peak memory" figure does not count. The README defines its host measurement as RssAnon + RssShmem, which leaves out file-backed pages; experts read through --mmap-experts live in the page cache and are not in that sum. Whatever produced 29 GB, it was a run with most of the expert pool outside the measured number, and the thread does not say what that run's speed was. There is one place a number like it does appear. Edge0 Chat's status page has a "System memory" meter, and in the launch video it moves between 28.20 GB and 35.22 GB over the Godot agent session (20 requests, 712,000 input tokens by the end); the setup video's reads 26.84 GB after one short chat. 29 GB sits inside that range. Every one of those readings is the 1280-wide build, on a machine whose GPU and CPU share the memory the meter counts.

The fair comparison sits on this site from yesterday. llama-modchip runs DeepSeek-V4-Flash, 284B total, in 64 GB of RAM with nothing streamed from disk, by putting 225B of experts in IQ1_S at 1.56 bits; its tokens read 5.36 GB and decode at 8.75 tok/s on a CPU box. Edge0-145B with everything resident is in the same memory class (54 GiB plus a 51 GB file on the SSD), keeps its experts at 3.8 to 4.5 bits, and runs ten times faster because a 24 GB card does the dense half. That is a good trade. It is a 64 GB machine with a 24 GB card.

The benchmark table

The averages in the chart check out for three models. Edge0-145B sums to 436.8, a mean of 72.8; Qwen3.8-Flash-Next to 430.0, 71.67; DeepSeek-V4-Flash to 420.6, 70.1; Qwen3.8-27B to 396.7, 66.1. Edge0-120B sums to 405.3, a mean of 67.55, and the chart prints 68.9.

expand

That footnote does not survive a comparison with Qwen's own model card. On DeepSWE 1.1, IFBench, GPQA-Diamond and HLE, every baseline cell in Edge0's table equals Qwen's published table to the decimal: Qwen3.8-Flash-Next 58.7, 81.3, 91.7, 35.9; Qwen3.8-27B 42.2, 79.5, 89.2, 30.8; DeepSeek-V4-Flash-0731 54.4, 79.2, 90.8, 33.8; Claude-Opus-4.6 (Max) 62.5, 91.3, 40.0 and a dash for DeepSWE. Fifteen numbers and one blank, all matching. Qwen's card has no Terminal-Bench or AA-LCR rows, which leaves eight cells. Two of those turn up elsewhere: 83.3 for Qwen3.8-Flash-Next on Terminal-Bench 2.1 is the FP8 baseline in NVIDIA's card for its NVFP4 checkpoint, and 82.7 for DeepSeek-V4-Flash is the number in the IQuest-Q1 release chart. The chart's own footnote also says "Claude Opus 4.6 has no public DeepSWE 1.1 score", which is a reason to leave a cell empty only if the cells were filled from public scores. An independent evaluation that reproduces fifteen of another lab's numbers exactly would be remarkable. The likelier reading is that the baselines are published numbers, Qwen's for four of the six columns, the Edge0 rows are Edge0's, and the two were run under different harnesses.

That also answers the question of why the comparison is with Opus 4.6 in November 2026. Qwen's August card compared against Opus 4.6, and the column came with the numbers. On Terminal-Bench 2.1 the frontier has moved: the IQuest-Q1 release chart this site read puts Claude Opus 5 at 89.1, and that is itself a vendor's chart. Neither Qwen's card nor Edge0's release names the Terminal-Bench harness, timeout or attempt count, and the score swings by many points with the agent around the model; Qwen's DeepSWE footnote reports "the highest score across the two harnesses" it tried.

Read honestly, the table says something narrower than the thread. The 145B leads its own base model by 1.1 points of average, all of it from two columns: +5.9 on DeepSWE and +3.2 on Terminal-Bench. It is behind the base on IFBench, GPQA-Diamond and HLE. A derivative that keeps the base's routers and cuts 30% of its expert weights is not expected to gain 5.9 points on agentic coding from compression. If it does, that came from training the release does not describe, and the single-number scores do not say how many runs or which seeds. The 120B is 7.4 points below the base on DeepSWE and 8.0 on Terminal-Bench. Halving the expert width costs something real, and the 120B's rows are the more believable half of the table for exactly that reason.

The demos

The thread's four "local vibe coding" demos are agent sessions in Blender, a game engine, a video pipeline and KiCad with Unreal Engine. They are the best evidence in the release that the model is a usable coding agent, and the least specific about speed or settings.

None of the four shows a status page, a token count or wall time, and the thread does not say which build or which machine ran them. The one agent session whose numbers are on screen is the Godot game in the launch video, at 78.5 tok/s average decode on the l1280 build.

Where I land

Edge0-145B is a real piece of engineering on top of two other people's work. Qwen trained the model and its 51B lookup table. Strata built the engine that computes cache misses on the CPU in the same window as the GPU, which is why any of this runs at 100 tok/s on one card. Edge0's contribution is the latent expert pool: experts 30% narrower, a 24 GB card holding a third of them, and 3.5B of dense projections added to every token to pay for it. The README's own measurements, 99.7 to 121.2 tok/s on an RTX 5090 D with 54 GiB of VRAM and RAM, are a good result and an honest table.

The thread is a different document. Its speed and memory headlines come from a 1280-wide build on a unified-memory machine, on 39- and 69-token prompts, with a peak-memory figure that leaves most of the experts out. Its benchmark baselines are published numbers, Qwen's in four of six columns, under a footnote that says otherwise. Its "on a Mac" has no macOS engine behind it. If you have a 24 GB NVIDIA card and 64 GB of RAM, download the 145B Q3_K and expect the README's rows, not the thread's. If you have a 48 GB laptop, the 120B Q3_K is the one whose weights fit, and the 75 tok/s you saw in the video was probably that model.