ARC-1.0: the router linked under a 1.7B agent clip, and why it cannot route to that model yet

On 11 November @0xSero posted a two-minute screen recording of a small model doing chores: "This is a 1.7B param model btw, using the browser, making html files, and managing itself in Pi. This model hits 4o levels on most benchmarks lol." The post did not name the model. Under it, Rayline's CEO David Gilmore replied with a link to rayline-ai/ARC-1.0 and a plan to train "hybrid edge/cloud semantic routing for it in our next ARC checkpoint."

So the first thing to settle is which model is in the video. It is not ARC. Pi's footer in every frame reads models/LFM3-1.7B-MLX-8bit-4d6d94c9, and the launcher window beside it is titled LFM3-1.7B, with "Pi is open · MLX · 256K" under the name. That is Liquid AI's LFM3-1.7B, quantized to 8 bits for Apple's MLX. ARC-1.0 cannot do anything in that video. It has no generation head. It is a 0.8B-class classifier that reads an agent's conversation and predicts which model should handle the next step.

The two belong in one article anyway, because they are the two halves of one design. A 1.7B model that can hold a tool loop on a laptop is cheap enough to run for everything. It is not good enough to run for everything. Something has to decide, step by step, when the local model will do and when to pay for a cloud model. ARC is a serious attempt at that something. Below I go through what the clip shows the small model doing, why that works at 1.7B and where it stops working, and then what ARC actually is in its files, what its decision rule does, and whether its headline numbers hold.

What the 1.7B model in the clip actually did

I stepped through the full-resolution video frame by frame. The task arrives mid-session. Before the clip starts, the model has already been browsing a YouTube channel through a browser skill Pi calls sitegeist, and has told the user it cannot get weekly analytics because they "require an authenticated YouTube Data API access token". That is a correct refusal. The user then types: "when done make and open an html file which breaks down the performance metrics and all".

The model's reasoning (Pi shows it in italics) plans the sections, then says: "There's no explicit 'open' tool, but writing to a path makes it exist; I can mention opening." It calls Pi's write tool with a 60-line HTML file. It then reasons that it "could run bash to check, but write already confirmed," runs ls -l /Users/sero/performance-breakdown.html anyway, and gets back a 4,543-byte file. Its final message summarises the page: the channel, 29.2k subscribers, two recent videos with view counts, the playlists, and a note on why true weekly numbers are not on the public site.

That is a real agent loop: read the request, plan, call a tool with a well-formed argument, verify with a second tool, report. Two smaller things in the same frames are worth looking at.

1. It did not open the file. The user asked for "make and open"; the model decided writing was enough, and the user had to type "open in brave when done". Pi has a bash tool, so open -a "Brave Browser" was one call away.
2. The CSS it wrote sets border-bottom: 2px solid #0xSero;. That is the username pasted into a hex colour. Browsers drop the declaration silently, so the page renders and nobody notices.

Neither breaks the demo. Both are the kind of error a small model makes and a larger one usually does not, and each would compound if the next step built on it. One thing I expected to be an error was not: the two "recent videos" are listed as "19 yr ago" and "20 yr ago", which looked like a misread date, but the first one's YouTube page gives an upload date of 23 February 2007. The model read the page correctly. Hold that thought for the section on twenty steps.

Pi's footer also says something about cost. By the end of the clip it reads ↑292k ↓7.8k 10.9%/262k: by my reading of the footer, about 292 thousand input tokens sent over the session and 7.8 thousand generated, with the current context about 11% of a 262,144-token window. An agent loop resends its history on every call, so input outnumbers output by more than 30 to 1 here. On a cloud API that ratio is the bill. On a laptop it is prefill time, and the launcher's counter shows prefill at about 1.4K tokens a second. ARC's usage heads exist because of that ratio.

Why a 1.7B model can hold a tool loop at all

A tool loop asks three things of a model, and none of them is knowledge.

The first is format. The harness parses the model's output to find tool calls, so one malformed bracket is a failed step. LFM3's chat template emits calls as XML, <tool_call> around a <function=…> block with one <parameter=…> per argument, and the card tells the harness to feed results back as tool messages. A fixed, rigid grammar is easy to learn at small scale. It is pattern completion with a tight distribution, and it is what supervised fine-tuning on tool traces is best at. Liquid's card says post-training included "1.75T tokens across various tasks, including chat, reasoning, tool calling, and agentic traces," and names the harnesses it was tuned for: "Hermes Agent, Pi, and OpenClaw." The model in the clip is not improvising Pi's protocol. It was trained on it.

The second is choosing the right tool and filling its arguments from the conversation. That is a short-range decision over a list in the prompt. LFM3 adds deferred tools: definitions stay out of the prompt until the harness's own search tool returns their names, so a 1.7B model sees a handful of schemas instead of a hundred. Most of the card's tool-use rows are strong: BFCL V4 71.39 against Qwen3.5-9B's 62.83, ToolSandbox 80.36 against 76.41. MCP-Atlas is the exception, 32.57 against 42.67.

The third is holding the thread over a long context. Here LFM3's architecture does real work. As my earlier piece on LFM3 counted from its config, only 6 of its 24 layers attend over the whole context; 13 are short convolutions and 5 are sliding-window attention over 256 tokens, so the cache at long context is about a quarter of a full-attention stack's. That is why a 256K window fits on a laptop at all.

Everything else is the harness. Pi decides what the tools are, runs them, truncates their output, shows the reasoning, keeps the transcript. The model never touches the filesystem; it emits a string, and Pi's write turns it into a file. The tokin piece has a closer look at pi as a training target, including what its requests carry, and PINE is a harness built on pi's SDK. A small model in a good harness looks much more capable than the same model in a bare chat window, because most of the competence in the clip is the harness's.

Other small agent models on this site make the same trade from different directions. Raon-1 nano is 3.1B on paper and 1.5B of arithmetic, built for on-device tool use. The tiny browser models go to the other extreme: tens of thousands of parameters, one task each, no loop at all.

"4o levels", checked against what GPT-4o published

The post says the model "hits 4o levels on most benchmarks." Two problems before any number.

LFM3's model card does not mention GPT-4o. Its 29-row table compares LFM3-1.7B with LFM2.5-2.6B, MiniCPM5-2B, Gemma 4 E4B, Qwen3.5-4B and Qwen3.5-9B. So "4o levels" is the poster's comparison, not Liquid's, and there is no table that makes it.

And to compare against GPT-4o you need GPT-4o's numbers from a primary source on the same benchmark. OpenAI's own simple-evals README lists three GPT-4o snapshots on MMLU, GPQA, MATH, HumanEval, MGSM, DROP and SimpleQA. Of those seven, LFM3's card reports exactly one, GPQA Diamond (simple-evals runs the diamond variant by default; I checked gpqa_eval.py).

On the one benchmark that can be checked, the 1.7B model is ahead of every GPT-4o snapshot by 10 to 17 points. That needs a large asterisk. LFM3 reasons by default (its card: "Reasoning is enabled by default"), and GPT-4o is a non-reasoning model, so this is a thinking 1.7B against a model that answers directly. Both numbers are self-reported by their publishers with different harnesses. And GPQA is a multiple-choice test with a 25% floor, where reasoning traces help a lot.

The rest of "most benchmarks" cannot be checked from primary sources, because the benchmarks do not overlap. The knowledge rows LFM3 does report are the weak ones for a 1.7B model: Humanity's Last Exam without tools 7.17, AA-Omniscience accuracy 7.07. I would not expect it to match GPT-4o on MMLU or SimpleQA, which measure stored facts, but I have no number that shows it either way. Liquid's own guidance is plainer than the post: "It is not recommended for agentic coding or complex multi-step reasoning due to its small size." My LFM3 piece has the full recount of its card against Qwen3.5-9B: 14 wins and 15 losses, with the wins in agent and instruction-following rows and the losses in knowledge and long-context retrieval.

Twenty steps at 95%

The reason Liquid warns off multi-step work is arithmetic. If each step of an agent run succeeds independently with probability $q$, a run of $n$ dependent steps succeeds with

$P(\text{run}) = q^{\,n}.$

That is pessimistic, because real agents recover from some errors, and optimistic, because a wrong value in step 3 often poisons step 9 rather than failing on the spot. As a rough guide it is good enough.

A model that gets 95% of steps right finishes a 20-step task about 36% of the time. Going from 95% to 99% per step more than doubles that, to 82%. The gap between a small model and a frontier model shows up far more in the second column than in any single-turn benchmark. The clip is roughly a ten-step job, and it had two small errors in it. Its Terminal-Bench 2.1 score on the card, 12.92, is what the long-horizon version of the same model looks like.

So the useful question to ask of a small model is whether it is good enough for this step. A tidy file write after a verified browse: probably. Planning a multi-file refactor: probably not. That per-step judgement is what a router is for.

ARC is a router, and 0.8B is generous

ARC (Adaptive Routing Controller) reads an agent's conversation so far and outputs numbers about it. It generates no text. Its model card and technical report describe five trained signals: each candidate model's probability of success, a reasoning-quality score per reasoning setting, the expected token usage split four ways (uncached input, cache reads, cache writes, output), PII presence and type, and request category. A sixth path, the question head, answers developer-written multiple-choice questions about the conversation, the way Jev-style decision models do.

expand

The files say more than the diagram. I range-read the safetensors headers of all three weight files without downloading them.

model.safetensors holds 615,054,751 parameters in 421 tensors. Of those, 254,279,680 are the token embedding (a 248,320-token vocabulary at width 1,024) and 249,056,160 are Qwen3.5-0.8B's text layers 0 to 11, and only those: the release's NOTICE says they are unchanged from the base, and provenance.json says "layers 0-11 of the base, unchanged". The pattern in the header is three Gated DeltaNet (linear_attn) layers to one full-attention layer, at layers 3, 7 and 11. Another 100,592,896 are the base's full vision tower, also unchanged, so ARC can read screenshots in a conversation. What is left, 11,124,991 parameters, is everything Rayline trained for routing.

The question head lives in a separate folder. upper.safetensors is layers 12 to 23, fine-tuned, 249,056,160 parameters stored in float32, which is why the file is 996 MB. pointer.safetensors adds 524,289 more: a query and key projection and a learned scale that point at the answer option. All three files together are 864,635,200 parameters. The routing path a request takes on every call runs half the backbone: 503,336,864 text parameters plus the heads. "0.8B decision model" describes the base it was cut from more than what runs.

The routing heads are small and peculiar. Most of stage_one is stored in float64, which is unusual for weights and suggests a fitting pipeline outside PyTorch's usual habits. Its tensors have a leading dimension of 5, matching the report's "five independently fitted members form the selector ensemble", and an inner dimension of 56, which I read as the number of candidates the selector scores; w_usage is [5, 56, 4, 1024], five members by 56 candidates by four token components by the encoder width. Separate tensors carry per-candidate calibration (cal_a, cal_b, a Platt flag) and usage bounds (low, high). stage_two, which picks the exact arm of the chosen model (provider route, reasoning level, budget), indexes 5,733 arms built from 23 model vectors, 8 effort vectors and 9 thinking vectors.

provenance.json names eleven models the release can route among: Claude Haiku, Sonnet and Opus 5.5, DeepSeek V4.1 Flash, GLM-5.3 and its Flash, GPT-5.6 Terra, GPT-6 Astra, Luna and 6.1 Sol, and Kimi K3. No model in that list fits on a laptop. LFM3 is not in it. Adding a candidate means new evidence and a refit, which the report says plainly: "Adding a candidate requires new evidence and refitting". That is the sense of Gilmore's reply. Routing to a 1.7B local model is planned for a later checkpoint. The goldens' illustrative price card also carries eleven withheld-model ids, and stage_two has 23 model vectors, so the catalogue is larger than the named list; nothing in the release says any of the withheld entries is a laptop-sized model.

The same file says where the training signal comes from: 187,825 rows to fit the selector, and the report says Workshop, Rayline's own agent product, "supplies approximately 90% of the training weight." The other tenth is verifiable agent tasks and Workshop Arena comparisons, which the report describes as several agents run on one request and judged by people. The per-benchmark task-group counts in provenance.json are where I slowed down: 93 DeepSWE groups, 89 Terminal-Bench groups, 107 SWE-Atlas QA groups and 198 GPQA Diamond groups, all benchmarks it is later evaluated on. GPQA Diamond has 198 questions (I counted the rows of the CSV simple-evals loads), and Terminal-Bench 2.1 has 89 tasks (the count the tokin piece uses). So the selector's fit data has as many groups as those benchmarks have tasks. The report does not say how evaluation tasks were kept out of the fit, by cross-fitting or otherwise, and I could not check it. Until it does, I read the benchmark tables as in-distribution results.

Price goes in at decision time

The design choice I like most in ARC is that it never learns a price. It predicts the work in tokens, and the caller multiplies by its own rates. The report's Appendix A writes the rule. For candidate $m$ with predicted success $\hat p(m)$, predicted token counts $\hat n_m$ (four components) and prices $\rho_m$ in dollars per million tokens:

$\hat C(m) = \frac{\rho_m^{\top}\,\hat n_m}{10^6}, \qquad m^\star = \arg\max_{m}\ \bigl[\,V\,\hat p(m) - \hat C(m)\,\bigr].$

$V$ is the dollar value of a success. The two presets in presets.json are Sage, value_usd: 4.5, and Swift, value_usd: 0.1; each also stores a cost_weight that is exactly $1/V$ (0.2222 and 10.0), because the selector actually scores $\hat p - \hat C / V$, which ranks candidates identically. At $V = 4.5$, one percentage point of predicted success is worth 4.5 cents, so a model that is 3 points more likely to succeed is worth 13.5 cents more per task. At $V = 0.1$ a point is worth a tenth of a cent and the cheapest adequate model wins nearly always.

The prefer-local policy is a gate in front of the same rule: if the local model is loaded, fits and has $\hat p \ge 0.85$ (the card's threshold), use it; otherwise run utility over the cloud models. Two further terms in the appendix handle what a chat router never sees. A switching cost charges a new model for rebuilding the cached prefix at a user turn, $L_t\,[\max(\rho_{in}, \rho_{write}) - \rho_{read}]^{+}/10^6$ for a prefix of $L_t$ tokens, so the router does not hop models mid-task for a fraction of a point. And a separate rule picks the reasoning setting of the held model on every call, with its own cost weight.

The widget runs the rule on three made-up candidates. The local model costs nothing per task (I ignore electricity and the laptop), a mid-tier cloud model costs 8 cents and a frontier one 90. Move $V$ and the predicted success of each.

Two behaviours fall out. Under utility, with $V$ at Sage's 4.5, a frontier model 5 points better than the mid tier is worth 22.5 cents and costs 82 cents more, so the mid tier wins; push $V$ up past about 16 and the frontier model takes over. Under prefer-local, the local model's predicted success is the only thing that matters until it drops under the threshold, and then the local model is not even a candidate. That second rule is crude on purpose: it is a privacy and latency preference wearing a probability, and it makes every call's routing as good as ARC's estimate of $\hat p$ for the local model and no better.

Checking 97.5% and 92.9%

The card's headline: "Across six benchmarks, the paper's hybrid configuration retains 97.5% of Opus 5.5's quality at 92.9% lower cost." The hybrid configuration is prefer-local at threshold 0.85, with $V = 4.5$ for fallback, and with DeepSeek V4.1 Flash as the sole local model, "served locally across multiple NVIDIA DGX Spark systems." The card says quality "averages per-benchmark ratios" and cost "pools spending across matched tasks."

expand

I recomputed the quality figure from that table. The six ratios are 1.034, 0.998, 1.016, 0.825, 1.034 and 1.006. Their mean is 98.5%, not 97.5%. Capping each ratio at 100% (so a benchmark where ARC beats Opus earns nothing) gives 97.0%. Pooling the success rates first gives 99.8%. The cloud-only Sage table, by the card's stated method, gives 97.4%, the nearest of the four to the headline, but the headline names the hybrid configuration. None lands on 97.5% exactly. The difference is a point either way, so this is not a scandal. It does mean the headline number is not reproducible from the published tables with the published method, and I would like to know which run it came from.

The two tables also disagree on the baseline. Opus 5.5 scores 91.0 on Terminal-Bench 2.1 and 89.9 on GPQA Diamond in the Sage table, and 90.7 and 89.5 in the prefer-local table, on what the report calls "the same tasks." Repeated Opus runs would explain that, and it suggests the two comparisons each have their own Opus run; the card does not say.

The 92.9% cost figure is pooled, which means it is weighted by how much each benchmark costs. The unweighted mean of the six cost percentages is 24.8%, a 75.2% saving. The pooled number is so much better because the most expensive benchmark is also the one sent almost entirely to the local model. The report's Figure 1 gives Opus at $2.606 per DeepSWE task and $0.327 per Terminal-Bench task. DeepSWE goes local 98.2% of the time and costs 1.1% of Opus; Terminal-Bench goes local 10.8% of the time and costs 75.7%. Pool just those two and ARC spends about 9.4% of what Opus does. A benchmark mix with fewer long, expensive coding tasks would show a smaller saving. Neither framing is wrong; they answer different questions, and the headline picks the kinder one.

Then there is what "local" costs. In this evaluation a locally served model is priced at whatever the configured self-hosting cost is, and the card does not give the figure. The local model is a frontier-class open model spread across several DGX Spark boxes. That is a fine setup for a company. It is not the laptop in the clip, and none of these numbers says anything about a 1.7B model.

The row that deserves the most attention is Tau3 Banking: Opus 42.8, prefer-local 35.3, Sage 33.2. It is the one customer-service benchmark in the set, and the router loses 7.5 and 9.6 points there. A router that keeps quality on code and drops roughly a fifth of it on the conversational benchmark is telling you its training data, which the report says is mostly coding and agentic work from Workshop.

Swift has an oddity of its own. In the report's Figure 8, ARC Swift matches Haiku 5.5's success to the decimal on four of six benchmarks (33.3, 71.5, 81.2 and 64.8) at 85% of Haiku's cost. Haiku is only 25.1% of Swift's picks overall, so matching it exactly four times is surprising. I cannot tell from the report whether those are the same tasks rescored or a coincidence of small task counts.

What the question head is for, and where it slips

The question head answers multiple-choice questions about the conversation: "Does this request need a stronger model?", "Will the proposed model succeed?" On Rayline's internal Router Question Bench of 3,056 questions it agrees with the target 56.2% of the time against 32.7% for TypeSafe's Jev 1.14, with a calibration error of 0.079 against 0.31.

expand

Two cautions. The benchmark is Rayline's own, built from the same kind of agent trajectories ARC was trained on, so a large win against a general-purpose decision model is expected. And the last panel goes the other way: ARC flips its answer when the options are shuffled 19.0% of the time, Jev 9.3%. For a router that is not a cosmetic number. The Sage, Swift and prefer-local policies score candidates through stage_one's learned vectors and never see a list, so this does not touch them. It matters if you route through the question head: ask it "which of these models should answer?" and the order your code lists them in changes the answer about one time in five. AgentJev-0.6B is a whole article about removing that dependence.

On the broader Decision Index 0.3 panel, ARC's overall skill is 46.9, below both Jev 1.14 (60.7) and StartLux 0.8B (53.8), and its knowledge score is 24.6. That fits what it is, a specialist trained to judge agent work.

provenance.json adds one more detail the card does not mention. Of the six request domains the classifier knows, five carry "low_support"; only software is "served". Four of the nine operation classes are low-support too. The topic heads are, in effect, a coding classifier with placeholders for the rest.

Can you run it

The weights are Apache 2.0, which is unusual and welcome for a router. Running them is another matter. The card is direct: "The public vllm-arc plugin is a work in progress," end-to-end serving and the decision and question APIs "are not yet supported," "Stock vLLM does not load this architecture by itself," and it says to use the hosted API for an operational integration. The goldens in the release were recorded on an NVIDIA L4 under a patched vLLM. So today the open release is something you can inspect and score with, and the product is the hosted arc-sage-1.0 and arc-swift-1.0 behind Rayline's API. Raydock, the edge-to-cloud gateway, is the piece meant to sit between a harness like Pi and a local model.

A routing call on every agent step also has its own cost. Running half of a 0.8B backbone over a growing conversation is cheap next to a frontier model's prefill, and the report describes keeping the encoder's attention and recurrent state between calls so only new blocks are encoded. On a laptop already busy prefilling at 1.4K tokens a second for the local model, a second model reading the same context is not free. The report gives no latency figures for the router itself, so I can't put a number on it.

Where I land

The clip shows a 1.7B model doing a real ten-step job in a real harness, with one silently broken line of CSS and one ignored instruction. That is about what Liquid's own card predicts, and it is impressive for the size. "4o levels on most benchmarks" is not a claim anyone can check: the only overlapping benchmark with OpenAI's numbers is GPQA Diamond, where a reasoning 1.7B beats a non-reasoning GPT-4o by 10 points, and the rest is vibes.

ARC is the more interesting release. Separating predicted work from price is the right decomposition, the policies are simple enough to reason about, and the files are unusually well documented. But it is a cloud router today. Its "local" model is a datacenter-class model on several boxes, the eleven candidates it names include nothing that runs on the laptop in the clip, its headline quality figure does not reproduce from its own tables (my recompute lands a point higher, so this is sloppiness rather than spin), it loses roughly a fifth of the quality on the one non-coding agent benchmark, and its fit data has as many GPQA and Terminal-Bench groups as those benchmarks have tasks. I would not wire it into a local agent yet. I would read the next checkpoint the day it lands, because a router that knows when LFM3 is enough for the next step is the thing that makes a 1.7B agent useful beyond demos.

What I ran and what I read

I read the ARC-1.0 release files (config.json, presets.json, provenance.json, arc_release.json, NOTICE, CHANGELOG.md, the question-head manifest and a golden), range-read the safetensors headers of model.safetensors, upper.safetensors and pointer.safetensors to count parameters by tensor group, and read the technical report PDF. The ratios, averages, pooled-cost example and compounding table are my arithmetic on the card's and report's numbers. For the clip I stepped through @0xSero's video at full resolution. LFM3's numbers are from its model card and GPT-4o's from OpenAI's simple-evals README. I did not run ARC or LFM3.