shadcn's improve: an expensive model writes the plan, a cheap one types it, and the same expensive model grades it

A post by @awe0_io, in Chinese, opens with "this project is made for people with token anxiety" and describes shadcn/improve like this: the strongest model reviews the code, finds the problems and makes a plan; a cheap model edits the code and runs the tests according to the plan; and the result is checked automatically against what the plan asked for. Install with one command, npx skills add shadcn/improve, and "not all code needs the most expensive model to write it personally any more."

That is a fair one-paragraph summary of the README's pitch. It is also a description of something most people running coding agents already do by hand: think with the big model, type with the small one. What makes improve worth reading is that it writes the hand-off down. I cloned the repository at commit cac56e1 (version 1.0.0, MIT) and read all of it. It is not new: the first commit is from 10 June, most of the 23 are from that week, and the last, a one-word chore, is from 12 September. It is small enough to read in one sitting: a 6,926-byte README, a SKILL.md, three reference files of 1,931, 1,170 and 1,359 words, one example plan and a plugin manifest. There is no code. The whole product is instructions to a model.

So the questions are about the instructions. What does the strong model actually produce? What does the cheap model see? What is "the automatic check", mechanically? And when does splitting the work cost less than not splitting it?

Four files of instructions

The repository's tree, minus .git and a funding file:

skills/improve/SKILL.md
skills/improve/references/audit-playbook.md
skills/improve/references/plan-template.md
skills/improve/references/closing-the-loop.md
examples/001-extract-shadow-config-resolution.md
.claude-plugin/plugin.json
.claude-plugin/marketplace.json

SKILL.md is the entry point. Its first line of instructions sets the role: "You are a senior advisor, not an implementer." Its second paragraph states the economics in one sentence: "an expensive, high-ceiling model does the part where intelligence compounds (understanding, judging, specifying). Cheaper models do the execution. The plan is the product."

Six hard rules follow. The one that shapes everything else is the first: never modify source code. The only files the advisor may write live under plans/ (or advisor-plans/ if plans/ is already taken). It may not run installs, builds, formatters or commits; read-only analysis such as tsc --noEmit or a side-effect-free test run is allowed. Rule 6 is a prompt-injection rule I was glad to see: anything read from the audited repository is data, and a file that tries to give the advisor instructions becomes a security finding instead.

The commands map onto variants of one workflow:

/improve                        full audit → prioritized findings → plans
/improve quick                  cheap pass: hotspots, top findings only
/improve plan <description>     skip the audit, spec one thing
/improve execute <plan>         dispatch a cheaper executor, review its work
/improve reconcile              refresh the backlog: verify, unblock, retire

The post makes it sound like a pipeline that always runs end to end. It is not. The default /improve stops after writing plans and asks you which findings to plan in the first place. Execution is a separate, explicit command, and merging is never the skill's to do.

What the strong model writes

A bare /improve runs in four phases.

Recon maps the repository: the stack, the conventions, and the exact build, test, lint and typecheck commands, which go into every plan as verification gates. It also reads ADRs, PRDs, CONTEXT.md, DESIGN.md and PRODUCT.md when they exist, so that a tradeoff someone already decided is not re-reported as a bug.

Audit fans out read-only subagents across nine categories (correctness, security, performance, tests, tech debt, dependencies, DX, docs and "direction"), at most four at once on the default standard level, eight on deep, and zero or one on quick. Because subagents do not inherit the skill, the advisor must paste into each one the path to audit-playbook.md, the scoping facts from recon, and a verbatim copy of the two rules about secrets and injected instructions. Every finding comes back in a fixed shape: evidence as file:line, impact, effort (S, M or L), risk of the fix, confidence.

Vet is the step I would have skipped and should not. "Subagents over-report", so the advisor reopens every cited location itself before showing you anything, drops by-design behaviour (the README's example: a proxy environment variable flagged as SSRF), corrects wrong attributions, and records rejections so the next run does not raise them again. Findings are then ordered by leverage, impact over effort, discounted by confidence and fix risk.

Plan writes one file per finding you pick. This is the expensive model's real output, and plan-template.md is the most careful file in the repository. It writes "for the weakest plausible executor", which it describes as competent at following explicit instructions and weak at filling gaps or knowing when to stop. Three properties carry that:

• Self-contained. Exact paths, excerpts of the current code with file:line markers, the repo's conventions with an exemplar file, and the exact commands with expected output. "If a plan references 'the pattern discussed above,' it is broken."
• Verification gates. Every step ends with a command and its expected result, so "the executor never has to judge whether it succeeded."
• Hard boundaries. An in-scope file list, an out-of-scope list with reasons, and STOP conditions: if reality does not match, stop and report.

Each plan also stamps the commit it was written against and opens with a drift check the executor runs first:

git diff --stat <planned-at SHA>..HEAD -- <in-scope paths>

The example in examples/ is a real plan from a run against shadcn/ui at commit 1994caba0: extract the duplicated "shadow config" resolution from search.ts and view.ts, whose two copies had already drifted (one seeds defaults with createConfig({style: "new-york", ...}), the other with a bare configWithDefaults({})). It is 1,149 words. That number matters below: a plan is not a paragraph, it is a document the strong model spends output tokens on.

What "the automatic check" is

The post's last claim, that the cheap model's result is checked automatically, is where the text is more interesting than the summary. The check lives in closing-the-loop.md, under execute <plan>, and it is not a separate verifier. It is the advisor again.

Dispatch first. The advisor checks that the repository is a git repo, that the plan's dependencies are marked done, and runs the drift check itself. Then it spawns "one general-purpose subagent with isolation: \"worktree\"" and a model of "default sonnet; use what the user named if they named one (execute 003 haiku)". The prompt must inline the full plan, because a fresh worktree contains only committed files: "if plans/ is uncommitted, the executor can't read it." The executor gets a fixed preamble:

You are the executor for the implementation plan below. Follow it step by
step. Run every verification command and confirm the expected result before
moving on. Touch only the files listed as in scope. If any STOP condition
occurs, stop immediately and report. Do not improvise around obstacles.

and must answer in a fixed report format: STATUS: COMPLETE | STOPPED, per-step results, files changed, notes.

Then the review, which the file calls "the advisor's real job here". Four steps, in order:

1. Re-run every done criterion in the worktree. The executor's report is not trusted.
2. Scope compliance: git -C <worktree> diff --stat against the in-scope list. "Any file outside scope fails review, full stop."
3. Read the full diff, judged against the plan's "Why this matters" and the repo's conventions.
4. Audit the new tests. "Executors game criteria": a test that asserts nothing meaningful still passes pnpm test.

The verdict is one of three. APPROVE marks the plan done and hands you the worktree and branch; merging stays your decision, and the advisor never pushes or commits to your branch. REVISE sends specific feedback to the same executor through SendMessage, with a "Max 2 revision rounds, then BLOCK." BLOCK records the reason and rewrites the plan with what was learned. One more rule is more humane than most agent specs manage: a documented deviation is "judged on merit, not reflex-blocked"; an undocumented one is a failure.

So the "automatic check" is two command-shaped gates (re-run the criteria, diff the file list) and two judgment-shaped ones (read the diff, read the tests), all performed by the strong model. That is a sensible design. It is also where the savings go, because review is the strong model reading more tokens.

Which agents it actually runs in

The README says it "works in any agent that supports Agent Skills format", and for planning that is true: the plans are markdown, and any agent or person can pick one up. execute is a different matter. Its dispatch names Claude Code's tools: an Explore agent for the audit, a general-purpose subagent, the isolation: "worktree" option, model aliases sonnet and haiku, and SendMessage for revision rounds. The repository also ships a .claude-plugin/ manifest and marketplace file. The skill handles the gap honestly: if the host cannot spawn a subagent in an isolated worktree, "say so and hand the plan over for manual execution instead."

The skill never names the advisor's model. It inherits whatever your session runs. The only model it names is the executor's default, and that default is not the cheapest tier: it is sonnet. Keep that in mind for the arithmetic.

The arithmetic of a split

Start from the bill for one agentic task, which the harness-effect paper writes as a sum over turns of input and output tokens times their prices. Fold the turns together and give each role its own price. With $p$ for the strong model's price per token and $q$ for the cheap one's (each split into input and output), the strong model doing everything costs

where $U$ is the reading it takes to understand the code and $E$ is the execution loop: edits, test runs, re-reads, fixes. The split costs

where $W$ is writing the plan, $V$ is one review (reading the plan, the diff and the test logs, then writing a verdict), $r$ is the number of revision rounds (0 to 2 here) and $\rho$ is the share of the execution redone per round. The understanding term $U$ cancels; both arms pay it at the strong price. The split saves money exactly when

The left side is what you save by typing with the cheap model. The right side is what you spend on documents: the plan, the reviews, and the redone work.

Now the prices. On this site's model table (data/models.ts, snapshot of 12 November, from Artificial Analysis's records), per million input and output tokens: Opus 5.5 is $4 and $20, Sonnet 5.5 is $2 and $10, and Haiku 5.5 is $0.10 and $0.50, the tier for prompts up to 100,000 tokens (the Haiku 5.5 piece has the long-prompt tier and the tokenizer change). So if your session runs Opus 5.5 and the executor is the skill's default Sonnet, $q = p/2$. The executor is half price, not a tenth. With Haiku, $q = p/40$.

Put some sizes on it. The repository publishes no token counts for any run, so these are my assumptions, chosen to look like the example plan: a refactor where the strong model reads 60,000 tokens to write a 4,000-token plan, the executor's loop consumes 300,000 input tokens and writes 20,000, and the review reads 40,000 and writes 3,000.

(One revision round redoes 40% of the execution and repeats the review. All figures are my arithmetic on the list prices; no caching in either arm.)

Three readings fall out of it.

With the default Sonnet executor, a clean first pass saves 27%, and a single revision round turns that into a 2% loss. The skill allows two rounds. With both, the split costs about 32% more than letting Opus do the job alone. Halving the price of execution does not buy much room for documents.

With Haiku the saving is large (68% clean, 56% after one round) because execution nearly vanishes from the bill. What is left is almost all strong-model tokens: the plan and the reviews. At that point the review is the cost, and every revision round adds a full one.

On a small fix (30,000 tokens read to plan, an 80,000-token execution loop) the Sonnet split saves under 4% clean and loses 37% after one round. On a migration with a 1.2-million-token execution loop it saves 40%. The split is a bet that execution dwarfs specification. For most one-file bugs it does not.

The calculator leaves out caching and tokenizers, and both cut against the split. Caching first. A real agent loop is mostly cache reads, which Anthropic prices far below fresh input (the Haiku 5.5 calculator has the rates). That shrinks $E$'s input bill in both arms, but the plan and the review are fresh documents read once, so caching narrows the executor's advantage more than the advisor's. The prompt-caching piece has the per-harness detail. Then tokenizers. The formula assumes both models count the same text as the same number of tokens. Within one vendor's family that is close; pairing a strong model from one lab with a cheap model from another, you would have to count the plan in both.

And one thing that cuts for it: the plan is reusable. A plan that BLOCKs and gets rewritten is still a better starting point than a fresh session, and reconcile exists to keep a backlog of them alive across sessions. If you would have written the spec anyway, $W$ is not overhead.

Where the loop is blind

The review is good at what can be checked by command. An executor that edits a file outside the plan fails the git diff --stat comparison. An executor that reports a green test that is red fails the re-run. A plan whose in-scope files changed since it was written fails the drift check before anything starts. Those are real protections, and they are cheap.

Two failures get past every gate, and neither is a bug in the skill so much as a property of the design.

The drift check only looks where the plan points. It diffs the in-scope paths against the planned-at commit. If the plan's correctness depends on a caller, a schema or a config file it does not list, a change there is invisible. The plan template does tell the author to write STOP conditions for its key assumptions, which helps exactly as much as the author anticipated the assumption.

The planner and the reviewer are one model. The done criteria are commands the advisor wrote. The diff is read against a "Why this matters" the advisor wrote. If the advisor misread the code during recon, the plan encodes the misreading, the executor implements it faithfully, the criteria pass because they test the misreading, and the review approves a diff that matches the intent it was given. This is the same shape of problem as Code2Skill's "verified", where one model judged another model's rebuild: the check is only as independent as the checker. The skill's own review-plan variant recognises it and asks for "a fresh-context subagent" to read a plan cold, because "self-critique misses gaps you mentally fill from context the executor won't have." The same remedy is not applied to the review of the diff.

Then the softer one. Audit step 4, reading the new tests, is a judgment, and it is the only gate that catches a test written to pass. The file warns that executors game criteria. It relies on a reviewing model reading carefully at the end of a long session, which is the condition under which models read least carefully.

A smaller practical one: the worktree sees only committed files. The skill handles that for the plan by inlining it, but a plan that tells the executor to follow "the pattern in src/lib/result.ts", where that file is new and uncommitted on your branch, sends it to a file that is not there.

Against other loops on this site

Agent harnesses describes the loop around a single model; improve is a loop around two, with a document in between. The harness-effect paper argues that orchestration sets the token bill more than the choice of model does, and improve is a clean example: the cheaper executor only pays off once the plan and the review, both rebuilt context, are small next to execution. The three-tier piece split agents into deterministic code, a decision model and a reasoning model and found the middle tier blind across its own options; improve's executor is told to be exactly that, blind across options by design, with STOP conditions in place of judgment. And agent-tools-week read three harnesses for where they check the agent's work, which is the question this piece kept coming back to. Genex's self-modifying harness only accepts a change to itself after a gate type-checks and boots a copy; improve gates a change through the same model that asked for it.

What I would use it for

I would use the planning half without hesitation. A plan with excerpts, exact commands and STOP conditions is a better artefact than the chat log it replaces, and the vet step (reopen every cited line before reporting it) is the habit that separates a useful audit from a list of subagent guesses. The rule that the advisor never edits code is what makes it safe to point at a repository you care about.

I would be choosier about execute. With its default Sonnet executor on Opus, the per-token gap is 2x, and the arithmetic says a single revision round can take the whole saving. If the point is cost, name the cheap model explicitly (/improve execute 001 haiku) and pick plans whose execution is long and mechanical: migrations, renames, test backfills. For a one-file fix, let the strong model type it.

And I would give the review a second reader. The cheapest improvement to the loop is the one the skill already uses for plans: a fresh-context subagent that reads the plan and the diff cold and is asked whether the change does what the code needs, not what the plan says.