Ranking the context levers

Every team building agent infrastructure has a list of context optimisations. Ours had four: de-duplicate repeat file reads, supersede stale tool results, age out old content, truncate oversized Bash output. They were ranked, tiered, and scheduled.

Then we measured each one against real sessions, and the list came apart. The lever we had scheduled first is worth 33 cents. The lever we had rated too risky to ship is the one we now ship first. And 71% of everything left rests on a single parameter nobody has tested.

This post is the measurement, lever by lever, with the arithmetic attached.

The corpus: one developer, one machine, 36 days. 149 sessions — 59 developer sessions plus the 90 subagent runs they spawned — 14,948 tool results, 14,385 API requests, 4.38 billion prompt tokens, $3,195.19 at Anthropic list price. Every figure is reproducible by re-running the script that produced it.

The dollar figures are list-price valuations, not money anyone paid. The developer was on a subscription for the whole window. The right reading is "what a BYOK or API-billed team with this workload shape would be charged". It also means the corpus was generated under zero marginal cost — nothing in it was shaped by price, so treat context growth as an upper bound.

One developer is one developer. The cost structure is a property of how agentic coding works and should generalise. The magnitudes are this fleet's. This is the single biggest caveat on the whole analysis, and the last section says what we are doing about it.

---

1. The unit, and the ceiling

The unit throughout is token-turns — tokens × turns resident. Content in the context is re-read on every subsequent request, so this is the quantity cache-read billing is proportional to. A 10,000-token file read at turn 100 of a 1,100-turn session costs 10 million token-turns, not 10,000 tokens.

Where that occupancy sits:

Bash is the largest resident tool, ahead of Read — which is already a hint that the ranking we published was wrong, because our ordering put Read work first.

Before any lever, the ceiling:

Every lever below reaches only that 24.2%. System prompt, tool definitions and message history are the other 75.8%, they are invisible in a transcript, and they remain the largest unexplored headroom we have. Nothing in this post changes that.

---

2. The ranking

Standalone — each lever alone against the full corpus, so the shares overlap and must not be summed:

The spread between first and last is 923×. That is the finding: these were four items on a list, discussed as though they were comparable, and they are not remotely comparable.

Note also that rows 1, 1b and 1c are the same lever at three settings, and the gap between them is larger than every other lever combined. Which brings up the thing a ranked list hides.

---

3. Neither big lever is a switch. Both are dials.

Both of the levers that matter have exactly one parameter, and the parameter moves the value by 5–17×. "Ship age-out" is not a decision. "Ship age-out at 400 turns" is a decision, and it is worth 40% of what "ship age-out at 100 turns" is worth.

The gap between the 400-turn and 100-turn windows is $182 — larger than every other lever in the analysis put together. Nothing in this corpus tells you what a dropped result costs in task success, because no session was ever run with content removed.

---

4. Age-out is not one of four levers. It is the lever.

Session e170b28f, turn 147 of 1,317.

A whole-file Read of simulator_studio.py returns 10,003 tokens. The string simulator_studio does not appear anywhere in the session after turn 198 — not in a later tool call, not in a tool result, and a manual scan including assistant text and thinking blocks puts the last occurrence at turn 199. The session then ran for another 1,118 turns.

96% of that read's lifetime cost accrued after the agent had provably stopped referring to the file. One read, in one session, is worth 17× the entire read de-dup lever across 36 days.

That is the conservative form of the evidence. Restricting to reads ≥2,000 tokens that go ≥200 turns past their last mention finds 11 such reads worth $12.62 — and that filter is deliberately strict, because it requires the filename to vanish from the transcript entirely. A file mentioned once in passing at turn 900 is excluded even though its turn-9 read was dead the whole time.

Risk: HIGH and unquantified. Age-out carries 71% of the BALANCED tier's value and 88% of AGGRESSIVE. With de-dup and supersede corrected downward, the entire token-removal thesis now rests on a single parameter whose quality cost nobody has measured.

---

5. Bash truncation: the objection, tested

Bash truncation was rated HIGH risk on an objection that sounds obviously correct:

"Bash output is the signal — stack traces, test failures, grep hits. A 2 KB cut removes most of a Python traceback."

True about tracebacks. Irrelevant to where the money is. Split the recovered value by whether the output's head carries a diagnostic (Traceback, FAILED, assert, panic:, an error line):

Nine tenths of the value is in sed/cat dumps, grep roll-ups, git diffs and black/ruff summaries — output the agent skimmed once and never consulted again. The risk is concentrated in the other tenth, and it is separable two ways: keep head and tail so a traceback's exception line and its final frame both survive, or exempt anything matching the diagnostic predicate outright and take 90% of the value at close to zero risk.

One call makes it concrete. Same session, turn 4 of 1,317:

sed -n '125,230p' docs/requirements.md      →  16,756 chars ≈ 4,189 tokens

Issued as the fourth request of the session, to read a section of a requirements doc. It stayed in context for all 1,313 remaining turns: 5,500,157 token-turns, $2.75. A 512-token cap recovers $2.41 of that single call — 7× the entire verified read de-dup lever across the whole corpus.

This is the lever to ship first. It is 5.8× supersede, 166× verified read de-dup, bounded in the worst case, and the only one of the four whose risk profile can be checked before shipping rather than after.

---

6. Supersede: small, real, and mostly screenshots

Session dcb050f8, turns 140→150 of 411. Five byte-for-byte identical calls:

mcp__claude-in-chrome__computer  {action: "screenshot", tabId: 2147251747}
   turn 140 · turn 142 · turn 146 · turn 148 · turn 150

A UI-verification loop: screenshot, adjust, screenshot again. Each result costs ≈1,677 tokens. The moment turn 142 fires, the turn-140 screenshot is a picture of a page that no longer exists — but it keeps being re-read for the remaining 269 turns.

Corpus-wide: 228 hits, $9.50, 0.30% of billed — and it is mostly this one behaviour. chrome/computer is $6.38 of the $9.50, against Read $2.29, the other browser tools $0.36 combined, and Bash $0.17.

Risk: LOW. Identity is on the full tool input, so re-reading a different slice of the same file supersedes nothing, and the semantic claim — running the identical command again makes the earlier output stale — holds for exactly the case it fires on. It is small, and worth building mainly because screenshot-heavy browser loops are the fastest-growing shape in the corpus.

One correction inside this lever is worth stating on its own. Screenshots were originally priced as text. A base64 blob charged at len(base64) / 4 bills a 1451×840 JPEG at 27,878 tokens; Anthropic bills it at roughly w·h/750 ≈ 1,625. A 17× overstatement — and because screenshots dominate this lever, it is most of why supersede looked like a headline item. It also moved the denominator for everything else: tool results are 24.2% of prompt volume, not the 40.1% we had published.

---

7. Read de-duplication: the lever that fell by 540×

This was supposed to be the safe, obvious, ship-first lever. The case for it was a single statistic: 59% of file reads are re-reads of a file already read in that session. That count is correct — this corpus puts it at 57.1%.

The inference drawn from it does not follow.

A repeat read of a file is almost never a repeat read of the same bytes. 97.6% of the "re-reads" justifying this lever are the agent walking a different part of a file it already opened — new content, carrying new information, impossible to collapse to a pointer.

What broke the earlier measurement was mundane. Our dataset fingerprinted a tool call by hashing file_path alone. Session 4b5ae4dd reads proxy.py at offset 1 limit 470, then offset 470 limit 450, then offset 920 limit 200 — three disjoint slices, one walk down a 1,119-line file. Scored as three reads of the same target, that single walk contributed $2.89 of "de-duplication value".

The most striking number in the original analysis was "73 reads of one file in one session". It reproduces exactly, and dissolves on inspection: those were 73 different offsets into one large file. No identical range is re-read more than 4 times in any session in the corpus.

Key on the whole tool input, require the bytes to already be resident, and the lever collapses:

The largest single verified hit in 36 days is five cents: authenticate.ts read whole at turn 715 and again at turn 895 of a 1,128-turn session, both hashing to 423c56c4d00b5e2b, with the writes landing at turns 896–898. That is the agent re-reading a file immediately before editing it — the most common repeat-read shape in the corpus, and the reason the staleness gate exists.

Recommendation: drop this lever, do not defer it. At $0.33 over 36 days it does not pay for the per-path staleness tracking it requires, and the tracking is the part that has to be correct.

---

8. Composed, the tiers do not look like the list

Applied together, first-match-wins, so no lever double-counts another's tokens:

Two things fall out.

The SAFE token-removal tier is empty. We had published de-dup + supersede at 6.84% of tokens. Verified, it is 0.69%, worth $9.77. There is no ship-now story on token volume that does not involve truncation.

One experiment gates 71–88% of the remaining value. Replay a session with aged context and measure task success. Until that runs, the honest recommendation is balanced-minus: ship Bash truncation at 512 tokens with head+tail retention and a diagnostic exemption ($55, LOW risk), ship supersede alongside it because it is nearly free ($9), and hold age-out.

---

9. What this does to the product

We launched the ACE sidecar last week: a local reverse proxy that sits between your coding agent and the model provider, relays your traffic unchanged, prices every request, reads your existing transcripts off disk, and renders the result on a dashboard at 127.0.0.1. No account, no upload, one environment variable. Its first release deliberately ships zero levers — it reports "levers": [] on its health endpoint and relays every request byte-for-byte.

This analysis is the reason that ordering exists, and it now feeds the sidecar in three concrete ways.

The dashboard ranks the levers on your corpus, not ours

The sidecar's left rail used to list four lever names in a fixed, arbitrary order — which told a reader nothing about which one to want. It now scores each lever alone against your own sessions over the selected date range and orders them by what they are worth:

Each row carries the dollars, the share of your bill, a bar proportional to the largest lever, and the risk class. Those particular figures are this machine's, over a corpus that has since grown to 162 sessions and $3,315.90 — which is exactly why the panel computes rather than hardcodes. Three details are deliberate:

• Standalone, not composed. Under the tier scorecards a lever is applied first-match-wins, so its contribution depends on which levers ran before it. That is the right model for "what does this tier save" and the wrong one for "which lever should exist". Scored alone the shares overlap, so the panel says so, and links to the composed version rather than inviting you to add four numbers that do not add.
• The operating points are the defensible ones, not the maximal ones — age-out at 400 turns rather than 100, a 512-token cap rather than 256. The tooltip states the upside of turning each dial further, and the risk that comes with it.
• Every row still renders disabled. Ranking a lever is not shipping it. Phase 0 measures only, and a control implying otherwise would be the dashboard misrepresenting the product.

The corrections had to be made in the instrument, not just the write-up

A dashboard that reproduced the old de-dup number while this post argued the new one would be worse than no dashboard. Three fixes went into the sidecar's own scanner alongside this analysis: tool calls are now keyed on the whole tool input rather than the target path, so a paginated walk stops looking like duplication; de-duplication requires a matching content digest, so redundancy is proved rather than inferred; and images are priced at w·h/750 rather than base64 length. On this machine's corpus the corrected panel puts age-out at $124.69 and truncation at $57.94, against $122.57 and $54.94 in the analysis above — the same conclusions, independently computed, on a corpus 13 sessions larger.

It reorders the roadmap

The published ordering was exactly inverted. Bash truncation is first by 166×; Read de-duplication is last by two orders of magnitude.

---

10. What we are least sure of

• One developer, one machine, 36 days. The cost structure should generalise; the magnitudes are one fleet's. This is why the sidecar computes on your corpus instead of shipping ours, and why the thing we most want from beta users is a second corpus.
• Tool results only — 24.2% of prompt volume. Everything here is a lower bound on what context reduction could reach, and says nothing about the other 75.8%.
• Residency is modelled, not observed. A result is assumed resident from its turn to the end of the session. Compaction truncates that in real sessions and fires late, and where it fires these figures overstate residency.
• The verified de-dup floor is a floor. Byte-identity and verbatim containment are provable; semantic redundancy is not measurable from a transcript and is certainly larger than $0.33. What is not larger is the subset a sidecar could suppress without risking stale content.
• chars / 4 for text. Fine in aggregate, wrong for any individual result.

Every number in this post moved downward at least once, and every revision was a correction to our own analysis rather than new information about the world. The pattern was identical each time: we measured a proxy for the thing instead of the thing. A file path instead of a tool call. A base64 length instead of an image. A re-read instead of a re-read of the same bytes.

That is the argument for building the instrument before the optimiser. We would have shipped per-path staleness tracking — the hard, correctness-critical kind — for 33 cents.

---

The sidecar runs on your machine, uploads nothing, and works with Claude Code today. The beta is opening in batches — details and signup here. The broader cost breakdown this analysis sits inside is in What 13,000 coding-agent requests actually cost.