Context editing gives you a number that feels like a receipt for savings: after each clear, the API tells you cleared_input_tokens: 50000, and you think you just took 50k tokens off the bill. Sometimes you did. But a clear also rewrites the deal with your prompt cache, and that side of the ledger doesn't show up in cleared_input_tokens at all. The honest question isn't "how many tokens did I clear?" — it's "did my cost per completed task go down?" Here is how to measure that, using only fields the API already returns.

Why the cleared-tokens number lies by omission#

Context editing's clear_tool_uses_20250919 strategy evicts old tool results once your window crosses a trigger, keeping the most recent few and replacing the rest with a placeholder (Claude docs). Smaller prompt, fewer input tokens — real.

The catch is prompt caching. Caching works by hashing a stable prefix of your prompt and serving it back cheaply on the next turn. A clear changes that prefix, so it invalidates the cache at the point of the edit. Your next request can't read the old cached prefix; it has to write a new one (Anthropic).

That matters because the three input buckets are priced very differently (pricing):

So a clear that removes 50k tokens which were being served as cache reads at 0.1x, and forces the remaining 120k-token prefix to be re-written at 1.25x, can easily cost more than it saved. cleared_input_tokens counts the gross tokens removed. It never nets them against the re-cache you just triggered.

What it means: treat cleared_input_tokens as an activity log, not a savings report. The savings, if any, live in the usage block.

Step 1 — Price every run from the usage block, not the token count#

For every request in an agent run, the response usage object gives you four numbers: input_tokens, cache_creation_input_tokens, cache_read_input_tokens, and output_tokens. Sum each bucket across the whole run and price them separately:

run_cost =
    input_tokens              * base_in
  + cache_creation_input_tokens * base_in * 1.25   # 5-min TTL
  + cache_read_input_tokens     * base_in * 0.10
  + output_tokens             * base_out

Do not collapse the three input buckets into one "input tokens" figure — that's exactly the move that hides the cache cost of clearing. Keep them separate.

What it means: a run's true price is four line items, and context editing moves tokens between those line items (cheap reads → expensive writes) as often as it removes them.

Step 2 — A/B the same task with editing off vs on#

Run a fixed, representative task — the same seed, same tools, same target — twice: once with context_management omitted, once with it configured. Capture run_cost from Step 1 for each, and divide by whether the task actually completed correctly.

The metric is dollars per completed task, not dollars per token. This is the same trap as inference throughput: a cheaper-per-token setup that needs an extra turn or a retry can cost more per finished job, which is why one tokens-per-second number lies to you. Context editing has a second failure mode here — clearing too aggressively can evict a result the model needed, forcing it to re-call the tool, which adds turns. Cost per completed task catches that; cleared-token totals don't.

What it means: if editing-on doesn't lower cost per completed task versus editing-off on your real workload, you're paying cache-rewrite tax for a prettier token graph.

Step 3 — Watch the cache-hit ratio collapse after each clear#

Instrument two derived signals per request:

Plot the hit ratio across the run. You'll see it dip every time a clear fires — that dip is the re-cache cost made visible. If the ratio never recovers (because clears fire faster than the prefix re-caches), context editing is actively fighting your cache. That's the pattern to kill.

What it means: a healthy configuration shows the hit ratio recovering between clears; a broken one shows it sawtoothing downward. The applied_edits timestamps line up exactly with the dips.

Step 4 — Tune clear_at_least, then exclude_tools#

Two knobs fix most of the damage:

Also set keep to hold the most recent N tool uses the model still reasons over, and remember the sibling lever: anything the agent must not lose should be written to the memory tool before a clear can remove it, because a clear is not a save.

What it means: clear_at_least converts context editing from "clear whenever the trigger trips" into "clear only when it pays" — which is the whole game.

Step 5 — Preview clears for free with count_tokens#

Before you spend a live request finding out what a clear does, dry-run it. The count_tokens endpoint accepts the same context_management config and the context-management-2025-06-27 beta header, and returns both input_tokens (after the hypothetical clear) and context_management.original_input_tokens (before) (token counting). The difference is exactly what a clear would remove at that point in the conversation — measured without paying for a generation.

What it means: you can calibrate trigger and clear_at_least against real conversation states in a loop that costs a fraction of live inference, then ship the config that actually lowers cost per completed task.

The discipline is one sentence: context editing is proven by the usage block and the completed-task count, never by the cleared-tokens headline. Instrument the four usage buckets, net savings against cache writes, A/B on real tasks, and tune clear_at_least until clears only fire when they pay. Do that and you'll know — with receipts — whether context editing is cutting your bill or just redecorating it.