The four-model week of July 2026 did something useful for anyone paying an inference bill: it made "just use the best model for everything" the expensive, lazy default. GPT-5.6 shipped in three tiers ($5 / $2.50 / $1 per million input tokens for Sol / Terra / Luna), Claude Sonnet 5 landed at $2 in / $10 out, and the practical upshot is that near-frontier quality now costs a dollar or two per million tokens. If your product still routes everything to one premium model you chose last quarter, you are leaving 60–80% of your inference spend on the table.

You don't need a rewrite. You need a router. Here's the afternoon version.

Step 1 — Measure before you touch a model string#

You can't route what you haven't bucketed. Add one log line at every model call site recording the task type and the token counts you're already getting back in the response.

# wrap your existing client call
resp = client.responses.create(model=MODEL, input=messages)
log.info("llm_call", extra={
    "task": task,                       # "classify" | "extract" | "chat" | "plan"
    "in_tokens": resp.usage.input_tokens,
    "out_tokens": resp.usage.output_tokens,
    "model": MODEL,
})

Let it run for a day of real traffic, then aggregate:

SELECT task,
       count(*)                         AS calls,
       sum(in_tokens)  / 1e6            AS m_in,
       sum(out_tokens) / 1e6            AS m_out
FROM   llm_calls
GROUP  BY task
ORDER  BY m_in DESC;

The output almost always shows the same shape: one or two "workhorse" tasks account for the bulk of your tokens, and they are the simplest tasks — classification, extraction, short structured answers. Those are exactly what a cheap tier handles at full quality. That's your savings, located.

Step 2 — Build a three-tier ladder with an explicit escalation rule#

The mistake is a two-tier "cheap vs. best" flip. You want three tiers and a rule for climbing them, not vibes.

TIERS = {
    "cheap": "gpt-5.6-luna",      # $1/1M in  — classify, extract, short answers
    "mid":   "gpt-5.6-terra",     # $2.50/1M  — reasoning, synthesis, agent steps
    "frontier": "gpt-5.6-sol",    # $5/1M     — the hard 10%
}

def route(task, ctx):
    # default everything to the cheap tier
    tier = "cheap"
    if task in ("plan", "synthesize") or ctx.get("tokens", 0) > 8000:
        tier = "mid"
    if ctx.get("prior_attempt_failed") or ctx.get("ambiguous"):
        tier = "frontier"          # escape hatch, not a default
    return TIERS[tier]

The important design choice is that frontier is an escape hatch reached by an escalation condition, never the starting point. A clean pattern: run the cheap tier first, and only escalate when a cheap validator (a schema check, a confidence threshold, a regex, or a one-line "did this answer the question? yes/no" call to the cheap model) says the cheap answer failed. Most of the time it won't, and you pay cheap-tier prices for cheap-tier work.

"Use the best model" is a pricing decision disguised as a quality decision. Almost no user can tell Terra from Sol on an extraction task — but your invoice can.

Step 3 — Turn on prompt caching (this is the big one)#

If your workload reuses context — RAG with a fixed system prompt, an agent replaying a long tool history, few-shot examples — caching is a bigger lever than the model swap. On the new GPT-5.6 tiers, cache reads are billed roughly 90% cheaper than fresh input; cache writes cost about 1.25× normal input. The rule: put everything static at the front of your prompt and everything dynamic at the back, so the stable prefix is what gets cached.

# structure the prompt so the cacheable prefix is first and identical every call
messages = [
    {"role": "system", "content": STABLE_SYSTEM_PROMPT},   # cached
    {"role": "system", "content": FEW_SHOT_EXAMPLES},      # cached
    {"role": "user",   "content": retrieved_context},      # cached if reused
    {"role": "user",   "content": user_question},          # the only fresh part
]

For a RAG endpoint firing the same 6k-token system + examples prefix on every request, moving that prefix onto cache reads can cut the input portion of the bill by most of 90%. Combined with a cheaper model tier underneath, this is where the 60–80% total number comes from.

Step 4 — Guardrail the swap with an eval, then ship#

Don't trust the price drop blindly — trust it after a diff. Take 50–100 logged real requests, run them through both the old model and the new cheap tier, and compare.

for case in golden_set:                 # 50–100 real logged inputs
    old = call(OLD_MODEL, case.input)
    new = call(TIERS["cheap"], case.input)
    record(case.id, agree=grade(old, new, case.rubric))
# ship the swap only if agreement on the workhorse task is ≥ your bar (e.g. 97%)

For classification and extraction, "grade" can be exact-match or schema-valid. For open-ended tasks, use the mid or frontier model as the judge — it's a one-time eval cost, not a per-request one. If the cheap tier clears your bar on the workhorse task, ship it; if it doesn't, you've learned exactly which task needs to stay on the mid tier instead of overpaying for all of them.

The 5-line summary#

  1. Log task + token counts at every call site for a day.
  2. Find your workhorse tasks — highest volume, usually the simplest.
  3. Route three tiers with frontier as an escape hatch, not a default.
  4. Cache the static prefix — up to ~90% off repeated input.
  5. Eval on 50–100 real cases, then ship the swap.

The models did the hard part this month by getting cheap and plentiful. Capturing it is a routing problem, and it's a smaller one than it looks — an afternoon of measurement and a switch statement stand between you and a materially better gross margin.