To cut your LLM bill, stop sending every request to one flagship. Sort your traffic into a few tiers by how hard the task actually is, and route each tier to the cheapest model that clears its bar. Most of what an app does — classify, extract, tag, reformat — does not need a $10/$50 model. The savings hide in the routing, not the model.
In one screen:
- Tier 0 — clerical, high-volume. Classification, extraction, tagging, boilerplate. Route to GPT-6 Luna ($0.10/$0.50) or DeepSeek V4.1 Flash ($0.15/$0.60 off-peak, cache hits $0.003).
- Tier 1 — reasoning & coding. Multi-step logic, code, synthesis. Route to GPT-6 Sol ($2/$10) or Claude Opus 5.5 ($4/$20).
- Tier 2 — the hard ~10%. Ambiguous, high-stakes, long-context judgment. Route to GPT-6 Astra ($10/$50) or an Opus flagship.
- Measure end-to-end cost per finished job, not per token — and put the router behind a gateway so swapping a tier is a config change.
All prices are per 1M tokens (input/output), as of the September 2026 launches: OpenAI's GPT-6 Astra, Sol, and Luna, Anthropic's Opus 5.5, and DeepSeek V4.1 Flash. For the full landscape, see our September pricing breakdown.
The worked cost math#
Say you process 50M input tokens and 10M output tokens a month (illustrative — plug in your own). Send all of it to the flagship GPT-6 Astra:
- Input: 50 × $10 = $500
- Output: 10 × $50 = $500
- Total: $1,000/month
Now tier it. Realistically about 70% of traffic is clerical, 20% is reasoning/coding, and 10% is genuinely hard. Split the tokens the same way and route each tier down:
| Tier | Model | Input | Output | Cost |
|---|---|---|---|---|
| 0 (70%) | Luna | 35M × $0.10 = $3.50 | 7M × $0.50 = $3.50 | $7 |
| 1 (20%) | Sol | 10M × $2 = $20 | 2M × $10 = $20 | $40 |
| 2 (10%) | Astra | 5M × $10 = $50 | 1M × $50 = $50 | $100 |
Tiered total: ~$147/month. That is a delta of about $853, roughly 85% off the same workload — and you still send the hard 10% to the best model. Nothing about output quality changed on the tasks that mattered; you just stopped overpaying for the easy 70%.
The cheapest model is a trap. The cheapest capable model, measured per finished job, is the win.
The router, in ~30 lines#
The whole trick is a dict from task class to model id, plus a thin wrapper that calls whatever model the router returns. Point the client at an OpenAI-compatible gateway so every provider looks the same:
# route.py — map each task class to the cheapest model that clears its bar.
# Change a value here and every caller picks it up; no code migration.
MODEL_BY_TIER = {
# Tier 0: clerical, high-volume, no real reasoning.
"classify": "gpt-6-luna",
"extract": "gpt-6-luna",
"format": "deepseek-v4.1-flash",
# Tier 1: multi-step reasoning and coding.
"reason": "gpt-6-sol",
"code": "claude-opus-5.5",
# Tier 2: the hard ~10% — ambiguous, high-stakes judgment.
"hard": "gpt-6-astra",
}
DEFAULT_MODEL = "gpt-6-sol" # unknown task? default to mid, never flagship.
def route(task_type: str) -> str:
"""Return the model id for a task class."""
return MODEL_BY_TIER.get(task_type, DEFAULT_MODEL)
def complete(task_type: str, prompt: str) -> str:
"""Pick the model, then call it through your gateway."""
model = route(task_type)
resp = client.chat.completions.create( # OpenAI-compatible gateway
model=model,
messages=[{"role": "user", "content": prompt}],
)
return resp.choices[0].message.content
# complete("extract", "Pull the invoice total from: ...") -> runs on Luna
# complete("hard", "Should we counter this clause?") -> runs on AstraThe only real work is the classifier that assigns task_type. Often it is a lookup — you already know a given endpoint is extraction — so no model call is needed. When the class is genuinely uncertain, run a cheap Tier 0 classifier first; a Luna call to sort the request costs a fraction of a cent and keeps the expensive tiers empty.
How to measure it#
Per-token price is not your cost. Cost per finished job is. A budget model that fails a task, retries twice, or pads its output with three paragraphs of hedging can quietly cost more than one flagship call that lands the first time.
So log, per job: the model used, input tokens, output tokens, retry count, and whether the result passed your acceptance check. Then compute:
cost_per_job = (in_tokens * in_price + out_tokens * out_price) / jobs_that_passedNote the denominator: failed jobs still burned tokens, so divide by the ones that finished correctly. Now you can compare tiers honestly. If Tier 0's cost-per-job on some task class beats Tier 1's once retries are counted, keep it there. If Luna keeps failing a class and bouncing it up to Sol anyway, promote that class — you were paying for both models. This is exactly where a cheap-per-token model can be the expensive choice, and the only way to see it is the end-to-end number.
Re-run this monthly. Prices move: GPT-6 launched only on September 22, 2026, and Opus 5.5 cut cache reads about 60% versus the prior generation — the kind of shift covered in our price-war dispatch. Your task mix drifts too. A routing table that was optimal in June is not optimal now.
Keep it swappable#
The reason to route behind a gateway is not elegance — it is optionality. When Luna gets undercut, or an open-weight coder you self-host clears Tier 1 for free, you change one string in MODEL_BY_TIER and every caller follows. No redeploy of business logic, no scattered SDK swaps, no migration.
That is the whole discipline: classify by difficulty, route to the cheapest capable model, measure per finished job, and keep the mapping in config. Do that and your bill tracks the market instead of your inertia.
