Here is the uncomfortable truth about your agent's bill: most turns are easy. "Format this as JSON," "summarize the last tool result," "which file do I read next" — none of that needs a frontier model. But if your agent loop hardcodes one big model, you pay frontier prices for every trivial step. That is the #1 avoidable cost in an agent, and a router fixes it: send easy turns to a cheap, fast model and escalate only the hard ones.

You do not need a research lab to do this. You need about 35 lines of Python and a way to tell "good enough" from "escalate." This piece shows you all three routing strategies, gives you a working cascade router you can paste in today, and — the part everyone skips — shows you when routing costs more than it saves.

The three strategies#

There are exactly three ways to decide which model gets a turn.

Cascade / retry-on-failure. Call the cheap model first. Run a verifier on its answer. If the answer passes, you are done cheaply. If it fails, escalate to the frontier model. No training, no upfront classification — the answer itself is the signal.

Router-model / classifier. A small model or a trained router predicts, before any answer exists, whether this turn is hard. This is what RouteLLM (open source, from lm-sys) and Not Diamond do — they route simpler queries to the weaker model and hard ones to the strong model, tuned to a cost-quality threshold you set. One extra hop, no double inference, but it can mispredict.

Heuristic / semantic rules. Route by static signals: task type, token length, whether tools are needed. Near-zero overhead, but brittle — the day your input distribution shifts, your rules silently misroute.

Cascade is the right default for a solo builder: it needs no training data and it is honest, because the cheap model's own output decides whether to escalate.

The code: a cascade router in ~35 lines#

This uses the standard OpenAI client pointed at any OpenAI-compatible endpoint (a gateway like OpenRouter, a local server, or a provider directly), so it is provider-neutral. Swap the two model IDs for your cheap/frontier pair.

from openai import OpenAI

client = OpenAI(base_url="https://openrouter.ai/api/v1", api_key="YOUR_KEY")

CHEAP = "openai/gpt-4o-mini"          # your fast/cheap model
FRONTIER = "anthropic/claude-opus-4"  # your strong/expensive model

def ask(model, messages):
    r = client.chat.completions.create(model=model, messages=messages)
    return r.choices[0].message.content

def verify(question, answer):
    """Cheap self-check: reuse the cheap model as the verifier."""
    prompt = [{
        "role": "user",
        "content": (
            f"Question:\n{question}\n\nProposed answer:\n{answer}\n\n"
            "Is the answer correct, complete, and directly responsive? "
            "Reply with exactly one word: PASS or FAIL."
        ),
    }]
    verdict = ask(CHEAP, prompt)
    return verdict.strip().upper().startswith("PASS")

def route(messages):
    answer = ask(CHEAP, messages)
    question = messages[-1]["content"]
    if verify(question, answer):
        return answer, CHEAP
    return ask(FRONTIER, messages), FRONTIER  # escalate

if __name__ == "__main__":
    reply, model_used = route([{"role": "user", "content": "What is 17 * 23?"}])
    print(f"[{model_used}] {reply}")

Two things make this real. First, prefer a deterministic verifier whenever your task has one — does the JSON parse, did the code compile, did the tests pass. A deterministic check is free and can't be fooled. Fall back to the LLM-as-verifier above only when correctness is genuinely subjective. Second, this is exactly the shape LiteLLM formalizes with its Router: you can express fallbacks so that a failure (or a content/context error) automatically retries on the next model. Use LiteLLM's version once you outgrow the toy — it handles retries, timeouts, and load balancing you would otherwise reinvent.

The trap: when routing loses money#

Routing is not free, and the failure mode is specific. Every escalation is double inference — you paid the cheap model to produce a throwaway answer, paid the verifier, and then paid the frontier model anyway. If your escalation rate is high, you have added a surcharge to your most expensive turns.

The break-even is arithmetic. Let p be the fraction of turns that escalate. Your routed cost per turn is roughly:

cheap_cost + verifier_cost + p * frontier_cost

versus a flat frontier_cost for every turn. Routing wins only when that expression is smaller — which means when cheap_cost + verifier_cost is small relative to frontier_cost, and p is well below 1. If more than roughly half your turns escalate, a naive cascade can cost more than just always calling the frontier model. Instrument it: log model_used on every turn, compute your real p over a day of live traffic, and only then decide. If p is high, that is your signal to switch to a classifier (which avoids the double-inference tax by deciding upfront) — the trained routers exist precisely for this regime, and Martian or Not Diamond sell it as a managed hop.

The classifier path has its own tax: latency and a per-call fee. RouteLLM avoids the fee (it is open source and self-hosted) but you own the tuning. Not Diamond and Martian charge for the routing hop. None of that matters until you know your p.

The recommendation#

Start with the cascade router above and a deterministic verifier. Log your escalation rate for a week. If p stays low, you are done and you cut cost with zero quality loss. If p is high, do not add more retry logic — that just multiplies the double-inference bill. Switch to a trained classifier so you stop paying for cheap answers you throw away. Do not buy a routing product on day one; measure first, because the only number that decides which strategy wins is your own.

For picking the actual cheap model to route to, see our head-to-head in Terra vs Muse Spark 1.1 vs Grok 4.5. And if you are still deciding what to pay for at all, which AI coding subscription makes sense for a solo founder in 2026 frames the build-vs-buy call.