The cheapest way to cut an AI agent's model bill is not a cheaper model. It's a ladder: send every call to a cheap, capable model by default, and escalate to a flagship only the calls that fail. Most agent steps — scoped edits, boilerplate, formatting tool arguments, simple reasoning — don't need a frontier model. Paying frontier prices for them is waste you can delete in about 40 lines.

Because nearly every provider speaks the OpenAI Chat Completions format, you can build this with one SDK and two base URLs. Here's the whole pattern.

1. Two clients, one interface#

Point one client at a cheap open-weight endpoint and one at your flagship. Same SDK, same method calls — only base_url and api_key change. Here the cheap tier is Moonshot's Kimi K2.7 Code (~$4 per 1M output); the flagship is anything OpenAI-compatible.

import os, json
from openai import OpenAI, APIError, RateLimitError

cheap = OpenAI(
    base_url="https://api.moonshot.ai/v1",
    api_key=os.environ["MOONSHOT_API_KEY"],
)
CHEAP_MODEL = "kimi-k2.7-code"

flagship = OpenAI(  # your escalation tier — GPT-5.6, or any OpenAI-compatible gateway
    base_url=os.environ.get("FLAGSHIP_BASE_URL", "https://api.openai.com/v1"),
    api_key=os.environ["FLAGSHIP_API_KEY"],
)
FLAGSHIP_MODEL = os.environ.get("FLAGSHIP_MODEL", "gpt-5.6")

2. Validate the shape, not just the status#

The dangerous failure isn't an exception — it's a 200 OK with a broken tool call. The cheap model returns something well-formed enough to pass, wrong enough to corrupt your run. So before you trust a response, check it: content present, and if you expected a tool call, that its arguments actually parse.

def is_usable(resp) -> bool:
    msg = resp.choices[0].message
    calls = getattr(msg, "tool_calls", None)
    if calls:  # if it called a tool, the arguments must be valid JSON
        try:
            for c in calls:
                json.loads(c.function.arguments)
        except (json.JSONDecodeError, TypeError):
            return False
        return True
    return bool(msg.content and msg.content.strip())

3. The fallback wrapper#

Try the cheap model. Escalate on a transport error, a rate limit, an unusable response, or a caller-supplied validate() that inspects the answer and rejects it. A per-call escalated flag lets you log which tier actually did the work.

def complete(messages, tools=None, validate=None):
    """Try cheap; escalate to flagship on failure or bad output."""
    try:
        resp = cheap.chat.completions.create(
            model=CHEAP_MODEL, messages=messages, tools=tools,
        )
        if is_usable(resp) and (validate is None or validate(resp)):
            return resp, "cheap"
    except (APIError, RateLimitError):
        pass  # fall through to the flagship

    resp = flagship.chat.completions.create(
        model=FLAGSHIP_MODEL, messages=messages, tools=tools,
    )
    return resp, "flagship"

That's the core. validate is where your judgment lives — a JSON-schema check, a "did the code compile" gate, a cheap heuristic. If it returns False, the same prompt is retried on the flagship, so a wrong-but-well-formed cheap answer never survives.

4. Two guardrails you'll want in production#

Cache-aware ordering. Put your stable system prompt and tool definitions first, variable context last, so repeated calls hit the provider's prompt cache. On Kimi K2.7 Code that drops cache-hit input from $0.95 to $0.19 per 1M — a free discount you only get if the prefix is stable.

An escalation cap. One pathological input can bounce every call up to the flagship and erase your savings. Count escalations per task and stop climbing past a ceiling:

class Budget:
    def __init__(self, max_escalations=5):
        self.max, self.used = max_escalations, 0
    def allow(self) -> bool:
        if self.used >= self.max:
            return False
        self.used += 1
        return True

Gate the flagship call behind budget.allow() and fail loud when it trips — a silent runaway is worse than a hard stop.

The economics#

If the cheap tier clears 80% of calls at a third of the flagship's output price and you escalate the other 20%, blended cost lands roughly 55–65% below running everything on the flagship — with no quality loss on the hard calls, because those still go to the flagship. Log which tier answered every call and tune the split from real data; the number that matters is your escalation rate, not anyone's benchmark.

If you'd rather not hand-roll it, LiteLLM's router ships fallbacks, retries, and per-model budgets as configuration — the same ladder, behind one gateway. Either way, the move is the same: stop paying frontier prices for work a cheap model already finishes.