The one idea: A published price of "$10 per million output tokens" is not a cost. It's a rate. Your cost is that rate multiplied by how many tokens the model actually emits — and that count changes with the model, the tokenizer, and your prompts. This week made the point unmissable: Claude Sonnet 5 shipped a tokenizer that can turn the same text into up to 1.35x more tokens, and GPT-5.6's Sol Fast tier runs at 750 tokens/second on Cerebras. Neither number lives on a rate card. You have to measure them.

Here's how, in code you can paste into a notebook today.

What to measure, and why each one decides something#

Four numbers turn a rate card into a real cost:

Cost per call falls straight out of the first two. UX quality falls out of the last two. Skip the measurement and you're optimizing a number (input price) that rarely drives either.

Step 1: read the tokens the meter actually counted#

Every major API returns a usage object. Read it — don't guess.

from openai import OpenAI

client = OpenAI()  # or any OpenAI-compatible endpoint

resp = client.chat.completions.create(
    model="gpt-5.6-terra",
    messages=[{"role": "user", "content": "Summarize the risks of agentic checkout in 3 bullets."}],
)

usage = resp.usage
print("input tokens :", usage.prompt_tokens)
print("output tokens:", usage.completion_tokens)

That usage block is the ground truth. A local tokenizer estimate can be off by exactly the margin that matters — the 1.0–1.35x spread Anthropic flagged for Sonnet 5 is the difference between a discount and a wash.

Step 2: time the stream for TTFT and throughput#

Latency only shows up when you stream. Start a clock, mark the first token, mark the last.

import time

def measure(model, prompt):
    t0 = time.perf_counter()
    ttft = None
    out_tokens = 0
    stream = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        stream=True,
        stream_options={"include_usage": True},  # get usage on the final chunk
    )
    usage = None
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            if ttft is None:
                ttft = time.perf_counter() - t0
        if chunk.usage:
            usage = chunk.usage
    total = time.perf_counter() - t0
    out_tokens = usage.completion_tokens
    return {
        "ttft_s": round(ttft, 3),
        "total_s": round(total, 3),
        "tok_per_s": round(out_tokens / max(total - ttft, 1e-6), 1),
        "in_tok": usage.prompt_tokens,
        "out_tok": out_tokens,
    }

include_usage makes the API attach the billed counts to the final stream chunk, so you get exact tokens and timing from a single call.

Step 3: turn tokens into dollars#

Keep rates in one dict, in dollars per million, and compute per call.

RATES = {  # $ per 1M tokens (input, output)
    "gemini-3.5-flash": (1.50, 9.00),
    "claude-sonnet-5":  (2.00, 10.00),   # intro rate through Aug 31
    "gpt-5.6-terra":    (2.50, 15.00),
}

def cost_usd(model, in_tok, out_tok):
    ci, co = RATES[model]
    return (in_tok * ci + out_tok * co) / 1_000_000

Now a single call gives you tokens, latency, throughput, and its true dollar cost — every dimension a rate card hides.

Step 4: compare models on YOUR traffic#

One call proves nothing. Run a representative sample — real prompts from your logs — through each candidate and compare the aggregates.

PROMPTS = [...]  # 50-200 real prompts from your app

for model in RATES:
    rows = [measure(model, p) for p in PROMPTS]
    total_cost = sum(cost_usd(model, r["in_tok"], r["out_tok"]) for r in rows)
    avg_ttft   = sum(r["ttft_s"] for r in rows) / len(rows)
    avg_tps    = sum(r["tok_per_s"] for r in rows) / len(rows)
    print(f"{model:20} ${total_cost:.4f}  ttft={avg_ttft:.2f}s  {avg_tps:.0f} tok/s")

This is the table that should decide a migration — not the vendor's. It's the only comparison that captures the tokenizer effect (it's baked into out_tok), the latency your users feel, and the cost your card actually pays. It's also exactly the harness you want before choosing between Terra, Sonnet 5, and Gemini 3.5 Flash — the rate card ranks them one way, your traffic may rank them another.

The payoff#

Run this once and two things happen. First, you stop being surprised by invoices — the "cheaper" model that emitted 30% more tokens shows up as more expensive in your own numbers, before you've committed to it. Second, you get a repeatable rig: next month, when the next model drops, you drop it into RATES, rerun the loop, and get a decision in minutes instead of a vibe. In a market where the frontier reshuffles every few weeks, the measurement harness outlasts every model in it.