If your agent can call models in a loop, cap it in dollars per run, not tokens. Wrap every model call in a small cost accumulator that reads the response's usage object, converts tokens to dollars with per-model input/output rates, and adds it to a running total — then check that total before dispatching the next step and stop the loop the moment it crosses the ceiling. Tokens are the wrong unit to reason about; a dollar budget is the one that maps to your bill.

Why max_tokens is not a budget#

max_tokens caps the length of one response. Your agent isn't one response — it's a loop: plan, call a tool, read the result, call another, summarize. A run with ten round-trips can cost ten-plus times what a single response costs, and max_tokens never sees the accumulated total. It's also denominated in tokens, which tells you nothing about money once you're mixing models. At the time of writing, a frontier Anthropic model is roughly $5/$25 per million tokens in/out and a small model is a fraction of that — so 50,000 tokens might cost a dollar or a dime depending on which model produced them. Always read current rates from each provider's pricing page; never hardcode them as permanent.

The unit you actually care about is dollars, accumulated across the whole run. So build exactly that.

The price table and the usage object#

Every provider hands you the billed token counts on each response. Read them; never estimate.

Pricing is quoted per million tokens, with separate input, output, and cached-input rates. So the cost of one call is (input × inRate + output × outRate), scaled by a million, with cached input priced at its lower rate.

type Rates = { input: number; output: number; cachedInput: number };

// Dollars per 1M tokens. Illustrative only — read live values from each
// provider's pricing page; do not treat these as current or permanent.
const PRICES: Record<string, Rates> = {
  "claude-opus-4-8": { input: 5, output: 25, cachedInput: 0.5 },
  "gpt-mini":        { input: 0.15, output: 0.6, cachedInput: 0.075 },
};

type Usage = { uncachedInput: number; cachedInput: number; output: number };

// The two providers account for cached tokens differently.
function fromAnthropic(u: any): Usage {
  return {
    // input_tokens already EXCLUDES cached reads
    uncachedInput: u.input_tokens + (u.cache_creation_input_tokens ?? 0),
    cachedInput: u.cache_read_input_tokens ?? 0,
    output: u.output_tokens,
  };
}

function fromOpenAI(u: any): Usage {
  const cached = u.prompt_tokens_details?.cached_tokens ?? 0;
  return {
    uncachedInput: u.prompt_tokens - cached, // prompt_tokens INCLUDES cached
    cachedInput: cached,
    output: u.completion_tokens,
  };
}

That difference is a real trap: OpenAI's prompt_tokens includes the cached tokens, so you subtract them out; Anthropic's input_tokens already excludes them. Get this wrong and your accounting drifts on every cache hit.

The cost accumulator#

Wrap the whole thing in one class. It converts usage to dollars, keeps a running total, and throws when the total crosses the budget.

class BudgetExceededError extends Error {}

class CostTracker {
  spent = 0;
  constructor(readonly budgetUsd: number, readonly tenantId: string) {}

  add(model: string, u: Usage): number {
    const r = PRICES[model];
    if (!r) throw new Error(`No price configured for ${model}`);
    const cost =
      (u.uncachedInput * r.input +
        u.cachedInput * r.cachedInput +
        u.output * r.output) / 1_000_000;
    this.spent += cost;
    return cost;
  }

  // Call this BEFORE dispatching the next step — never after.
  assertUnderBudget(): void {
    if (this.spent >= this.budgetUsd) {
      throw new BudgetExceededError(
        `Tenant ${this.tenantId}: $${this.spent.toFixed(4)} of $${this.budgetUsd} spent`,
      );
    }
  }
}

Check before each step, not after the damage#

Here's the non-obvious part, and the one that separates a real budget from a spend report: you check the budget before dispatching the next step, not after.

Post-hoc accounting only tells you how much you already overspent. A single tool-call round-trip can blow a whole run's budget in one call — the check has to happen before you pull the trigger, not after.

Each agent step can be an expensive tool-call round-trip — the model reads a large tool result back into context, which is billed input. If you only add() after the call and check at the top of the next iteration, you've already paid. Gate the dispatch instead:

async function runAgent(tracker: CostTracker, messages: any[]) {
  let last = "";
  while (true) {
    tracker.assertUnderBudget();               // gate BEFORE spending
    const res = await client.messages.create({ /* model, messages, tools */ });
    tracker.add(res.model, fromAnthropic(res.usage));

    last = extractText(res);
    if (res.stop_reason !== "tool_use") return { status: "ok", text: last };

    messages.push({ role: "assistant", content: res.content });
    messages.push({ role: "user", content: await runTools(res) });
  }
}

Do the same per retry — retries multiply cost fast, so give them their own retry budgets rather than letting a flaky call quietly drain the run.

Per-tenant, per-run, and a graceful stop#

Instantiate one tracker per run, keyed to the tenant, so one customer's runaway loop can't spend another's headroom. When the budget is hit, don't let BudgetExceededError bubble up as a 500 — catch it and return a structured result with the spend and whatever partial output you have:

try {
  return await runAgent(tracker, messages);
} catch (e) {
  if (e instanceof BudgetExceededError) {
    return { status: "budget_exhausted", spentUsd: tracker.spent, partial: lastAssistantText(messages) };
  }
  throw e;
}

A dollar budget is your last, cleanest line of defense; pair it with coarser spend caps and rate limits at the account level so a single bug can't take the whole org down. One is a scalpel, the other a blast shield — you want both.

With agent software spending now a real line item, a per-run dollar cap is the difference between a predictable bill and a support ticket. The remaining rough edge is pre-flight estimation. This pattern is post-hoc: you know a call's cost only after it returns. For the rare step whose input is genuinely huge, count tokens first and estimate before dispatching — but for almost every agent, checking the running total before each step is enough. Denominate in dollars, wrap every call, check before you spend, and the next surprise on your bill won't be the agent.