Open your agent's request payload and look at how much of it is tool definitions. For an agent with a dozen or two tools — each with a name, a description, and a JSON parameter schema — that block is routinely thousands of input tokens. The model is stateless, so you resend all of it on every call. And it almost never changes between calls. That combination — large, stable, sent every turn — makes tool definitions the single highest-leverage thing in your prompt to cache. Here's how, on both major APIs, and how not to break it.

Why the tools block is the token you overpay for#

Prompt caching bills a previously-seen token at roughly 10% of a fresh input token — a 90% discount — on both Anthropic and OpenAI. The savings scale with how big and how repeated a chunk is. Your tool definitions are both: fat and identical on every turn of a multi-step run. System prompts qualify too, but tool schemas are usually the largest stable chunk in an agent request, which is why they're the first place to look.

The one rule that governs everything: caching is prefix-based. The cache matches from the very first byte of the request forward, up to a breakpoint. Everything before the breakpoint is cacheable as a unit; one changed byte early in the prompt invalidates the entire cache from that point on. So the goal is to get your tool definitions into a stable, unchanging prefix — and keep them there.

Anthropic: mark the tools block with cache_control#

On the Claude API you opt in. Anthropic assembles the cache prefix in a fixed order — tools → system → messages — so tools sit at the very front and cache most durably. Put a cache_control breakpoint on your last tool definition and the whole tools block is cached:

tools = [
    {"name": "search", "description": "...", "input_schema": {...}},
    # ... more tools ...
    {
        "name": "create_invoice",
        "description": "...",
        "input_schema": {...},
        "cache_control": {"type": "ephemeral"},   # breakpoint after the tools block
    },
]

The first request that writes the cache costs 25% more than normal for those tokens; every request after it reads them at the 90% discount. The cache has a 5-minute minimum lifetime (refreshed on each hit), or 1 hour with the extended option — comfortably long enough to cover a burst of turns in one agent run. For a step-heavy agent, that turns a per-turn full-price tools block into a one-time write plus near-free reads.

OpenAI: it's automatic — don't break the prefix#

OpenAI does this for you. Caching kicks in automatically for any request whose prefix exceeds 1024 tokens, and the cacheable prefix explicitly includes tool definitions and structured-output schemas. There's no markup to add. On GPT-5.5-class models the cache is retained up to 24 hours.

Because it's automatic, your entire job is prefix hygiene: keep the tools block byte-identical and at the front of the request across calls. OpenAI's own agent guidance is blunt about it — hold system instructions, tool definitions, and ordering constant between requests to preserve a long, stable prefix. If you're on OpenAI and not seeing cached tokens in your usage, something upstream of the tools is mutating between calls.

The cache-miss traps#

Almost every blown tool cache comes from the same handful of mistakes:

Prune before you cache#

Caching makes a big tool block cheap; it doesn't make it free, and it doesn't make it good. A bloated tool list still hurts selection accuracy and still writes a large cache. So cut first, then cache: expose only the tools a task actually needs — the same discipline behind how many tools an agent should have and dynamic MCP tool management. Trim verbose descriptions and giant enums. Then wrap the lean, stable result in a cache prefix.

Do both and the math flips. Tool definitions stop being a tax you pay on every turn and become a one-time cost you amortize across an entire run — which, on a long agent trajectory, is most of your input bill. It's the same lever behind explicit vs. implicit prompt caching, pointed at the fattest stable block you have.