Most agent reliability advice is about calls that fail: back off, retry, surface the error. That advice is correct and it is also the easy half. The hard half is the tool call that succeeds — and then the response gets lost on the way back to you.

A dropped connection. A timeout that fires after the server already committed. A gateway that 502s on the return trip. From the agent's side, all three look identical to a genuine failure, so the retry wrapper does exactly what you told it to: it runs the call again. If that call charged a card, you just charged it twice. This is the failure that turns a self-healing agent into a customer-support incident, and no amount of backoff fixes it — backoff makes retries politer, not safer.

Reads retry for free; writes don't#

The split is clean. A read — fetch a record, check a status — is naturally idempotent: run it a hundred times and the world is unchanged, so retrying is always safe. A write with a side effect — a payment, an order, a refund, an outbound email, a row insert — is not. Running it twice is worse than running it once.

A useful test before you let an agent retry a tool: if this call ran twice, would that be a problem? If yes, it needs protection before it goes near a retry loop. If no, retry away.

The fix: one key, generated once, reused on every retry#

An idempotency key is a unique token you attach to the operation so the server executes it at most once. Stripe's Idempotency-Key header is the reference implementation, and the mechanics are worth stating exactly:

So the retry after a lost response doesn't create a second charge. It returns the first charge's result. The duplicate becomes a safe no-op.

The idempotency key doesn't prevent the retry. It makes the retry harmless — which is the only thing you actually need.

The mistake that quietly defeats it#

Here's the trap agents fall into. You add idempotency, you wire a UUID into the call, you feel safe — and you generate the UUID inside the retry loop:

# BROKEN: a fresh key per attempt — every retry looks like a new operation
def charge(input_):
    def _do():
        key = str(uuid.uuid4())            # <-- regenerated on every attempt
        return stripe_charge(input_, idempotency_key=key)
    return with_retries(_do)               # retries now duplicate freely

Each attempt carries a different key, so the server sees each one as a brand-new operation and dedupes nothing. You've added a header and kept the bug.

The key must be created once per logical operation, before the first attempt, and reused unchanged across every retry:

# CORRECT: one key per operation, reused on every retry
def charge(input_):
    # bound to the operation, not the attempt; persisted so a replay reuses it
    key = input_["idempotency_key"]        # e.g. deterministic per (customer, purpose, period)
    def _do():
        return stripe_charge(input_, idempotency_key=key)
    return with_retries(_do)               # retries are now at-most-once

Make the key survive a restart#

There's one more failure an agent adds that a normal service doesn't: it can crash and replay the whole plan. If the key lived only in memory, the replay mints a new one and you're back to double-charging. Two ways to close that:

  1. Derive the key deterministically from the operation's identity — hash a stable descriptor like customer_id + purpose + period — so the same logical action always produces the same key, even across a restart.
  2. Or persist a random key with the operation's state before the first attempt, so a replay reads the original key back instead of generating a fresh one.

Either way, the invariant is the same: the key is a property of the operation, not of the attempt or the process.

What it means: idempotency is the one piece that makes agent retries safe for anything that touches money, inventory, or a customer's inbox. Retries and backoff — covered in AI agent tool-call error handling — handle the calls that fail loudly. The idempotency key handles the call that succeeds silently and then makes you pay for it twice. If your agent can spend money, this isn't optional hardening; it's the difference between a retry and a duplicate. (And if it can spend money, decide up front what to log when your agent spends money — the key is one of the fields you'll want.)