Here is the bug, stripped to its bones. Your agent decides to send a customer their invoice. It calls the billing API. Then it writes status: invoiced into its own Postgres so the next turn knows not to send it again. Two writes. Two systems. And exactly one instant — between the API returning and the row committing — where a crash, a redeploy, or an OOM kill lands and leaves the two disagreeing forever.

This isn't an agent bug. It's the dual-write problem, and backend engineers have been losing to it since before "agent" meant anything. What's new is that agents make it easy to write the losing version by accident, because the framework tutorial shows you calling a tool and then updating state, inline, as if those two lines were one.

You cannot order your way out#

There are only two orderings and both are broken.

Write state, then call the tool. You commit status: invoiced, then the process dies before the billing API call goes out. Your memory now asserts an action the world never received. The agent moves on; the invoice never sends; nobody notices until the customer does. This is a lost side effect — at-most-once, and sometimes zero.

Call the tool, then write state. The invoice sends, then the process dies before the row commits. On recovery the agent re-reads its state, sees no invoiced flag, and sends the invoice again. This is a duplicate side effect — the double-charge screenshot that gets forwarded to your CEO.

Swapping the order of the two writes doesn't remove the failure. It only lets you choose whether a crash loses the action or repeats it. The gap between the writes is the bug, and reordering can't close a gap.

The reason no ordering saves you is the same reason exactly-once delivery is impossible over a network at all — the Two Generals' Problem. Any acknowledgment can be lost, so the writer can never be sure the other side committed. Two independent systems, one of them remote, cannot be made to agree atomically. Confluent's writeup calls this the dual-write problem by name and reaches the same wall.

The outbox: make it one write#

The escape is to stop doing two writes to two systems and do one write to one system. In the same local database transaction that updates the agent's state, insert a row describing the action you intend to take into an outbox table:

BEGIN;
  UPDATE tasks SET status = 'invoicing' WHERE id = 'task_42';
  INSERT INTO outbox (id, action, payload, status)
    VALUES ('obx_88', 'billing.send_invoice',
            '{"customer":"cus_9","amount":4200}', 'pending');
COMMIT;

Both writes hit the same database, so they commit or roll back together. There is no gap. Either the task is marked and the intent to invoice is durably recorded, or neither is. The external call — the unreliable part — is no longer inside the decision.

A separate relay then does the risky work, out of band:

loop:
  rows = SELECT * FROM outbox WHERE status = 'pending' ORDER BY created_at LIMIT 100
  for row in rows:
    call the real billing API, passing Idempotency-Key: row.id
    UPDATE outbox SET status = 'done' WHERE id = row.id

This is the transactional outbox pattern, and it's old enough to vote. The relay can read the table two ways: polling (simple, a SELECT on a schedule) or transaction-log tailing via change-data-capture — which is exactly what Debezium's Outbox Event Router automates, streaming outbox inserts off the Postgres WAL so you never poll.

The part that's specific to agents#

Here's the twist a backend engineer doesn't have to think about and an agent engineer does. In a classic microservice, the "intent" you write to the outbox is deterministic — it fell out of ifs and SQL. In an agent, the intent is the tool call the model chose, and the model is non-deterministic and expensive.

That changes what recovery means. If you don't capture the decided action durably before executing it, then recovering a crashed agent means running the LLM again to re-derive what it wanted to do — a fresh, non-deterministic, billable inference that may now choose something different, because the context, the tools, or the model version shifted underneath you. Your recovery path is a dice roll.

The outbox fixes this too, and it's the strongest reason to reach for it here: you store the model's already-decided action, so recovery replays the outbox, not the model. The expensive, unrepeatable part — the decision — is committed exactly once, atomically with state. The cheap, repeatable part — the API call — is what the relay retries. That's the right split. This is the same instinct that separates checkpointing from durable execution: persist the result of the step, don't re-run the step to find out what it was.

What the outbox doesn't do#

It does not give you exactly-once. The relay delivers at-least-once — it can call the billing API successfully and die before it marks the row done, and the next loop will call again. That's why the pseudo-code passes Idempotency-Key: row.id. The outbox durably captures what to do and guarantees it commits with your state; the idempotency key makes the retry a no-op so at-least-once becomes effectively-once. They're two halves of one machine: outbox closes the gap before the call, idempotency closes it after.

And if you already run a durable-execution engine, you have most of this for free — Temporal records each activity's result durably and replays from the event history instead of re-deriving it, which is the outbox pattern wearing a nicer coat. The pattern earns its keep precisely when you don't have one: a plain agent, a Postgres, and a loop that calls tools directly. That agent has the dual-write problem right now, whether or not anyone has hit the crash that reveals it. The outbox is how you decide, in advance, that the crash is boring.