The moment your agent stops drafting text and starts doing things — approving a refund, denying an application, flagging a transaction, opening a ticket that pages someone — a new question lands on you that never came up with a chatbot: why did it do that? Not rhetorically. A customer, an auditor, or you-at-2am will ask, and "the model decided" is not an answer you can survive.

This is the practical companion to the agent frontier moving from chat to decisions. The fix isn't more logging bolted on after an incident. It's a decision audit trail: a structured record, written at the moment of decision, that you can replay months later to reconstruct exactly what the agent saw and why it acted. Here's how to build one.

The 30-second version#

1. The schema: what a decision record has to contain#

Design the record first — the agent code hangs off it. It needs five sections, all keyed to one id:

{
  "decision_id": "dec_01J9...",
  "created_at": "2026-07-11T09:14:22Z",
  "agent": { "name": "refund-agent", "version": "2026.07.02" },
  "inputs": {
    "snapshot_hash": "sha256:8f3a...",
    "data": { "order_id": "...", "amount_cents": 4200, "reason": "..." }
  },
  "rules": [
    { "id": "cap.auto_refund", "outcome": "pass", "detail": "4200 <= 10000" },
    { "id": "escalate.chargeback_history", "outcome": "fire", "detail": "2 prior chargebacks" }
  ],
  "model": {
    "provider": "claude-sonnet",
    "verdict": "escalate",
    "confidence": 0.71,
    "rationale": "Amount is within cap but chargeback history warrants human review."
  },
  "action": { "type": "escalated_to_human", "final": null },
  "override": null
}

Two design rules make this replayable rather than decorative. First, freeze the inputs. Store the exact data the agent saw plus a hash of it, so a replay runs against those facts, not whatever the database holds today. Second, keep rules and model strictly separate. A reviewer must be able to see at a glance which part of the decision was non-negotiable policy and which part was model judgment — that line is the whole point.

2. Rules fire before the model — and get logged either way#

Anything that is hard policy — a legal cap, a mandatory escalation, a sanctions hit — belongs in deterministic code that runs before you spend a token. If a rule forces the outcome, you may not need the model at all, and you've made the non-negotiable parts actually non-negotiable instead of hoping a prompt honors them.

from dataclasses import dataclass

@dataclass
class RuleResult:
    id: str
    outcome: str   # "pass" | "fire"
    detail: str

def evaluate_rules(inp: dict) -> list[RuleResult]:
    results = []

    # Hard cap: never auto-refund above $100.
    if inp["amount_cents"] > 10_000:
        results.append(RuleResult("cap.auto_refund", "fire",
                                  f'{inp["amount_cents"]} > 10000 -> block auto-approve'))
    else:
        results.append(RuleResult("cap.auto_refund", "pass",
                                  f'{inp["amount_cents"]} <= 10000'))

    # Mandatory escalation on chargeback history.
    if inp.get("prior_chargebacks", 0) >= 1:
        results.append(RuleResult("escalate.chargeback_history", "fire",
                                  f'{inp["prior_chargebacks"]} prior chargebacks'))

    return results

def rules_force_outcome(results: list[RuleResult]) -> str | None:
    fired = {r.id for r in results if r.outcome == "fire"}
    if "cap.auto_refund" in fired:
        return "deny"          # policy blocks auto-approval outright
    if "escalate.chargeback_history" in fired:
        return "escalate"      # must go to a human
    return None                # no hard rule; ask the model

Notice evaluate_rules records every rule it checked, passed or fired — not just the ones that triggered. "Rule X was evaluated and did not fire" is itself an auditable fact.

3. The model returns a structured verdict, never prose#

If the rules don't force the outcome, the model decides — but it hands back a structured object, not a paragraph. A verdict you can query and test; prose you can only re-read. Force the shape with a schema-constrained call (Pydantic + a tool/JSON-mode call here; the same idea works with any structured-output API).

from pydantic import BaseModel, Field

class ModelDecision(BaseModel):
    verdict: str = Field(pattern="^(approve|deny|escalate)$")
    confidence: float = Field(ge=0, le=1)
    rationale: str  # one sentence, for the record — not for the user

def model_decide(inp: dict) -> ModelDecision:
    # Your provider call, constrained to the ModelDecision schema.
    # The rationale is logged, not shown to the customer, so it can be
    # blunt ("thin file, weak signal") without being a support message.
    return call_llm_structured(
        system="You approve, deny, or escalate refund requests. "
               "Return a verdict, a 0-1 confidence, and a one-sentence rationale.",
        input=inp,
        schema=ModelDecision,
    )

Keeping the rationale in the record (not in the customer-facing message) is deliberate: it can be honest and diagnostic without being a communication you have to soften.

4. Assemble, write once, append-only#

Now stitch it together. The decision function's job is to produce a record and return the action — the two are never separated. Write the record append-only; a decision is a historical fact, and updating it in place destroys the trail. Corrections are new events (see the override below), not edits.

import hashlib, json, uuid, datetime

def decide(inp: dict) -> dict:
    decision_id = "dec_" + uuid.uuid4().hex
    snapshot = json.dumps(inp, sort_keys=True)
    record = {
        "decision_id": decision_id,
        "created_at": datetime.datetime.now(datetime.UTC).isoformat(),
        "agent": {"name": "refund-agent", "version": "2026.07.02"},
        "inputs": {"snapshot_hash": "sha256:" + hashlib.sha256(snapshot.encode()).hexdigest(),
                   "data": inp},
        "rules": [r.__dict__ for r in (rules := evaluate_rules(inp))],
        "model": None,
        "action": None,
        "override": None,
    }

    forced = rules_force_outcome(rules)
    if forced:
        verdict = forced                      # policy decided; model never called
    else:
        md = model_decide(inp)
        record["model"] = md.model_dump()
        verdict = md.verdict

    record["action"] = {"type": _action_for(verdict), "final": verdict}
    audit_log.append(decision_id, record)     # append-only store
    return {"decision_id": decision_id, "verdict": verdict}

That audit_log.append is the load-bearing line. Back it with an append-only table (Postgres with no UPDATE grant on the table, an event log, or an object-store write per id) so the record is immutable by construction, not by convention.

5. Capture the human override as a first-class event#

When the escalated 5% reaches a person and they change the call, that override is the most important row in the system — it's proof the human-in-the-loop exists and it's your best training signal. Record it against the same decision_id, as a new event, never by mutating the original.

def record_override(decision_id: str, reviewer: str, new_verdict: str, reason: str):
    audit_log.append_event(decision_id, {
        "type": "human_override",
        "reviewer": reviewer,
        "new_verdict": new_verdict,
        "reason": reason,
        "at": datetime.datetime.now(datetime.UTC).isoformat(),
    })

Now the full lineage of any decision — what the agent saw, which rules fired, what the model judged, what shipped, and whether a human corrected it — is reconstructable from one id.

The replay test: are you actually done?#

There's one check that tells you whether you have an audit trail or just logs. Pick any decision_id and reconstruct the entire decision from your own store, with no live external call. Same inputs (you froze them), same rules (they're in the record), same model rationale (structured, not paraphrased), same action, same override. If you can do that months later and explain the outcome to a stranger, you're regulator-ready — and, not coincidentally, you can now debug any bad decision and mine every override to make the agent better.

Most teams add this after their first ugly incident. On any surface where the agent acts, build it first: the decision record is your compliance story, your debugger, and your eval set at once. Ship the proof, then ship the authority to act — that's the order the fintech players already learned.