The short version: When a normal service fails, you open the stack trace and it points at the line that threw. When an agent fails, the stack trace is perfectly intact and almost useless — nothing threw. The model just decided to do the wrong thing: called the wrong tool, trusted a bad result, hallucinated an argument, or looped. The bug is a decision, and you can't grep a decision out of a 500. So you have to log the decision and everything the model saw when it made it. Here are the seven fields that turn "the agent broke" into a reproduction, and a record you can copy today.

We've written before on why agents fail in production; this is the observability companion — what to capture so that when they do, you can fix it instead of guessing.

The seven fields#

1. The resolved prompt — not the template

The most common agent failure is a context failure: the wrong document got retrieved, a stale value got inlined, the history got truncated one message too soon. If you logged the template with {{context}} still in it, you threw away the evidence. Log the fully assembled input the model actually saw — system prompt, retrieved context inlined, and the message list — after all substitution.

2. The model-chosen tool arguments

The model decides what to call and with what. Log those arguments exactly as emitted, before your code coerces or validates them. Wrong arguments are a reasoning problem; you can only see them if you captured the raw choice.

3. The raw tool result — and whether it was an error

Capture what the tool returned, verbatim, plus an explicit isError flag. This is the field people skip and regret: a correct-looking call that returned a misleading result is how a model gets confidently, catastrophically wrong. Separating "bad arguments" (field 2) from "bad result" (field 3) is separating two bugs with opposite fixes.

4. Model identity and decoding params

model, its version, temperature, seed (if you set one), and the stop_reason. Without these you cannot distinguish a model regression (the provider rolled a new snapshot) from a prompt regression (you shipped a change). One of those is your fault; you want to know which fast.

5. Versioned identity: tenant + agent + prompt

Beyond user_id, stamp the agent version and the prompt/template version. When failures spike, the first question is "what changed," and the answer is usually a version number. If your prompts aren't versioned, start there — it's the cheapest reliability win an agent product has.

6. Trace id + step index

An agent run is a loop, not a request. Give every step the same trace_id and a monotonic step index so the whole run reads as one story. Pair this with real distributed tracing across your tool calls and a failure becomes a flame graph you can walk backwards.

7. Cost, limits, and the terminal state

Tokens in/out, latency, budget remaining against the per-run spend cap, and the terminal state: done, asked_for_help, hit_cap, errored — with a reason. A run that halted because it hit a cap is a completely different incident from one that errored, and your dashboards should never blur them.

A record you can copy#

import json, logging

log = logging.getLogger("agent")

def log_step(*, trace_id, step, agent_ver, prompt_ver, tenant,
             model, temperature, seed, resolved_prompt, messages,
             tool_name, tool_args, tool_result, tool_is_error,
             tokens_in, tokens_out, latency_ms, budget_left,
             terminal_state, reason):
    record = {
        "trace_id": trace_id, "step": step,
        # 1 + 4 + 5: what the brain saw and who it was
        "resolved_prompt": redact(resolved_prompt),
        "messages": redact(messages),
        "model": model, "temperature": temperature, "seed": seed,
        "agent_version": agent_ver, "prompt_version": prompt_ver,
        "tenant": tenant,
        # 2 + 3: the action and its outcome, kept separate
        "tool_name": tool_name,
        "tool_args": redact(tool_args),        # what the MODEL chose
        "tool_result": redact(tool_result),    # what came BACK
        "tool_is_error": tool_is_error,
        # 6 + 7: shape, cost, and how it ended
        "tokens_in": tokens_in, "tokens_out": tokens_out,
        "latency_ms": latency_ms, "budget_left": budget_left,
        "terminal_state": terminal_state, "reason": reason,
    }
    log.info(json.dumps(record))   # one JSON line per step → your log pipeline

redact() is not optional — run every free-text field through a secret/PII filter before it's written, and sample successes (log 100% of failures, a few percent of the happy path) so volume stays sane.

The test#

Here's the bar, and it's a good one to hold every agent log line to:

Could a teammate who wasn't there reconstruct exactly what the agent saw, chose, and got back — and reproduce the failure — from your logs alone?

If yes, you're logging an agent. If all you have is a status code and a stack trace, you're logging a service that happens to contain a model, and the next production failure will be a mystery you get to solve twice. Capture the seven fields, hold the reconstruction bar, and "the agent broke" becomes a diff. When it's time to actually review one of these failures with your team, we wrote the postmortem playbook for autonomous agents next.