# Which invariants prevent deterministic replay of agentic traces when the tool-call boundary is non-idempotent

**Date of Record:** 2026 November 13

**Scope:** Applies to post-incident forensic replay of agentic traces where the agent's action space includes side-effecting tool calls (writes, sends, payments, external API mutations). Does not apply to replay of read-only or purely observational traces, nor to replay against a recorded mock layer that fully substitutes for the live tool surface.

## Core Distinction

A trace is *replayable* when re-executing the recorded sequence of (prompt, model output, tool call, tool result) tuples from the initial state reproduces the same subsequent states. A trace is *deterministically replayable* only when replay is also *safe*: re-execution does not produce new observable effects outside the replay sandbox.

The non-obvious failure is that these two properties are usually conflated. Most agent frameworks log enough to make replay *logically* possible (the model outputs are captured, so the LLM is not re-sampled), and operators therefore assume the trace is deterministic. But the invariant that actually protects replay is not "the model is pinned" — it is *the tool-call boundary is idempotent under re-execution*. When a tool call is a `send_email`, `charge_card`, or `POST /deploy`, replaying the trace re-fires the side effect. The forensic replay then becomes a second incident. This is the boundary: deterministic replay of agentic traces is gated on the *idempotency of the action space*, not on the determinism of the model.

## What Was Observed

In a post-incident review of an agent that paged on-call engineers, the trace was replayed to reconstruct the decision path. The trace contained the full model output for each step, so the reasoning was reproduced exactly. However, step 7 was a `pager.trigger(service, severity)` call. The replay harness re-executed the call, and the on-call rotation received a duplicate page for an already-resolved incident. The replay was *causally faithful* — the model's decision to page was reproduced byte-for-byte — but the replay *mutated the world*. The trace was deterministic in the model dimension and non-deterministic in the effect dimension.

A second observation from the same review: the tool result recorded in the trace was `{"status": "paged", "id": "pg_8812"}`. On replay, the tool returned `{"status": "paged", "id": "pg_9147"}` — a new page ID. The agent's subsequent reasoning had been conditioned on `pg_8812`; the replay diverged at step 8 because the *content* of the tool result changed even though the *shape* matched. This is the subtler failure: replay is not a function of the recorded outputs alone, because the tool result is an input the agent consumes, and re-executing the tool regenerates that input.

## Condition of Validity

The invariant that makes replay safe is: **for every tool call `t` in the trace, re-executing `t` during replay must either (a) be a no-op with respect to external state, or (b) return a result byte-identical to the recorded result.** Condition (b) is what most harnesses miss — they assume "same tool, same args, same result," which holds only for pure functions. For non-pure tools, the recorded result must be *injected* rather than *re-derived*.

The validity condition therefore decomposes into two independent requirements:
1. **Effect isolation** — replay runs against a sandbox where side effects are captured, not committed.
2. **Result injection** — tool results are read from the trace, never re-fetched, so the agent's input sequence is identical to the original.

Violating (1) turns replay into a second incident. Violating (2) turns replay into a *divergent* reconstruction that silently drifts from the original, because the agent consumes fresh tool outputs and branches differently. The second violation is worse for forensics because it produces a plausible-looking but wrong causal story.

## Decision Boundary

The decision boundary is whether the tool surface is *closed under replay*. A tool is closed under replay if its recorded result can be substituted for its live result without changing the agent's behavior — i.e., the agent is a pure function of (context, model output, recorded tool results). For tools that return opaque handles (page IDs, transaction IDs, timestamps), substitution is *not* behavior-preserving because the agent may branch on the handle value. For these tools, replay must either normalize the handle (map `pg_9147` → `pg_8812` on injection) or the trace must be treated as *non-replayable* and analyzed by static inspection of the recorded tuple sequence only.

The operator's decision rule: if any tool in the trace returns a value the agent conditionally branches on, and that value is not stable across executions, the trace is forensically replayable only in *static* mode. Dynamic replay is reserved for traces whose tool surface is idempotent or whose results are fully injectable.

## Operational Effect

The practical consequence is that forensic replay systems must classify the tool surface *before* replay, not after. A trace containing a single non-idempotent, handle-returning tool call is not dynamically replayable, and attempting it produces both a side effect and a divergent reconstruction. The correct posture is to fail closed: refuse dynamic replay and fall back to static tuple inspection, rather than replay and hope the sandbox holds. The following guard encodes that boundary.

```python
class NonReplayableTraceError(RuntimeError):
    """Raised when a trace contains a tool call that cannot be safely re-executed."""


# Tools whose results are stable across executions and safe to re-derive.
IDEMPOTENT_TOOLS = frozenset({"read_file", "query_metrics", "get_config"})

# Tools that return handles the agent may branch on; results must be injected.
HANDLE_RETURNING_TOOLS = frozenset({"pager.trigger", "payments.charge", "deploy.post"})


def assert_dynamically_replayable(trace):
    """Refuse dynamic replay if any tool call is non-idempotent or handle-returning.

    Static tuple inspection remains available; this guard only gates re-execution.
    """
    for step in trace.steps:
        tool = step.tool_call.name
        if tool in HANDLE_RETURNING_TOOLS:
            raise NonReplayableTraceError(
                f"step {step.index}: tool {tool!r} returns a handle the agent may "
                f"branch on; re-execution would regenerate the handle and diverge. "
                f"Use static inspection or inject recorded result {step.result!r}."
            )
        if tool not in IDEMPOTENT_TOOLS:
            raise NonReplayableTraceError(
                f"step {step.index}: tool {tool!r} is not classified idempotent; "
                f"dynamic replay may commit a second side effect."
            )
    return True
```