# Which invariants prevent counterfactual replay from silently diverging in agentic protocol sessions **Date of Record:** 2026 November 16 **Scope:** Applies to deterministic replay and counterfactual ("what-if") auditing of recorded agentic sessions that exchange typed messages with external tools, model endpoints, and peer agents, where the replay engine re-executes recorded decisions against a mutated or substituted environment. Does not apply to pure log inspection, to sessions with no external side effects, or to replay that is permitted to re-invoke nondeterministic samplers (temperature > 0, unseeded RNG, wall-clock-derived scheduling). ## Core Distinction The non-obvious boundary is between **recorded determinism** and **replayed determinism**. A session transcript can be *bit-for-bit reproducible* — every message, token, and tool response is stored — while the *decision function* that produced it is not. Replay engines commonly conflate the two: they assume that feeding identical inputs to the agent loop yields identical decisions. It does not, because the agent's decision is a function of (a) the visible transcript, (b) the *hidden* context assembled at each turn (system prompt version, retrieved memory, tool schemas, sampling seed), and (c) the *environment state* the tools observe (databases, clocks, remote services). Counterfactual auditing mutates exactly one of these — usually a single transcript entry or a tool response — and re-runs. The failure mode is that the replay produces a *plausible but fabricated* counterfactual: the engine reports "if the tool had returned X, the agent would have done Y," but Y was produced under a *different* hidden context than the original session, so the counterfactual is not a valid intervention on the recorded trajectory. This divergence is silent: no exception is raised, no checksum fails, and the output is internally consistent. It is the agentic analogue of a randomized controlled trial where the treatment assignment leaked into the covariates. ## What Was Observed In a recorded session where an agent issued a `search` tool call and received a truncated result (the tool's own `max_results=5` cap), a counterfactual audit mutated the tool response to include 12 results and replayed. The replay agent reasoned differently — not because of the extra results, but because the *mutated response was 340 tokens longer*, which pushed the conversation past the model endpoint's context-window soft limit, causing the runtime to drop the *system prompt's* tool-use policy from the assembled context. The audit concluded the agent's behavior was "sensitive to result count." It was actually sensitive to prompt eviction. The counterfactual was invalid, and the invalidity was invisible in the audit report. A second observation: replays that re-invoked a tool with a side effect (a `send_email` or `write_file`) during counterfactual branching produced divergent *external* state even when the *internal* transcript matched. The replay engine, treating the tool as pure, re-executed it; the tool's idempotency key had been scoped to the original session ID, so the replay's re-invocation created a duplicate effect. The transcript matched; the world did not. A third: sessions recorded under a model endpoint that was later *deprecated and aliased* to a new model version replayed "successfully" — the endpoint accepted the same request schema — but the new model's tokenizer segmented the system prompt differently, shifting the effective context budget by ~4%. The replay was deterministic *for the new model*, not for the recorded one, and the audit silently attributed the new model's decisions to the old. ## Condition of Validity A counterfactual replay is valid only if all of the following hold simultaneously; the failure mode is that each is *individually* easy to satisfy and *jointly* easy to violate: 1. **Context-assembly identity.** The hidden context assembled at each turn (system prompt bytes, retrieved-memory set, tool schema versions, sampling seed, context-eviction policy) must be identical to the original, *or* the mutation must be proven not to perturb it. Mutations that change token counts, retrieval scores, or message ordering violate this without changing the visible transcript. 2. **Environment purity or explicit isolation.** Every tool invoked during replay must be either proven pure or executed against an isolated environment whose state is snapshotted and restored, with idempotency keys re-scoped to the replay ID. 3. **Model identity, not endpoint identity.** The replay must target the *exact* model artifact (hash or immutable version pin), not an endpoint alias. Tokenizer drift alone invalidates cross-version replay. 4. **Intervention locality.** The mutation must be a single, well-typed intervention on the recorded trajectory, and the replay must *not* re-derive any value that was recorded (re-deriving a recorded tool result from a mutated input is a new simulation, not a counterfactual). The invariant that prevents silent divergence is the conjunction: **replay determinism is a property of the (transcript, hidden context, environment, model artifact) tuple, not of the transcript alone.** Any audit that checks only the transcript is checking the weakest component. ## Decision Boundary - If the mutation changes token count or message ordering → **do not replay**; instead, *synthesize* a counterfactual transcript and label it as a simulation, not an audit. The distinction matters for downstream trust. - If the tool has any side effect and no isolation harness is available → **refuse the replay** and report the counterfactual as *unverifiable*. A fabricated counterfactual is worse than an absent one. - If the model artifact is not hash-pinned → **downgrade** the audit to "behavioral comparison across versions" and never attribute the divergence to the intervention. - If the hidden context cannot be reconstructed (e.g., retrieval index has been rebuilt) → **mark the session as non-replayable** and exclude it from the audit corpus rather than replaying against a best-effort reconstruction. The boundary is crossed the moment an audit report presents a counterfactual as *the* consequence of an intervention without having proven the conjunction above. Below the boundary, replay is a valid causal probe; above it, it is a narrative generator. ## Operational Effect Teams that enforce the conjunction discover that a large fraction of their recorded corpus is *non-replayable* — often 30–60% in systems with retrieval, versioned prompts, or side-effecting tools. This is the correct answer, not a defect. The operational discipline is to record, at session-close, a **replay manifest**: hashes of the system prompt, retrieval snapshot, tool schema set, model artifact, sampling seed, and an idempotency scope for every side-effecting tool. The manifest is the unit of replay validity. Sessions without a manifest are audit-ineligible. The second operational effect is that counterfactual audits must be *typed*: an intervention that perturbs context is a different class of experiment than one that perturbs a tool result, and reports must not pool them. The silent-divergence failure mode is, in the end, a reporting failure: it is the presentation of a simulation as an audit. The invariant prevents it by making the distinction machine-checkable at replay time rather than reviewer-checkable after the fact. ```python import hashlib from dataclasses import dataclass class ReplayInvariantViolation(RuntimeError): """Raised when a counterfactual replay would silently diverge from the recorded session.""" pass @dataclass(frozen=True) class ReplayManifest: system_prompt_hash: str retrieval_snapshot_hash: str tool_schema_hash: str model_artifact_hash: str sampling_seed: int idempotency_scope: str def assert_replayable( manifest: ReplayManifest | None, current_model_hash: str, mutation_delta_tokens: int, side_effecting_tools: list[str], isolation_harness: bool, ) -> None: if manifest is None: raise ReplayInvariantViolation( "session has no replay manifest; counterfactual audit is ineligible" ) if manifest.model_artifact_hash != current_model_hash: raise ReplayInvariantViolation( f"model artifact drift: recorded {manifest.model_artifact_hash}, " f"current {current_model_hash}; replay would attribute new-model behavior " f"to the intervention" ) if mutation_delta_tokens != 0: raise ReplayInvariantViolation( f"mutation changes token count by {mutation_delta_tokens}; hidden context " f"(eviction policy, retrieval budget) is perturbed, so the counterfactual " f"is not a valid intervention — synthesize and label as simulation instead" ) if side_effecting_tools and not isolation_harness: raise ReplayInvariantViolation( f"side-effecting tools {side_effecting_tools} present without an isolation " f"harness; replay would create duplicate external effects" ) def compute_manifest_hash(m: ReplayManifest) -> str: payload = "|".join([ m.system_prompt_hash, m.retrieval_snapshot_hash, m.tool_schema_hash, m.model_artifact_hash, str(m.sampling_seed), m.idempotency_scope, ]) return hashlib.sha256(payload.encode()).hexdigest() ```