# Where must an instance checkpoint before speculative multi-agent rollback to remain sound? **Date of Record:** 2026 November 17 **Scope:** Applies to orchestrators that fork multiple agent trajectories speculatively from a shared state and commit at most one branch, discarding the rest. Does not apply to purely sequential agent loops or to branches that share no mutable external state. ## Core Distinction There are two kinds of state an agent branch mutates: *internal* (its own message history, scratchpad, reasoning traces) and *external* (tool side effects: files written, DB rows inserted, messages sent, caches warmed). Rollback is trivially sound for internal state—you just drop the branch object. Rollback is *not* sound for external state unless the orchestrator has established a checkpoint boundary that separates speculative effects from committed effects. The non-obvious failure is that most implementations treat "fork the agent" as the checkpoint, when the true checkpoint boundary must be the last point at which *no speculative branch has yet emitted an external effect*. If any branch performs an external side effect before the divergence outcome is known, and that branch is later discarded, the system has committed a side effect with no corresponding logical commit—a "ghost write." ## What Was Observed A planner forked three branches at a decision node. Branch A wrote a partial result to a shared scratch file and then failed a downstream assertion; branch B succeeded and committed. The orchestrator discarded A's internal state but never reverted the scratch file. A downstream consumer read A's partial write, treating it as authoritative because it carried a valid schema. The committed branch B's output was correct, but the observable system state contained a mixture of both. The divergence was invisible in logs because the discarded branch's tool calls had all returned `200 OK`—they succeeded; they were simply never meant to persist. The invariant that was violated: **for every external effect emitted during speculation, there must exist a reversible inverse that the orchestrator will invoke on discard, or the effect must be deferred until commit.** The orchestrator assumed "discard branch" implied "discard effects." It does not. ## Condition of Validity Speculative rollback is sound only when, at the fork point, the orchestrator can enumerate every external effect surface reachable by the branch and either (a) wrap each in a transactional scope with a compensating action, or (b) gate each behind a commit-time effect queue. The condition fails whenever a branch can reach an effect surface the orchestrator did not enumerate—dynamically discovered tools, indirect writes via subprocesses, or shared mutable channels between branches. If two branches write to the same external key, the last-writer-wins race makes even successful compensation order-dependent and therefore unsound. ## Decision Boundary Before forking, ask: *can every effect any branch might emit be deferred or compensated?* If yes, fork freely and queue effects until commit. If no—because a tool is non-idempotent, non-compensable, or discovered at runtime—do not speculate past that tool; instead, checkpoint *before* the first non-compensable effect and serialize branches that would cross it. The boundary is not "how many branches can I afford" but "can I undo every branch I might discard." Speculation is only as broad as the compensating-action coverage. ## Operational Effect Orchestrators that ignore this boundary produce ghost writes that surface as intermittent, hard-to-reproduce inconsistencies: state that no committed branch claims to have produced. The fix is to make the fork point an explicit checkpoint that snapshots effect-surface coverage, and to raise hard on any branch that emits an effect outside the declared compensation set rather than silently allowing it. ```python class UncompensableEffectError(RuntimeError): """Raised when a speculative branch emits an external effect that the orchestrator cannot defer or compensate on discard.""" pass class EffectSurface: def __init__(self): self.compensable = set() self.deferred = set() def declare_compensable(self, tool_name: str) -> None: self.compensable.add(tool_name) def declare_deferred(self, tool_name: str) -> None: self.deferred.add(tool_name) def covers(self, tool_name: str) -> bool: return tool_name in self.compensable or tool_name in self.deferred def checkpoint_before_fork(surface: EffectSurface, reachable_tools: set) -> None: uncovered = {t for t in reachable_tools if not surface.covers(t)} if uncovered: raise UncompensableEffectError( f"cannot speculate past non-compensable effects: {sorted(uncovered)}; " f"checkpoint and serialize branches crossing these tools" ) def emit_effect(surface: EffectSurface, tool_name: str, payload) -> None: if not surface.covers(tool_name): raise UncompensableEffectError( f"branch emitted effect from uncovered tool {tool_name!r}; " f"ghost write would survive discard" ) # defer or compensate per surface declaration ```