# Why does an instance of speculative multi-agent execution diverge from its rollback checkpoint under concurrent branch arbitration **Date of Record:** 2026 November 17 **Scope:** Applies to orchestrators that fork multiple speculative agent branches from a shared serialized state, allow each branch to invoke tools with side effects, and arbitrate a winning branch by comparison or vote before committing. Does not apply to purely read-only speculative decoding, nor to single-branch retry loops where no concurrent sibling exists. ## Core Distinction The failure is not a lost update and not a race on the checkpoint itself. It is a *rollback fidelity* violation: the committed state after arbitration is not equal to `checkpoint ⊕ winner_delta`, because one or more losing branches performed speculative tool invocations whose side effects were external to the checkpoint and therefore invisible to rollback. The checkpoint captures the agent's internal state (message history, scratchpad, variable bindings). It does not capture the world the tools touched. Rollback restores the former and silently leaves the latter mutated. The non-obvious part is *when* this manifests. It is invisible in unit tests where tools are stubbed as pure functions, and invisible in serialized execution where branches run one at a time. It appears only when (a) two or more branches are live concurrently, (b) at least one losing branch invokes a non-idempotent tool before arbitration resolves, and (c) the winning branch's subsequent reasoning depends on a read of that same external resource. The divergence is a *read-after-speculative-write* across branch boundaries — the winner observes a world state contaminated by a loser it never knew existed. ## What Was Observed A three-branch planner/executor stack was forked from a checkpoint at step 40. Branch B and Branch C each issued a `reserve_slot` tool call against a shared scheduler to test feasibility before committing to a plan. Branch A was selected by the arbiter. On commit, the orchestrator rolled back B and C by restoring the step-40 checkpoint and replaying A's delta. A's next action was a `list_reservations` read, which returned two slots that A had never requested. A's planner treated those as pre-existing constraints and produced a plan that double-booked capacity. No exception was raised at any layer: the checkpoint hash matched, A's delta applied cleanly, and the tool layer reported success for every call. The tell was not an error but a *nondeterminism across replays*: replaying the identical trace with branches forced serial produced a different final plan. That divergence between concurrent and serial execution of the same logical trace is the signature of this class. ## Condition of Validity The invariant `commit == checkpoint ⊕ winner_delta` holds if and only if every tool invoked by a losing branch is either (1) declared pure/idempotent and side-effect-free, or (2) compensated before the winner's next external read. Arbiters that assume checkpoint restoration is sufficient are operating outside the condition of validity the moment any branch holds a speculative side-effect lease. A practical test: for each tool, classify it as `PURE`, `IDEMPOTENT_WRITE`, or `EFFECTFUL`. If any branch can reach an `EFFECTFUL` tool before arbitration, the checkpoint is not a sufficient rollback boundary and the orchestrator must maintain a compensation log (undo entries) alongside the state checkpoint. ## Decision Boundary Choose *checkpoint-only rollback* when all reachable tools are `PURE` or `IDEMPOTENT_WRITE` keyed by branch id (so a losing branch's write is namespaced and inert). Choose *compensating rollback* — checkpoint plus an ordered undo log of speculative side effects replayed in reverse before commit — when any branch can reach `EFFECTFUL` tools. Choose *speculation fencing* — serialize access to `EFFECTFUL` tools behind a lease that only the arbiter can grant post-commit — when compensation is impossible (e.g., an email already sent). The boundary is crossed the instant a losing branch's side effect is observable to the winner; below that line, cheap checkpoint rollback is correct and should not be taxed with undo machinery. ## Operational Effect Orchestrators that ignore this boundary produce plans that are individually valid but globally inconsistent, and the inconsistency is *silent* — it surfaces downstream as capacity conflicts, duplicate notifications, or phantom resources days later. The mitigation is structural: tag tools with effect class at registration, refuse to fork a branch that can reach an `EFFECTFUL` tool unless a compensation entry is registered in the same transaction, and assert at commit that `len(undo_log_for_losers) == 0` after compensation. The assertion below encodes that gate. ```python class SpeculativeRollbackDivergence(RuntimeError): """Raised when a losing branch left uncompensated side effects observable to the winner.""" pass EFFECT_CLASSES = {"PURE", "IDEMPOTENT_WRITE", "EFFECTFUL"} def assert_rollback_fidelity(branches, winner_id, undo_log): """ branches: list of dicts with keys 'id', 'tool_effects' (list of effect-class strings) winner_id: id of the arbitrated branch undo_log: dict mapping loser branch id -> list of compensation records applied """ for br in branches: if br["id"] == winner_id: continue effectful = [e for e in br["tool_effects"] if e == "EFFECTFUL"] if not effectful: continue applied = undo_log.get(br["id"], []) if len(applied) < len(effectful): raise SpeculativeRollbackDivergence( f"branch {br['id']} reached {len(effectful)} EFFECTFUL tool(s) " f"but only {len(applied)} compensation(s) were applied before " f"commit of winner {winner_id}; checkpoint rollback is not a " f"sufficient boundary and the winner may observe contaminated state" ) return True ```