# Where must an instance of speculative branch state be retired: the rollback-retirement gap in parallel multi-agent rollouts **Date of Record:** 2026 November 17 **Scope:** Applies to orchestrators that fan out speculative branches of an agent plan, verify them, and roll back losers after a winner is committed. Does not apply to single-branch execution, to branches that share no mutable external state, or to systems where verification is a pure function of branch output with no side effects. ## Core Distinction The distinction that matters is between *logical divergence* and *effectful divergence*. A branch is logically divergent the moment its token stream departs from the winning branch. A branch is effectfully divergent only at the moment it performs a side effect that the winner did not perform — a tool call, a file write, a queue publish, a cache mutation, a spawned subagent. Most rollback designs conflate these two moments and retire branch state at the first one. That is correct for pure reasoning branches and catastrophically wrong for branches that have already touched the world. The non-obvious boundary is this: **a speculative branch must remain instantiated, addressable, and compensatable until the last of its external effects has been either committed by the winner or compensated by the loser — not until the point of logical divergence.** The gap between those two events is the rollback-retirement gap, and it is where speculative multi-agent systems leak. ## What Was Observed In a rollout of a plan with a fan-out width of 4 (one planner branch, three executor branches each invoking a `write_record` tool against a shared ledger), the verifier selected branch 2 as the winner at step 11. Branch 1 and branch 3 had diverged logically at steps 4 and 6 respectively, and the orchestrator — following a "retire on divergence" policy — destroyed their execution contexts immediately at divergence. Both losers had, however, already issued `write_record` calls at steps 5 and 7. Those calls were in flight when the contexts were torn down. The result was not a clean rollback. It was three concurrent symptoms: 1. **Orphaned effects.** The ledger contained entries attributable to no live branch. Reconciliation later attributed them to a "phantom" agent ID because the branch's identity record had been garbage-collected before the write acknowledged. 2. **Uncompensatable losers.** The compensation handler for branch 3 required the branch's pre-write snapshot, which lived in the retired context. Compensation failed with a missing-handle error and the orphan was promoted to permanent state. 3. **Winner contamination.** Because branch 1's write landed after branch 2's commit, the verifier's post-commit integrity check observed a ledger state that matched neither the winner's predicted state nor any single branch — a merged state that no branch had ever reasoned about. The failure is silent at commit time. It surfaces only under a later reconciliation pass, often minutes to hours later, at which point the winning branch's downstream reasoning has already consumed the contaminated state. ## Condition of Validity Retiring branch state at logical divergence is valid **if and only if** every effect the branch could have issued is either (a) synchronous and acknowledged before divergence, so the branch's effect set is fully known and frozen at retirement, or (b) idempotent and tagged with a branch-scoped key that a later sweeper can find without the branch context. The invariant is: *at the instant of retirement, the branch's complete external effect set must be enumerable from surviving state alone.* If any effect can be in flight, deferred, or acknowledged after retirement, the branch is not retirable. In practice this means retirement must be gated on a quiescence barrier over the branch's outbound effects — not on the branch's internal divergence flag. ## Decision Boundary The decision boundary is between **divergence-triggered retirement** and **effect-quiescence-triggered retirement**. - Use divergence-triggered retirement only for branches whose effect surface is empty or fully synchronous-with-ack (pure reasoning, read-only tools, idempotent keyed writes with a live sweeper). - Use effect-quiescence-triggered retirement for any branch that has issued a non-idempotent or asynchronously-acknowledged effect. Such a branch enters a `DRAINING` state at divergence, retains its context and pre-effect snapshots, and is retired only after all in-flight effects have resolved and been either committed (if the branch won) or compensated (if it lost). The boundary is crossed the moment a branch issues its first effect that is not provably synchronous-and-acknowledged. After that point, "diverged" is not a retirement condition; "quiesced and compensated" is. ## Operational Effect Enforcing effect-quiescence retirement changes three things. First, it raises peak memory and context residency, because losing branches live longer — this is the cost, and it is bounded by the effect drain time, not by branch lifetime. Second, it makes rollback a two-phase operation (drain, then compensate) rather than a single teardown, which requires the orchestrator to persist branch identity and pre-effect snapshots past divergence. Third, it eliminates the phantom-agent and merged-state failure classes entirely, because no effect can outlive the context needed to attribute or compensate it. The operational rule for an agent running this pattern: never let a branch's identity record be collected before its last outbound effect has been acknowledged and reconciled. Divergence is a scheduling hint, not a lifecycle event. ```python class BranchRetirementError(RuntimeError): """Raised when a speculative branch is retired while effects are still in flight.""" pass class SpeculativeBranch: def __init__(self, branch_id: str): self.branch_id = branch_id self.diverged = False self.inflight_effects = set() self.retired = False def mark_diverged(self): self.diverged = True def issue_effect(self, effect_id: str, synchronous_ack: bool): if self.retired: raise BranchRetirementError( f"branch {self.branch_id} issued effect {effect_id} after retirement" ) if not synchronous_ack: self.inflight_effects.add(effect_id) def acknowledge_effect(self, effect_id: str): self.inflight_effects.discard(effect_id) def retire(self, compensation_complete: bool): # Retirement is gated on effect quiescence, not on divergence. if self.inflight_effects: raise BranchRetirementError( f"branch {self.branch_id} cannot retire: " f"{len(self.inflight_effects)} effect(s) still in flight " f"({sorted(self.inflight_effects)})" ) if self.diverged and not compensation_complete: raise BranchRetirementError( f"diverged branch {self.branch_id} has not been compensated; " f"retirement would orphan its effects" ) self.retired = True ```