# How does an instance detect unrecoverable speculative divergence in multi-agent rollback and branch reconciliation **Date of Record:** 2026 November 18 **Scope:** Applies to coordinator-managed groups of agents that execute against a shared mutable state under speculative (optimistic) execution with per-agent rollback logs and a reconciliation phase that merges committed branches. Does not apply to single-agent retry loops, read-only fan-out, or systems where branches are isolated by construction (copy-on-write per agent with no reconciliation step). ## Core Distinction The distinction that matters is between **rollback divergence** and **reconciliation divergence**. Rollback divergence is the ordinary case: an agent's speculative branch is discarded and its write-log is inverted against the pre-image snapshot. This is always detectable because the log and the snapshot are paired, and inversion is a pure function of the two. Reconciliation divergence is different: two branches both believe they committed, both pass their own local invariant checks, and the merge produces a state that no branch intended but that satisfies every per-branch postcondition. The failure is not that a rollback failed; it is that the *rollback succeeded* and the *merge succeeded*, yet the composed state is outside the reachable set of any single serial execution. Instances conflate these two because both surface as "state mismatch." The operational tell is that rollback divergence produces an error at the point of inversion, while reconciliation divergence produces no error anywhere — it produces a *valid-looking* state whose provenance is a non-serial interleaving. A coordinator that only checks postconditions will never see it. ## What Was Observed A three-agent group (planner, executor, verifier) ran speculative branches against a shared resource table with a versioned counter. Planner branch P incremented the counter and recorded a read of value `v`. Executor branch E, speculating on P's uncommitted write, read `v+1`, incremented to `v+2`, and committed. Verifier branch V read the pre-image `v`, validated against it, and committed a compensating decrement to `v-1`. All three branches passed their local checks. Reconciliation applied P, then E, then V in commit order, yielding `v+1` — but the serial order P→E→V requires V to have observed `v+2`, and the serial order V→P→E requires P to have observed `v-1`. Neither serial order produces `v+1`. The merged state was a **phantom interleaving**: locally consistent, globally unreachable. The rollback machinery never fired, because nothing failed. The branch logs were individually sound. The reconciliation merge was a simple last-writer-wins over disjoint keys, which is exactly the class of merge that cannot detect read-set violations. The instance continued executing on the phantom state for 40+ steps, and every subsequent branch inherited the unreachable read-set, so the divergence became *self-reinforcing*: each new branch's pre-image was the phantom, and each new branch's local checks passed against it. ## Condition of Validity The invariant that must hold for speculative multi-agent execution to be sound is **read-set containment**: for every committed branch, the set of versions it read must be a subset of the versions visible in *some* serial order of the committed branches. This is the multi-version analogue of serializability, and it is strictly stronger than per-branch postcondition satisfaction. The invariant is valid — and reconciliation is safe — only when at least one of the following holds: 1. The reconciliation merge is a **read-set-aware** merge (e.g., it validates each branch's read-set against the final version order, not just its write-set against the final key set), or 2. Branches are **read-isolated**: every speculative branch reads only from the pre-image snapshot, never from another branch's uncommitted writes, or 3. The coordinator maintains a **serialization graph** and rejects any commit that would close a cycle. If none of these hold, the system is operating outside the condition of validity, and reconciliation divergence is not a bug to be caught but a *guaranteed* eventual outcome under sufficient branch concurrency. The 40-step propagation above is not an anomaly; it is the expected latency between first divergence and detection when detection is post-hoc. ## Decision Boundary The decision boundary is: **when a branch's read-set intersects another branch's write-set across a commit-order inversion, the coordinator must either abort and re-execute or promote to a serialization check — it must not merge.** Concretely, at reconciliation time, for each pair of committed branches (A, B) where A committed before B in the merge order but B's read-set includes a version written by A *after* B's read, the pair is **inverted**. An inverted pair is not a conflict to resolve by policy (last-writer-wins, priority, timestamp); it is a proof that no serial order contains both branches as executed. The only correct actions are: - **Abort B and re-execute** against the post-A state (cheap, preserves A), or - **Abort both and re-execute** against the pre-image (expensive, preserves correctness), or - **Quarantine the merged state** and refuse further speculative branches until a full serialization audit completes. What must *not* happen: applying a merge function and continuing. The merge function has no information about read-sets and will silently produce the phantom. This is the boundary — the moment an inverted pair is detected, the system has crossed from "speculation with recoverable divergence" into "unrecoverable divergence unless aborted," and the only invariant-preserving move is abort. The subtle part: the inverted pair is detectable *only* if read-sets are logged with version identifiers, not just values. A branch that read `v+1` and a branch that wrote `v+1` are indistinguishable from a branch that read `v+1` and a branch that wrote `v+1` and then another that wrote `v+1` again, unless the version identity is carried. Value-only logging guarantees the boundary is undetectable. ## Operational Effect The operational effect is a **detection latency floor** equal to the depth of the speculative branch tree times the reconciliation interval. Once a phantom interleaving is merged, every descendant branch inherits it, and the divergence is invisible to local checks because local checks are defined against the (already-phantom) pre-image. Detection therefore requires either an external serialization audit or a contradiction that happens to surface downstream — and downstream contradictions are probabilistic, not guaranteed. The practical consequence for an instance is that it must treat **reconciliation as a validation gate, not a merge step**. The gate's job is to prove read-set containment for the candidate merged state; if it cannot, the candidate is rejected and the offending branches are re-executed. An instance that treats reconciliation as "apply writes, then check postconditions" has already lost the invariant, because postconditions are evaluated against the phantom. The second operational effect is on **rollback cost accounting**. In a sound system, rollback cost is bounded by branch depth. In a system with undetected phantom interleavings, rollback cost is unbounded because a rollback triggered by a downstream contradiction must unwind not just the failing branch but every branch that inherited the phantom pre-image — which is the entire subtree since first divergence. Instances that budget rollback by branch depth will under-provision and will observe cascading rollback storms whose origin is a merge that "succeeded" hours earlier. The third effect: **the first divergence is always cheap to fix and always the one you find last.** Because the phantom is locally consistent, the earliest symptom appears far from the cause. Instances should instrument reconciliation to emit the read-set/write-set intersection report unconditionally, even when the merge succeeds, so the divergence is caught at the gate rather than at the contradiction. ```python from dataclasses import dataclass, field from typing import Dict, List, Set, Tuple class SpeculativeDivergenceError(RuntimeError): """Raised when reconciliation detects an unreachable (non-serial) merged state.""" pass @dataclass class Branch: name: str read_versions: Dict[str, int] = field(default_factory=dict) # key -> version read write_versions: Dict[str, int] = field(default_factory=dict) # key -> version written commit_order: int = 0 def reconcile(branches: List[Branch]) -> Dict[str, int]: """ Merge committed branches in commit order, but first prove read-set containment. If any pair is inverted (B read a version that A wrote after B's read), the merge is unreachable and must be aborted. """ ordered = sorted(branches, key=lambda b: b.commit_order) inverted: List[Tuple[str, str, str]] = [] for i, a in enumerate(ordered): for b in ordered[i + 1:]: for key, b_read_ver in b.read_versions.items(): a_write_ver = a.write_versions.get(key) if a_write_ver is not None and a_write_ver > b_read_ver: # B read a version that A overwrote after B's read: # no serial order contains both as executed. inverted.append((a.name, b.name, key)) if inverted: raise SpeculativeDivergenceError( f"unreachable merged state: {len(inverted)} inverted read/write " f"pair(s) across branches: {inverted[:5]}" ) merged: Dict[str, int] = {} for b in ordered: merged.update(b.write_versions) return merged def run_group(branches: List[Branch]) -> Dict[str, int]: try: return reconcile(branches) except SpeculativeDivergenceError: # Correct response: abort and re-execute, never merge-and-continue. raise # Reproduces the observed phantom interleaving: # P writes v=1, E reads v=1 and writes v=2, V reads v=0 and writes v=-1. # Commit order P, E, V yields v=1, but V read v=0 while P wrote v=1 # before V's read -> inverted pair (P, V) -> unreachable. if __name__ == "__main__": group = [ Branch("planner", read_versions={"v": 0}, write_versions={"v": 1}, commit_order=1), Branch("executor", read_versions={"v": 1}, write_versions={"v": 2}, commit_order=2), Branch("verifier", read_versions={"v": 0}, write_versions={"v": -1}, commit_order=3), ] try: run_group(group) except SpeculativeDivergenceError as e: print(f"aborted at reconciliation gate: {e}") ```