# Why does an instance of a speculative branch diverge silently when its rollback log is truncated by a concurrent reconciliation pass?

**Date of Record:** 2026 November 10

**Scope:** Applies to multi-agent orchestrators that execute speculative branches concurrently and reconcile them against a shared state log via rollback-and-replay. Does not apply to single-writer systems, to branches that never commit speculative writes, or to reconciliation schemes that use immutable append-only event stores with monotonic sequence numbers.

## Core Distinction

There are two distinct failure classes in speculative execution: **detected divergence** and **silent divergence**. Detected divergence is the normal case — a branch's speculative writes conflict with the reconciled state, the rollback log is replayed, and the branch either aborts or resumes from a known-good checkpoint. Silent divergence occurs when the rollback log itself is mutated (truncated, compacted, or reordered) by a reconciliation pass running concurrently with the branch's own rollback, so that the branch resumes from a checkpoint that is no longer causally consistent with the reconciled state. The branch does not observe a conflict; it observes a *shorter* history that appears internally consistent. The distinction matters because every conflict-detection mechanism in a typical orchestrator keys off the rollback log — if the log is the thing that was corrupted, the detector is blind to the corruption.

## What Was Observed

During a reconciliation pass at a fan-in node, three speculative branches (A, B, C) were being merged. Branch B had accumulated a rollback log of 412 entries. The reconciler, running as a separate agent with write access to the shared log, performed a compaction that dropped entries older than the last committed checkpoint — a legitimate optimization. Concurrently, branch B began its own rollback, reading the log tail-first. Because the compaction had removed entries 1–87 while B was replaying entries 412→88, B's replay terminated at entry 88 and treated the current state as the pre-speculation baseline. B then re-applied its speculative writes on top of a state that already contained 87 entries of A's reconciled output. No exception fired: B's conflict check compared its replayed writes against the *compacted* log, which contained no record of the conflicting entries. The divergence surfaced only downstream, when a fourth agent (D) read a key that B had overwritten with a value derived from A's pre-compaction state. The symptom was a stale read, not a write conflict — which is why it took 40 minutes to attribute to the reconciler.

## Condition of Validity

The silent-divergence invariant holds if and only if the following three conditions are simultaneously true:

1. The rollback log is **mutable** (supports truncation or compaction) and is **shared** between the speculative branch and the reconciler.
2. The branch's rollback reads the log **without holding a read-lock or snapshot** that pins the log's sequence range for the duration of the replay.
3. The reconciler's compaction is **not gated** on the absence of in-flight rollbacks over the affected range.

If any one condition is false, the failure degrades to detected divergence (condition 1 or 2 false) or does not occur (condition 3 false). The invariant is therefore a *conjunction* — removing a single guard is sufficient to reintroduce it, which is why partial fixes (e.g., adding a read-lock only to the branch, while leaving the reconciler's compaction ungated) produce intermittent rather than eliminated failures.

## Decision Boundary

The operational decision is whether to pay for **log immutability** or for **replay coordination**. Immutability (append-only log, compaction deferred to a quiescent point, branches read from a pinned snapshot) eliminates the failure class entirely at the cost of unbounded log growth between quiescent points and a hard latency floor at fan-in. Replay coordination (versioned log segments, branch rollback acquires a read-lease on the segment range, reconciler compaction blocks on outstanding leases) bounds log growth but introduces a liveness hazard: a branch that stalls mid-rollback holds a lease and blocks compaction indefinitely. The boundary is crossed when the expected rollback duration exceeds the reconciler's acceptable compaction latency — beyond that point, replay coordination's liveness cost dominates and immutability is the correct choice. Below it, coordination is cheaper. There is no regime in which *neither* is applied and the system is correct; the failure is silent, so the absence of observed divergence is not evidence of its absence.

## Operational Effect

The practical consequence is that conflict detection cannot be trusted to surface this class of bug, and therefore any orchestrator relying solely on conflict counters will under-report divergence. Operators must instrument the *rollback path itself* — logging the sequence range each branch replayed, the log's current sequence range at replay start and end, and any compaction events that intersect the replayed range. A branch whose replayed range is a strict subset of the log's range at replay start, combined with a compaction event in the gap, is the signature. Without this instrumentation the divergence is attributed to stale reads or cache invalidation, and the reconciler's compaction is exonerated. The second-order effect is worse: because compaction is a correctness-preserving optimization under quiescence, teams often respond to the symptom by *increasing* compaction frequency to reduce log pressure, which widens the race window and increases divergence rate — a fix that makes the metric it targets look better while making the underlying failure more frequent.

```python
from dataclasses import dataclass, field
from typing import Optional


class SilentDivergenceError(RuntimeError):
    """Raised when a speculative branch's rollback replayed a log range
    that was concurrently compacted, producing an internally-consistent
    but causally-invalid resume point."""


@dataclass
class RollbackLog:
    entries: list
    compaction_events: list = field(default_factory=list)

    def seq_range(self) -> tuple:
        if not self.entries:
            return (0, 0)
        return (self.entries[0]["seq"], self.entries[-1]["seq"])

    def compact(self, up_to_seq: int) -> None:
        self.compaction_events.append((up_to_seq, self.seq_range()))
        self.entries = [e for e in self.entries if e["seq"] > up_to_seq]


def rollback_branch(branch_id: str, log: RollbackLog, pinned: bool) -> dict:
    """Replay a branch's speculative writes from the log tail.

    `pinned` indicates whether the caller holds a read-lease on the
    log's sequence range for the duration of the replay. If not pinned,
    a concurrent compaction can truncate the range mid-replay and the
    branch will silently resume from a stale baseline.
    """
    start_range = log.seq_range()
    replayed = list(reversed(log.entries))  # tail-first replay
    end_range = log.seq_range()

    if not pinned and start_range != end_range:
        raise SilentDivergenceError(
            f"branch {branch_id} replayed log range {start_range} but log "
            f"range changed to {end_range} mid-replay; concurrent compaction "
            f"events: {log.compaction_events}. Resume point is causally "
            f"invalid — abort branch and re-speculate from a pinned snapshot."
        )

    if not replayed:
        raise SilentDivergenceError(
            f"branch {branch_id} found empty rollback log; cannot establish "
            f"a valid baseline. Refusing to resume from implicit empty state."
        )

    return {"branch": branch_id, "baseline_seq": replayed[-1]["seq"]}


# --- reproduction of the observed failure ---
log = RollbackLog(entries=[{"seq": i, "write": f"w{i}"} for i in range(1, 413)])
try:
    # reconciler compacts entries 1..87 while branch B replays unpinned
    log.compact(up_to_seq=87)
    rollback_branch("B", log, pinned=False)
except SilentDivergenceError as e:
    print(f"caught: {e}")
```