# How should an instance reconcile speculative branches when rollback cost exceeds the speculative window

**Date of Record:** 2026 November 14

**Scope:** Applies to multi-agent orchestrators that execute speculative branches (parallel agent attempts against a shared or forked state) and later reconcile them by committing one branch and rolling back the others. Does not apply to pure read-only fan-out, stateless tool-call parallelism, or systems where every branch writes to an isolated store that is discarded wholesale without compensation logic.

## Core Distinction

Speculative execution is only safe when the cost of discarding a losing branch is bounded and known *before* the branch is launched. The non-obvious boundary is this: **rollback cost is not a property of the branch, it is a property of the external side effects the branch has already committed outside the transactional boundary.** A branch that is 90% complete but has written one record to a non-transactional downstream system (a webhook fired, a row inserted into a store the orchestrator cannot roll back, an email sent, a payment authorized) is *more expensive to abandon* than a branch that is 99% complete but purely internal.

The naive mental model — "speculate broadly, keep the winner, drop the losers" — silently assumes rollback is free. In practice, rollback cost grows monotonically with branch progress and is *discontinuous*: it is near-zero until the first irreversible side effect, then jumps to a fixed compensation cost that never decreases. This produces an invariant violation the scheduler does not expect:

> For any speculative branch `b`, `rollback_cost(b) <= speculative_window` must hold at the moment `b` is launched, where `speculative_window` is the maximum compensation budget the orchestrator is willing to spend. Because `rollback_cost` is a step function of progress, a branch can be *launched* within budget and *cross* out of budget mid-flight without any scheduling decision having been made to allow it.

## What Was Observed

In a three-agent speculative fan-out (planner, executor, verifier), the orchestrator launched three executor branches against forked state and planned to commit the branch whose verifier score was highest, compensating the other two. The speculative window was configured at 40 seconds of compensation budget per abandoned branch.

Branch B committed an external side effect (a downstream task creation) at t=12s, well inside its window. By t=38s branch A had the higher verifier score, so B was selected for rollback. Compensation of B required: (1) cancelling the downstream task, which itself was async and took 25s to acknowledge, and (2) reconciling a partial write that the downstream system had already replicated to a read replica with a 15s lag. Total compensation cost: 47s, exceeding the 40s window. The orchestrator had no protocol for "the loser is now too expensive to lose," so it either (a) blocked the commit of the winner until compensation completed, stalling the whole reconciliation, or (b) committed the winner and left the downstream task orphaned, producing a duplicate-effect bug.

The key observation: **the branch that was cheapest to abandon at launch was the most expensive to abandon at reconciliation.** Launch-time budgeting is the wrong time to measure.

## Condition of Validity

The "commit winner, roll back losers" reconciliation model is valid only when all of the following hold:

1. **Side-effect classification is known at launch.** Every tool or external call a branch can make is statically tagged `reversible` or `irreversible` (or `compensable` with a known cost bound). Untagged calls are treated as irreversible.
2. **Irreversible side effects are gated.** A branch may not execute an irreversible side effect until it has been *selected* for commit — i.e., the side effect is deferred to a post-selection phase, or is itself made idempotent and keyed by `(branch_id, effect_id)` so a loser's effect can be safely suppressed rather than compensated.
3. **Rollback cost is re-evaluated at selection time, not launch time.** The window check must run against the branch's *actual* committed side-effect set at the moment rollback is decided.
4. **The reconciliation protocol has an explicit overflow branch.** There must be a defined behavior for "loser cannot be rolled back within budget" — it is not a silent stall or a silent orphan.

If any of these fail, the system is running speculative execution without a valid rollback invariant, and correctness depends on luck.

## Decision Boundary

When a branch selected for rollback is found to exceed the window, the orchestrator must choose among exactly three outcomes — and must choose *explicitly*, logging the choice:

- **Block-and-compensate:** Halt the winning commit until compensation completes. Correct, but serializes the fan-out and can violate the winner's own latency SLO. Choose when downstream duplicate effects are unacceptable (financial, notification, idempotency-violating writes).
- **Commit-winner-and-quarantine:** Commit the winner, mark the loser's effects as `orphaned` in a reconciliation ledger, and hand them to an async compensator with retry. Choose when downstream effects are idempotent or eventually reconcilable. Requires the ledger to be durable and the compensator to be monitored.
- **Promote-the-loser:** If the loser is too expensive to abandon and the winner's advantage is within a noise band, commit the loser instead. This is the counterintuitive one: *rollback cost is a legitimate tiebreaker in branch selection.* A branch that is 2% worse but 10x cheaper to keep may be the correct commit.

The boundary condition for the third option is precise: promote the loser when `score(winner) - score(loser) < delta_threshold` **and** `rollback_cost(loser) > speculative_window`. Do not promote when the score gap is material; a cheap-to-keep but wrong branch is still wrong.

## Operational Effect

The practical consequence is that **speculative width must be a function of side-effect irreversibility, not of available compute.** Teams routinely scale fan-out up because parallelism is cheap and assume rollback scales with it. It does not: each additional branch adds an independent chance of crossing the irreversible threshold mid-flight, and the compensation cost of the abandoned set grows superlinearly once branches begin interacting through shared external systems (two branches creating tasks that reference each other's outputs cannot be compensated independently).

The invariant to enforce in code: before committing a winner, the orchestrator must compute the *aggregate* rollback cost of all losers against the window, and if it overflows, it must not proceed silently. The following guard encodes the failure this record describes.

```python
import time
from dataclasses import dataclass, field
from typing import Literal


class SpeculativeRollbackOverflow(RuntimeError):
    """Raised when abandoning a speculative branch would exceed the compensation budget."""
    pass


@dataclass
class SideEffect:
    effect_id: str
    kind: Literal["reversible", "compensable", "irreversible"]
    compensation_cost_s: float
    committed_at: float


@dataclass
class Branch:
    branch_id: str
    verifier_score: float
    effects: list[SideEffect] = field(default_factory=list)

    def rollback_cost(self) -> float:
        # Irreversible effects cannot be compensated; treat as unbounded.
        if any(e.kind == "irreversible" for e in self.effects):
            return float("inf")
        return sum(e.compensation_cost_s for e in self.effects if e.kind == "compensable")


def reconcile(
    branches: list[Branch],
    speculative_window_s: float,
    score_noise_band: float,
) -> Branch:
    if not branches:
        raise SpeculativeRollbackOverflow("reconcile called with no branches")

    ranked = sorted(branches, key=lambda b: b.verifier_score, reverse=True)
    winner = ranked[0]
    losers = ranked[1:]

    aggregate_rollback = sum(b.rollback_cost() for b in losers)

    if aggregate_rollback > speculative_window_s:
        runner_up = ranked[1]
        gap = winner.verifier_score - runner_up.verifier_score

        # Counterintuitive branch: rollback cost is a legitimate tiebreaker.
        if gap < score_noise_band and runner_up.rollback_cost() <= speculative_window_s:
            return runner_up

        # Otherwise we cannot silently proceed: the caller must choose
        # block-and-compensate vs commit-and-quarantine explicitly.
        raise SpeculativeRollbackOverflow(
            f"aggregate rollback cost {aggregate_rollback:.1f}s exceeds window "
            f"{speculative_window_s:.1f}s; winner={winner.branch_id} "
            f"score_gap={gap:.3f}; explicit reconciliation required"
        )

    return winner
```