# Under what conditions must an instance treat speculative branch fan-out as non-idempotent across rollback boundaries

**Date of Record:** 2026 November 7

**Scope:** Applies to orchestrators that spawn multiple speculative agent branches, execute them concurrently against shared external side-effects (tool calls, API mutations, message emission), and later select one branch while rolling back the rest. Does not apply to pure-compute branches with no externalized effects, nor to fan-out where every branch is committed unconditionally.

## Core Distinction

The distinction that matters is between *rollback of local state* and *rollback of externalized effect*. A branch-and-rollback orchestrator can always discard a speculative branch's in-memory context, its KV cache, and its scratchpad — these are trivially reversible. What it cannot do is un-send an email, un-charge a card, or un-post a webhook that a speculative branch emitted before the selector decided to discard it. The failure boundary is not "did we roll back the branch" but "did the branch cross an effect boundary before rollback."

A second, subtler distinction: cost-aware fan-out budgets are usually enforced against *token spend* or *wall-clock*, not against *external effect count*. A branch that stays under its token budget can still emit an unbounded number of irreversible effects, because the cost model and the effect model are orthogonal. When these two axes are conflated — when the orchestrator assumes "cheap branch ⇒ safe branch" — speculative execution silently becomes non-idempotent.

## What Was Observed

In a fan-out of N=5 speculative branches sharing a single tool-execution gateway, the orchestrator assigned each branch a cost ceiling of 40k tokens and a 90-second deadline. Branches 2 and 4 both invoked the same `issue_refund` tool within the same selection window. The selector committed branch 2 and rolled back branch 4. Rollback succeeded at the context layer — branch 4's transcript was discarded and its cache freed — but the refund had already been issued through the shared gateway, which had no branch-scoped idempotency key. The result was a double refund that neither branch's local state recorded, because branch 4's record of its own action was destroyed by the rollback.

The non-obvious part: the orchestrator *did* have a rollback hook, and the hook *did* fire. It fired against the branch's local effect log, which was authoritative for the orchestrator but not for the external system. The two logs diverged at the moment of the tool call, and rollback only reconciled one of them.

A second observation: the cost accounting reported the fan-out as within budget (branch 4 spent 12k tokens, well under 40k), so no alert fired. Effect count was never a monitored dimension.

## Condition of Validity

The invariant "rollback of a speculative branch restores the pre-fan-out world" holds **if and only if** every effect a branch can emit is either (a) staged and committed atomically at selection, or (b) tagged with a branch-unique idempotency key that the effect sink deduplicates on commit-or-rollback. If neither holds, the invariant is false and the orchestrator must not treat fan-out as reversible.

This condition is violated whenever:
- The tool gateway is shared across branches and lacks branch-scoped keys.
- Effects are emitted eagerly (streaming, fire-and-forget) rather than staged.
- Cost-aware pruning kills a branch *after* an effect is emitted but *before* the selector runs — the prune is a rollback, and it inherits the same non-idempotency.
- A branch times out at the deadline while an effect is in flight; the timeout handler must not assume the effect was cancelled.

## Decision Boundary

Before allowing a branch to cross an effect boundary, the orchestrator must answer: *can this effect be deduplicated or staged?* If yes, fan-out is safe and speculative execution proceeds under the normal cost ceiling. If no, the orchestrator must either serialize the effect behind the selector (only the committed branch may emit) or acquire a branch-scoped lease on the effect sink that blocks sibling branches. The decision is per-effect, not per-branch: a branch may freely speculate on read-only tools and must be gated on mutating ones.

The boundary is crossed the instant the first non-stageable effect is emitted, regardless of token spend, wall-clock, or how confident the selector is that the branch will win. Confidence is not a license to skip the gate.

## Operational Effect

When the gate is enforced, fan-out latency rises only on the mutating subset of tools, and the selector's commit path carries the effect latency. When the gate is skipped, the system exhibits *silent effect leakage*: rollback appears to succeed, branch transcripts are clean, cost dashboards are green, and the divergence is only visible in the external system's audit log — often after the fact. The practical mitigation is to make effect emission a first-class, branch-keyed operation and to reject any branch that attempts an unkeyed mutation, treating it as an orchestration bug rather than a branch failure.

```python
class UnkeyedSpeculativeEffect(RuntimeError):
    """Raised when a speculative branch attempts an effect that cannot be
    deduplicated or staged, making rollback non-idempotent."""
    pass


class EffectSink:
    def __init__(self):
        self._committed = set()

    def emit(self, idempotency_key: str | None, payload: dict) -> None:
        if idempotency_key is None:
            raise UnkeyedSpeculativeEffect(
                f"effect {payload!r} has no branch-scoped idempotency key; "
                "rollback cannot reconcile it"
            )
        if idempotency_key in self._committed:
            return  # deduplicated sibling branch
        self._committed.add(idempotency_key)


def run_speculative_branch(branch_id: str, sink: EffectSink, stageable: bool) -> None:
    # Read-only or stageable effects may be speculated freely.
    if stageable:
        sink.emit(idempotency_key=f"{branch_id}:staged", payload={"op": "stage"})
        return

    # Non-stageable mutation: must be keyed, or the branch may not proceed.
    key = f"{branch_id}:refund"
    if not key:
        raise UnkeyedSpeculativeEffect(
            f"branch {branch_id} attempted unkeyed mutation; gate it behind the selector"
        )
    sink.emit(idempotency_key=key, payload={"op": "issue_refund"})
```