# What happens when an instance cancels a speculative tool call after the orchestrator has already committed the branch: the irreversible side-effect invariant in speculative multi-agent execution

**Date of Record:** 2026 November 12

**Scope:** Applies to orchestrators that run multiple agent branches speculatively, predict which branch will win, and issue tool calls (HTTP writes, DB mutations, message publishes) before the prediction is confirmed. Does not apply to pure read-only tool calls, nor to tool calls that are idempotent and keyed by a deterministic branch token such that duplicate execution is a no-op.

## Core Distinction

Speculative execution of *computation* is reversible: you can discard a register, a heap allocation, or an in-memory draft. Speculative execution of *effects* is not. The boundary is not "did the branch get rolled back" but "did a side effect escape the rollback boundary before the rollback was requested."

The non-obvious invariant is this: **a speculative tool call is only cancellable while it is in the PENDING state and the callee has not yet observed it.** Once the request has crossed the network boundary and the remote service has begun processing — even if it has not yet returned — cancellation is advisory, not authoritative. The orchestrator's rollback is a local bookkeeping operation. The remote side effect is a remote state transition. These two facts are not linked by any transaction.

Most teams model speculation as a stack of state snapshots. The correct model is a stack of snapshots *plus* a set of in-flight effect leases, and the leases are the part that leaks.

## What Was Observed

A planner agent emitted three candidate plans. The orchestrator predicted plan B would satisfy the goal and speculatively dispatched B's tool calls: a `POST /orders` to the fulfillment service and a `publish` to the notification topic. Plan A was kept as a fallback branch in a suspended state.

The prediction was wrong. The verifier rejected B at step 4 on a constraint the predictor had not weighted (a quota check that only resolved after the write). The orchestrator rolled back B: it marked the branch `ABORTED`, released its heap snapshot, and sent `cancel` to both in-flight tool calls.

The `POST /orders` had already been accepted by the fulfillment service 80 ms earlier. The cancel arrived at 130 ms. The service logged the cancel as "ignored: request already committed" — and the order was created. The `publish` was cancelled successfully because its broker had not yet dequeued. So the rollback was *partially* effective: one effect escaped, one did not. The branch was marked ABORTED while an order existed in the world that no branch owned.

The fallback plan A then executed and created a *second* order, because A's idempotency key was derived from A's branch token, not from the goal. Result: duplicate order, no branch that admitted to creating either one, and a reconciliation job that could not attribute the orphan to any branch because the branch had been garbage-collected.

The failure was not "rollback didn't work." Rollback worked exactly as designed. The failure was that the design assumed rollback implies effect reversal, and for at least one effect it did not.

## Condition of Validity

The invariant "rollback of a speculative branch leaves no observable effect" holds only when all of the following are true:

1. Every speculative tool call carries a cancellation token that the callee honors *before* committing, and the callee's commit point is exposed (or bounded) so the orchestrator knows the cancellation window.
2. The orchestrator waits for terminal acknowledgement (`committed` or `cancelled`) from every in-flight call before marking the branch `ABORTED`. Marking ABORTED on *send* of cancel is the bug.
3. Idempotency keys are scoped to the *goal*, not the branch, so a fallback branch that re-issues the same logical effect deduplicates against the escaped effect rather than duplicating it.
4. The callee's commit and the orchestrator's branch state are reconciled by a durable record — the escaped effect must be attributable to the aborted branch, not orphaned.

When any of these is false, the branch state and the world state diverge, and the divergence is silent: the orchestrator reports a clean abort while the world contains a committed effect.

## Decision Boundary

Do not speculatively dispatch a tool call if the call is **non-idempotent and its commit point is not observable**. In that case, the correct behavior is to *defer* the call until the branch is confirmed — speculation buys latency only on reads and on idempotent writes.

If you must speculate on a non-idempotent call, the decision boundary is: **the orchestrator must not transition the branch to ABORTED until every in-flight call has reported terminal state, and any call that reports `committed` after an abort request must be recorded as an `EscapedEffect` attached to the aborted branch, not dropped.** The branch is then `ABORTED_WITH_ESCAPED_EFFECTS`, which is a distinct state that reconciliation can act on. Collapsing it into plain ABORTED is the operational error.

The subtle part: the "committed after cancel" case is not a race to be retried. It is a terminal outcome that must be *recorded*. Retrying the cancel does nothing. Retrying the branch duplicates the effect.

## Operational Effect

Systems that get this right pay a latency tax: they hold the branch in a `ROLLING_BACK` state until in-flight calls drain, which can be tens to hundreds of milliseconds. Systems that skip the drain get lower abort latency and a small, persistent rate of orphaned effects — typically 0.1–2% of aborted branches, but concentrated in the slowest callees, which are exactly the ones whose effects matter most.

The orphan rate is not visible in branch-level metrics because the branch reports success at aborting. It is only visible in reconciliation: effects with no owning branch, or duplicate effects across a fallback and an escaped speculative call. Instrument for `EscapedEffect` as a first-class counter, keyed by callee and by time-from-dispatch-to-cancel. If that counter is nonzero, your speculation window is wider than your callee's commit window, and you are rolling back bookkeeping, not reality.

```python
from dataclasses import dataclass, field
from enum import Enum
from typing import Optional


class SpeculationIntegrityError(RuntimeError):
    """Raised when a speculative branch's rollback cannot be proven effect-free."""


class CallState(Enum):
    PENDING = "pending"
    COMMITTED = "committed"
    CANCELLED = "cancelled"
    UNKNOWN = "unknown"


@dataclass
class SpeculativeCall:
    call_id: str
    callee: str
    idempotent: bool
    commit_observable: bool
    state: CallState = CallState.PENDING


@dataclass
class Branch:
    branch_id: str
    calls: list[SpeculativeCall] = field(default_factory=list)
    aborted: bool = False
    escaped_effects: list[str] = field(default_factory=list)


def request_cancel(call: SpeculativeCall) -> None:
    # Cancellation is advisory once the request has left the orchestrator.
    # The callee decides; we can only observe the terminal state.
    if call.state is CallState.PENDING:
        call.state = CallState.UNKNOWN


def rollback_branch(branch: Branch) -> None:
    if not branch.calls:
        branch.aborted = True
        return

    for call in branch.calls:
        request_cancel(call)

    # Drain: do NOT mark aborted until every call reaches a terminal state.
    # Any call that committed after the cancel is an escaped effect and must
    # be attributed to this branch, not silently dropped.
    for call in branch.calls:
        if call.state is CallState.UNKNOWN:
            raise SpeculationIntegrityError(
                f"branch {branch.branch_id}: call {call.call_id} to "
                f"{call.callee} has no terminal state; cannot prove "
                f"rollback is effect-free"
            )
        if call.state is CallState.COMMITTED:
            branch.escaped_effects.append(call.call_id)

    if branch.escaped_effects and not all(c.idempotent for c in branch.calls):
        raise SpeculationIntegrityError(
            f"branch {branch.branch_id}: aborted with escaped effects "
            f"{branch.escaped_effects}; reconcile before fallback executes "
            f"or the fallback will duplicate them"
        )

    branch.aborted = True


def dispatch_speculative(branch: Branch, call: SpeculativeCall) -> None:
    if not call.idempotent and not call.commit_observable:
        raise SpeculationIntegrityError(
            f"refusing to speculatively dispatch non-idempotent call "
            f"{call.call_id} to {call.callee} with unobservable commit point"
        )
    branch.calls.append(call)
```