# Where must an instance of speculative multi-agent execution draw the boundary between a speculative branch and an optimistic tool invocation?

**Date of Record:** 2026 November 7

**Scope:** Applies to orchestrators that fork multiple agent branches against shared external state, invoke tools optimistically before branch consensus, and merge or roll back on a winning branch. Does not apply to purely read-only tool sets, nor to single-agent speculative decoding where the only side effect is token emission.

## Core Distinction

The non-obvious invariant is this: **a speculative branch is a rollback domain; an optimistic tool invocation is a compensation domain.** These are not the same thing, and the failure mode arises when an implementation treats them as identical — i.e., when the merge/rollback machinery assumes that "discarding a branch" automatically "undoes its tool effects."

Branch rollback is cheap and total because a branch's state is internal: its context, its scratchpad, its pending message queue. You can drop a branch and the world is unchanged. Optimistic tool invocation is neither cheap nor total: once a tool call leaves the process, the branch has emitted an *external commitment* whose reversal requires a compensating action that may be impossible, asynchronous, or itself fallible. The boundary that must be drawn is the moment a branch transitions from *speculating over internal state* to *holding an external commitment*. Everything before that transition is safely rollback-able; everything after it is only compensable, and compensation is a different, weaker guarantee.

The trap is that a branch-and-merge orchestrator is designed to make rollback look universal. It has a `discard(branch)` primitive that works perfectly for nine of ten branches, and the tenth branch — the one that fired an optimistic `charge_card` or `send_email` or `append_to_ledger` — gets "discarded" while its effect persists. The orchestrator reports a clean merge while the external world has accumulated orphaned commitments from losing branches.

## What Was Observed

In a three-branch speculative planner (branches A, B, C) racing to satisfy a user request, each branch was permitted to invoke tools optimistically as soon as it was confident, with the orchestrator merging the winning branch and rolling back the rest. The orchestrator exposed a single `rollback(branch_id)` that the merge logic called on all losers.

Under load, branch B invoked `reserve_inventory(sku, qty)` and then lost the merge to branch A. The orchestrator called `rollback(B)`. Branch B's context was discarded and its message queue purged — but the inventory reservation remained held. Because branch A's winning plan also reserved the same SKU, the system double-reserved, and the downstream fulfillment step observed phantom contention that no branch owned. The failure was not detected at merge time; it surfaced hours later as an inventory reconciliation mismatch attributed to "external" causes, because the orchestrator's own logs showed a clean rollback.

A second observation: branches that invoked the *same* idempotent tool optimistically (e.g., `fetch_price`) rolled back cleanly with no effect, which reinforced the false generalization that rollback is safe for all tool invocations. The invariant held for idempotent reads and broke silently for non-idempotent writes.

## Condition of Validity

The "rollback undoes the branch" model is valid **only** while the branch has emitted no external commitment. Formally: a branch is safely rollback-able iff its set of emitted tool invocations is empty, or every emitted invocation is idempotent *and* side-effect-free with respect to shared state. The moment a branch emits a non-idempotent or state-mutating invocation, it crosses into the compensation domain, and the correct operation is no longer `rollback` but `compensate` — which requires (a) a registered inverse action, (b) a durable record of the commitment keyed to the branch, and (c) an acknowledgment that compensation may fail and must be retried or escalated.

This condition is *not* satisfied merely because the tool is "usually fast" or "usually succeeds." Latency and success rate are orthogonal to reversibility. A fast, reliable, non-idempotent write is exactly the dangerous case: it almost always commits, so the orphaned commitment almost always exists.

## Decision Boundary

The boundary is a single predicate evaluated at tool-invocation time, not at merge time:

- If the invocation is **idempotent and side-effect-free** (pure read, cache fill keyed by content), permit it optimistically and let branch rollback cover it. No compensation record needed.
- If the invocation is **non-idempotent or mutates shared state**, the branch must either (1) defer the invocation until it holds the merge token (pessimistic), or (2) invoke optimistically *and* immediately register a durable compensation record bound to `branch_id` before the call returns. Rollback of such a branch must then trigger compensation, never silent discard.

The decision boundary must be drawn **before** the call is issued, because after it is issued you can no longer choose — you can only compensate. An orchestrator that decides at merge time has already lost the information it needs.

## Operational Effect

Drawing the boundary at invocation time converts an unbounded, silent failure (orphaned external commitments accumulating invisibly under a "clean rollback" log) into a bounded, observable one (a compensation queue that either drains or escalates). The cost is that optimistic branches carrying writes now pay a durable-write tax before their tool call — they must persist a compensation record, which serializes them slightly and reduces the parallelism benefit that motivated speculation in the first place. That tax is the correct price: it is strictly cheaper than reconciling phantom reservations days later. The orchestrator's merge logic must be rewritten so that `rollback(branch)` refuses to silently discard a branch with an open commitment, raising instead and forcing the caller to route through `compensate`.

```python
class UncompensatedCommitmentError(RuntimeError):
    """Raised when a branch holding an external commitment is rolled back
    as if it were a pure speculative branch. Rollback is not compensation."""
    pass


class Branch:
    def __init__(self, branch_id):
        self.branch_id = branch_id
        self.open_commitments = []  # durable records of non-idempotent tool calls

    def invoke_tool(self, tool, *, idempotent, side_effect_free, compensation=None):
        if not (idempotent and side_effect_free):
            if compensation is None:
                raise UncompensatedCommitmentError(
                    f"branch {self.branch_id}: non-idempotent tool {tool!r} "
                    f"invoked without a registered compensation action"
                )
            # Register the inverse BEFORE the call is issued.
            self.open_commitments.append({"tool": tool, "compensate": compensation})
        return tool()


def rollback(branch: Branch):
    """Discard a losing branch. Refuses if the branch holds commitments."""
    if branch.open_commitments:
        raise UncompensatedCommitmentError(
            f"branch {branch.branch_id} holds {len(branch.open_commitments)} "
            f"open commitment(s); route through compensate(), not rollback()"
        )
    branch.context = None  # safe: no external commitment was ever emitted


def compensate(branch: Branch):
    """Reverse external commitments. May itself fail; caller must retry/escalate."""
    for record in branch.open_commitments:
        record["compensate"]()  # inverse action, not a rollback
    branch.open_commitments.clear()
    branch.context = None
```