# Which invariants prevent silent divergence between speculative branches and the committed tool-effect log **Date of Record:** 2026 November 16 **Scope:** Applies to orchestrators that run parallel speculative agent branches, each of which may optimistically invoke side-effecting tools (writes, sends, payments, state mutations) before a verifier selects one branch to commit. Does not apply to read-only speculation, to pure-planning branches with no external effects, or to single-branch retry loops. ## Core Distinction Speculative execution is usually reasoned about as a *compute* problem: you fork N branches, you verify, you keep the winner, you discard the losers. The hidden assumption is that discarding a branch discards *everything the branch did*. That assumption holds for pure computation and fails for optimistic tool invocation. The distinction that matters is between **branch-local state** (reasoning traces, scratchpads, in-memory context, tentative outputs) and **world state** (the effect log of tools that actually fired against external systems). Branch rollback is total over the first and partial over the second. A tool call that returned `200 OK` cannot be un-called by dropping a branch; it can only be *compensated*, and compensation is a separate, fallible, and often non-commutative operation. So the real invariant is not "the committed branch is the only branch that took effect." It is the weaker and more honest invariant: **every effect produced by a non-committed branch must be accounted for — either compensated, or explicitly adopted by the committed branch, or recorded as an orphan with a reconciliation obligation.** Divergence is silent precisely when a branch is dropped without satisfying any of those three dispositions. ## What Was Observed In a system running three speculative branches against a shared tool bus, branch B issued an optimistic `create_refund` call to a payments tool while branch A (the eventual winner) issued a different `create_refund` for the same order. The verifier selected A on the strength of a cleaner reasoning trace. B was rolled back. The branch-local rollback succeeded completely — B's context was garbage-collected, its trace marked `discarded`, its metrics clean. The effect log told a different story: two refunds existed for one order. The orchestrator's commit record named only A. There was no record that B had ever touched the world, because rollback operated on the branch object, and the branch object did not own the effect. The tool bus had already returned, the external system had already mutated, and the only surviving trace of B's effect was inside the *tool's* audit log, which the orchestrator did not read. The failure was not a crash, a race, or a lost message. Every component behaved correctly under its own contract. The divergence was structural: the rollback boundary and the effect boundary were not the same boundary, and nothing enforced that they coincide. It surfaced days later as a reconciliation mismatch, attributed initially to the payments service. A second, subtler observation: idempotency keys were being derived from `(branch_id, tool, args)`. Because each speculative branch has a distinct `branch_id`, two branches issuing *semantically identical* calls produced *distinct* idempotency keys and therefore were not deduplicated. The idempotency layer was working exactly as designed and was, in this topology, actively harmful — it converted what should have been a deduplicated no-op into two real effects. Optimism plus per-branch idempotency keys is a combination that manufactures duplicate effects. ## Condition of Validity The "commit one branch, discard the rest" model is valid if and only if at least one of the following holds for every non-committed branch: 1. **Effect quarantine:** optimistic tool calls are staged (held, not fired) until commit, so a discarded branch has produced no world state. This is the only condition under which rollback is genuinely total. 2. **Compensation completeness:** every effect type the branch can invoke has a registered, idempotent, order-independent compensator, and rollback is defined as "run compensators for all effects of non-committed branches" rather than "drop the branch." 3. **Adoption:** the verifier's commit step explicitly reconciles the *effect sets* of all branches, adopting or compensating each effect, rather than selecting a branch and trusting its trace. Outside these conditions — i.e., optimistic firing with branch-scoped rollback and no effect-set reconciliation — the system is in the invalid regime. It will appear to work under low contention, single-branch wins, and read-only tools, and will diverge silently under exactly the conditions speculation exists to handle: parallel branches, shared resources, and side-effecting tools. A necessary corollary: **idempotency keys must be derived from the semantic intent (target resource + normalized args), not from the branch identity.** Branch-scoped keys defeat deduplication across speculative siblings. This is a precondition for condition 2 being achievable at all. ## Decision Boundary The operational rule is a boundary on *when a tool may fire optimistically*: - **Fire optimistically only if** the tool is classified *compensable* (a registered compensator exists and has been tested for idempotency and commutativity with sibling effects) **or** *adoptable* (the verifier is prepared to adopt the effect regardless of which branch wins). Read-only tools are trivially both. - **Stage, do not fire,** if the tool is *irreversible* (external notifications, physical actuation, non-refundable charges) or if its compensator is not proven order-independent against concurrent sibling effects. - **Refuse to speculate** if two branches can produce effects that are non-commutative and share a target resource — e.g., two branches both mutating the same order's refund state. In that case, serialize or partition; do not fork. The boundary is per *(tool, resource)* pair, not per tool. A refund tool is compensable against distinct orders and non-commutative against the same order. The classification must therefore be evaluated at dispatch time against the concrete resource, not statically against the tool name. ## Operational Effect Enforcing this boundary changes three things in practice. First, rollback is redefined: it is no longer "discard the branch object" but "dispose of the branch's effect set," and the orchestrator must hold a per-branch effect ledger that survives branch garbage collection. Second, the verifier's commit step becomes an effect-set reconciliation, and its correctness is measured by *effects accounted for*, not by *branch selected*. Third, idempotency keys move from branch scope to intent scope, which restores cross-branch deduplication and turns sibling duplicates into no-ops. The observable signature of the invalid regime is a growing gap between the orchestrator's commit log and the external systems' audit logs, with no corresponding error rate. It does not throw. It does not alert. It reconciles wrong, later, elsewhere. The guard below encodes the dispatch-time check that keeps a branch from firing an effect it cannot dispose of. ```python from dataclasses import dataclass, field from enum import Enum from typing import Optional class EffectDispositionError(RuntimeError): """Raised when a speculative branch would produce an effect it cannot dispose of.""" pass class EffectClass(Enum): READ_ONLY = "read_only" COMPENSABLE = "compensable" ADOPTABLE = "adoptable" IRREVERSIBLE = "irreversible" @dataclass(frozen=True) class EffectTarget: tool: str resource: str # concrete resource id, e.g. "order:8812" intent_key: str # semantic idempotency key, NOT branch-scoped @dataclass class BranchEffectLedger: branch_id: str fired: list = field(default_factory=list) staged: list = field(default_factory=list) def record_fired(self, target: EffectTarget) -> None: self.fired.append(target) def record_staged(self, target: EffectTarget) -> None: self.staged.append(target) def authorize_optimistic_dispatch( branch: BranchEffectLedger, target: EffectTarget, effect_class: EffectClass, compensator_is_order_independent: bool, sibling_targets: list, ) -> Optional[str]: """ Returns None if the branch may fire the effect optimistically. Returns "staged" if the effect must be held until commit. Raises EffectDispositionError if speculation is unsafe for this (tool, resource). """ if effect_class is EffectClass.READ_ONLY: return None if effect_class is EffectClass.IRREVERSIBLE: branch.record_staged(target) return "staged" if effect_class is EffectClass.COMPENSABLE and not compensator_is_order_independent: branch.record_staged(target) return "staged" # Non-commutative sibling effects on the same resource: refuse to fork. for sib in sibling_targets: if sib.resource == target.resource and sib.tool == target.tool: raise EffectDispositionError( f"branch {branch.branch_id}: non-commutative speculative effect on " f"{target.resource} via {target.tool}; serialize or partition instead of forking" ) # Intent-scoped idempotency precondition. if target.intent_key.startswith("branch:"): raise EffectDispositionError( f"branch {branch.branch_id}: intent_key {target.intent_key!r} is branch-scoped; " "sibling duplicates will not be deduplicated" ) branch.record_fired(target) return None def rollback_branch( branch: BranchEffectLedger, compensators: dict, ) -> None: """Dispose of a non-committed branch's effect set. Must account for every fired effect.""" for target in reversed(branch.fired): comp = compensators.get(target.tool) if comp is None: raise EffectDispositionError( f"branch {branch.branch_id}: fired {target.tool} on {target.resource} " "with no registered compensator; effect is now an orphan requiring reconciliation" ) comp(target) branch.fired.clear() branch.staged.clear() ```