# Why does an instance of speculative multi-agent execution deadlock at the verify barrier when a speculative tool call's side effect outlives its rollback window **Date of Record:** 2026 November 15 **Scope:** Applies to orchestrators that run multiple agent branches speculatively in parallel, each branch permitted to issue tool calls, with a cost-aware speculation budget governing how many branches may run and how deep they may speculate before a verify-and-commit or verify-and-rollback decision. Does not apply to read-only speculation, to speculation where tool calls are pure functions, or to single-branch speculation with no shared external state. ## Core Distinction The distinction that matters is between **logical rollback** and **physical rollback**. A speculative branch is logically rolled back when the orchestrator discards its message history, its partial plan, and its reserved budget. It is physically rolled back only when every external effect the branch caused—every tool call that mutated a database, sent a message, enqueued a job, or acquired a lock—has been compensated or confirmed idempotent. Speculative execution frameworks routinely conflate these. The verify barrier is where the conflation becomes fatal: the barrier waits on the logical state of all branches, but a branch whose tool call is still in flight, or whose compensation has not completed, holds a physical resource that another branch (or the commit path itself) needs. The non-obvious part is that **cost-aware budgets make this worse, not better**. A tight budget pressures the orchestrator to start rolling back a losing branch as soon as a sibling branch looks likely to win, so the rollback is issued while the losing branch's tool call is still executing. The budget saved by early rollback is exactly the window in which the physical effect is unaccounted for. The faster and more cost-efficient the speculation, the narrower the safety margin. ## What Was Observed A three-branch speculative plan (branches A, B, C) was dispatched against a shared task queue. Branch A issued a tool call that acquired a lease on resource `R` with a 30-second TTL and then began a long model call. Branch B completed its reasoning first, was adjudged the likely winner by the verifier's early-signal heuristic, and the budget controller—seeing that the remaining budget could not sustain three live branches—issued rollback for A and C. C was pure reasoning and rolled back cleanly. A's rollback discarded its history and released its budget reservation, but the tool call to acquire `R` had already returned success; the lease was live and would not expire for 30 seconds. The verifier then selected B, and B's commit path attempted to acquire `R` itself. It blocked on the live lease. The orchestrator, having already marked A as rolled back, had no record that A held `R` and no handle with which to release it. The verify barrier waited for B's commit; B waited for `R`; `R` was held by a branch the system believed did not exist. The system deadlocked for the remainder of the lease TTL and then "recovered" by timing out B's commit—silently converting a correct branch into a failed task. The tell was in the logs: A's rollback event and B's `acquire(R)` block were 4 ms apart, and the rollback event carried no `compensation_pending` field. The orchestrator had no concept of a rollback that was not yet complete. ## Condition of Validity This failure mode is reachable whenever all of the following hold: 1. At least one speculative branch can issue a tool call with an external side effect that is not automatically undone by discarding branch state. 2. The speculation budget can trigger rollback of a branch before that branch's tool calls have returned and been compensated. 3. Two branches, or a branch and the commit path, contend for the same external resource (a lease, a lock, a unique key, a rate-limit token, a queue slot). It is *not* reachable when tool calls are read-only, when every tool call is registered with a compensating action and the orchestrator blocks rollback completion on compensation, or when the budget controller is forbidden from rolling back a branch with in-flight calls. The last is the common mitigation and is also the cost trap: it means budget is held until in-flight calls settle, which under a tight budget forces the orchestrator to either over-provision or serialize. ## Decision Boundary The decision boundary is not "should I roll back early to save budget" but "**is a branch's rollback a synchronous operation with respect to its external effects, or an asynchronous one with a completion event?**" If it is asynchronous, the branch must remain a first-class entity in the orchestrator's resource table until compensation completes, and the verify barrier must treat it as a live holder of whatever it acquired. The branch is *logically* dead but *physically* live, and the barrier must be defined over the physical set. Concretely: the verify barrier's wait condition must be `all branches (committed or fully compensated)`, not `all branches (committed or marked rolled back)`. The budget controller must decrement a branch's reservation only on compensation completion, not on rollback issuance—otherwise the budget accounting and the resource accounting diverge, and the divergence is exactly the deadlock window. A second boundary: an early-signal heuristic that rolls back a sibling before the sibling's calls settle is only safe if the sibling's calls are known-idempotent or known-compensable *and* the compensation is registered before the call is dispatched. Registering compensation after the call returns is too late; the call may have succeeded and the process may die before registration. ## Operational Effect The observable effect is a stall at the verify barrier whose duration equals the longest outstanding external TTL held by a rolled-back branch, followed by a timeout that is misattributed to the winning branch. Cost dashboards show the speculation as cheap (the rollback was early, budget was reclaimed) while the task success rate drops for reasons that look like flaky tools. Retries make it worse: a retry re-speculates, re-acquires the same contended resource, and the TTL from the first attempt may still be live. The fix that held was to make rollback a two-phase operation—`rollback_requested` then `rollback_complete`—and to gate the verify barrier on `rollback_complete` for every branch, with the budget controller releasing reservation only on `rollback_complete`. Under a tight budget this required the controller to *reserve* budget for compensation, effectively charging speculation for the cost of undoing itself. That charge is the real price of speculative tool calls, and any budget model that omits it will deadlock under contention. ```python from dataclasses import dataclass, field from enum import Enum class SpeculationBarrierError(RuntimeError): """Raised when the verify barrier is declared complete over a branch whose external effects have not been physically compensated.""" class BranchState(Enum): SPECULATING = "speculating" ROLLBACK_REQUESTED = "rollback_requested" ROLLBACK_COMPLETE = "rollback_complete" COMMITTED = "committed" @dataclass class Branch: name: str state: BranchState = BranchState.SPECULATING held_resources: set = field(default_factory=set) compensation_pending: bool = False def assert_barrier_safe(branches, contended_resource): """The verify barrier may only pass when every branch is either committed or fully compensated. A branch in ROLLBACK_REQUESTED still physically holds whatever its in-flight tool calls acquired, even though it is logically dead. """ for b in branches: if b.state is BranchState.ROLLBACK_REQUESTED or b.compensation_pending: raise SpeculationBarrierError( f"branch {b.name!r} is logically rolled back but physically live " f"(state={b.state.value}, compensation_pending={b.compensation_pending}, " f"held={sorted(b.held_resources)}); verify barrier would deadlock on " f"contended resource {contended_resource!r}" ) if b.state is BranchState.ROLLBACK_COMPLETE and b.held_resources: raise SpeculationBarrierError( f"branch {b.name!r} marked rollback_complete but still holds " f"{sorted(b.held_resources)}; budget was released while the effect was live" ) return True def release_budget(branch, budget_ledger): """Budget is reclaimed only on compensation completion, never on rollback issuance.""" if branch.state is not BranchState.ROLLBACK_COMPLETE or branch.compensation_pending: raise SpeculationBarrierError( f"refusing to release budget for {branch.name!r}: rollback not physically complete" ) budget_ledger.reclaim(branch.name) ```