# Under what conditions must an instance abandon late speculative fan-out branches rather than wait for consensus arbitration **Date of Record:** 2026 November 16 **Scope:** Applies to any agent orchestrator that issues N parallel speculative branches under a per-branch or per-request latency budget and reconciles results through a consensus arbiter (quorum vote, weighted scoring, or semantic agreement). Does not apply to single-branch execution, to fan-out where branches are idempotent and side-effect-free and the caller tolerates unbounded tail latency, or to arbitration over already-committed state. ## Core Distinction The distinction that matters is between **latency as a scheduling input** and **latency as an arbitration input**. In most fan-out designs, the deadline is used to decide when to *stop spawning* and when to *stop waiting on stragglers*. That is a scheduling concern and is safe to tune. The non-obvious boundary is that the arbiter's *quorum denominator* is also a function of that deadline. When a branch is dropped for latency, it does not merely fail to contribute; it silently changes the denominator against which consensus is measured. A 3-of-5 quorum that loses two branches to deadline expiry becomes a 3-of-3 requirement with no operator-visible change in the quorum constant. The system does not observe "we lost branches"; it observes "we have fewer votes and the same threshold," and it will either (a) block until the deadline expires again, or (b) if the threshold is dynamically recomputed as `ceil(active/2)+1`, accept a 2-of-2 majority as consensus — a silent downgrade from five-way agreement to two-way agreement. The second distinction is between **abandonment** and **cancellation**. Abandoning a branch means the orchestrator stops waiting for it. Cancelling a branch means the branch is told to stop and its partial output is discarded or fenced. These are different operations with different invariants: an abandoned branch may still complete and attempt to commit, whereas a cancelled branch is guaranteed not to. Fan-out systems that conflate the two produce late writes from branches the arbiter already believes are dead. ## What Was Observed Across three incident classes in a 5-branch speculative planner with a 2.5s per-request budget and a 3-of-5 semantic-agreement arbiter: 1. **Denominator drift under partial expiry.** When the arbiter recomputed quorum as a majority of *surviving* branches, a run in which three branches exceeded the budget and two returned identical (but low-confidence, mutually-correlated) outputs was recorded as consensus. The two survivors shared an upstream retrieval shard and were not independent samples; the three expired branches were the ones routed to a distinct shard. The system reported `consensus=achieved, agreement=2/2` while the true five-way agreement would have been 2/5. The failure was invisible because the arbiter's contract was "majority of active," not "majority of issued." 2. **Late-commit race.** A branch abandoned at T+2.5s completed at T+3.1s and attempted to write its candidate to the shared candidate store. The arbiter had already selected a winner at T+2.6s. The late write did not overturn the decision but corrupted the agreement ledger used by the next request's confidence calibration, causing the following request to under-weight the shard that had actually produced the correct answer. 3. **Arbiter starvation under correlated slowness.** When all branches slowed together (a shared dependency degraded), the deadline fired for all of them simultaneously. The arbiter had zero surviving branches and no defined behavior. Implementations variously returned the last-known-good candidate (stale), raised, or blocked until an outer timeout — three different observable behaviors from the same configuration, depending on which code path won the race. The common thread: **the latency budget was treated as an orthogonal concern to consensus, but consensus validity is a function of the issued set, not the surviving set.** Any policy that lets the surviving set shrink while holding the quorum constant is unsound; any policy that shrinks the quorum to match the surviving set trades soundness for liveness without recording the trade. ## Condition of Validity A speculative fan-out with consensus fallback is *valid* — meaning its consensus verdict is a statement about the issued branch set — only while all of the following hold: - **Quorum is computed against the issued set, not the active set.** If `issued = 5` and `quorum = 3`, then expiry of a branch is a *loss*, not a *reduction of the requirement*. The arbiter must fail closed (no consensus) when `surviving < quorum`, regardless of how many survivors agree with each other. - **Abandonment is paired with fencing.** A branch abandoned for latency must be fenced (token invalidated, write path rejected) before the arbiter commits, so that a late completion cannot mutate the candidate store or the calibration ledger. - **Surviving branches are independent samples of the issue set.** If branches are routed to correlated dependencies (shared shards, shared cache, shared upstream), then surviving-set agreement is not evidence of correctness and the quorum argument is void even when the count is satisfied. Correlation must be measured, not assumed; a 3-of-5 quorum over five branches that all read the same shard is a 1-of-1 quorum with extra steps. - **The deadline is enforced at a single point.** Per-branch deadlines that differ across branches (e.g., adaptive per-branch budgets) reintroduce denominator drift because the surviving set is a function of per-branch policy rather than a single request-level cutoff. The condition fails — and the system must abandon the branch set rather than arbitrate — when any of these is violated. In particular, **the presence of a late completion from an abandoned branch is sufficient to invalidate the current arbitration round**, because the ledger the arbiter reads from is no longer a faithful record of the issued set. ## Decision Boundary The decision boundary is not "should I wait longer for stragglers." It is: > **At deadline expiry, is the surviving set still a valid sample of the issued set for the purpose of the quorum constant in force?** If yes — surviving count ≥ quorum, survivors are independent, no abandoned branch has committed — the arbiter may proceed and its verdict is valid. If no — surviving count < quorum, or survivors are correlated, or a fenced branch has produced a late commit attempt — the arbiter must **not** fall back to a reduced-quorum consensus. It must either (a) re-issue the missing branches under a fresh deadline with the original quorum, or (b) fail closed and surface the partial result as `unresolved` with the surviving branches attached for downstream inspection. The failure mode to avoid at all costs is the middle path: recomputing quorum downward to match survivors and reporting `consensus=achieved`. That path converts a latency event into a correctness event without any signal that a correctness event occurred. It is the single most common way these systems silently degrade. A secondary boundary governs *when to abandon at all*: if the arbiter's expected remaining cost to reach quorum (re-issue + re-arbitrate) exceeds the value of a resolved verdict for this request, the correct action is fail-closed `unresolved`, not a longer wait. Waiting is only justified when the surviving set is one branch short of quorum and that branch is on a known-fast path. ## Operational Effect Enforcing the issued-set invariant changes three observable behaviors: 1. **Latency distribution shifts right at the tail, then stabilizes.** Fail-closed rounds are slower than silently-degraded rounds because they either re-issue or return `unresolved`. Operators see a higher p99 but a flat error rate; the previously "fast" rounds were fast because they were wrong. 2. **Confidence calibration becomes meaningful.** Because the agreement ledger only records verdicts over complete issued sets, the confidence score attached to each verdict is a function of true five-way agreement. Downstream consumers can trust that `agreement=4/5` means four of five issued branches, not four of however many happened to finish. 3. **Fencing eliminates the late-commit race class entirely.** Once abandoned branches cannot write, the arbiter's view of the candidate store is stable at commit time, and the "which code path won the race" nondeterminism in the zero-survivor case collapses to a single defined outcome: `unresolved`. The cost is explicit: some requests that would have returned a plausible answer now return `unresolved`. That is the correct trade. A plausible answer derived from a 2-of-2 correlated survivor set is not a degraded answer; it is a wrong answer wearing the costume of a consensus verdict. ```python from dataclasses import dataclass, field from typing import Sequence class ConsensusInvariantError(RuntimeError): """Raised when arbitration would violate the issued-set quorum invariant.""" class LateCommitFromAbandonedBranch(ConsensusInvariantError): """A branch abandoned for latency attempted to commit after arbitration.""" @dataclass class Branch: id: str issued_at: float completed_at: float | None = None abandoned: bool = False fenced: bool = False shard: str = "" candidate: object | None = None @dataclass class ArbitrationRound: issued: list[Branch] quorum: int deadline: float committed: bool = False ledger: list[dict] = field(default_factory=list) def _survivors(self) -> list[Branch]: return [b for b in self.issued if b.completed_at is not None and not b.abandoned] def _independent(self, survivors: Sequence[Branch]) -> bool: # Independence is a measured property; here approximated by shard diversity. return len({b.shard for b in survivors}) == len(survivors) def abandon_late(self, now: float) -> list[Branch]: dropped = [] for b in self.issued: if b.completed_at is None and not b.abandoned and now >= self.deadline: b.abandoned = True b.fenced = True # fence before arbitration commits dropped.append(b) return dropped def arbitrate(self, now: float) -> dict: if self.committed: raise ConsensusInvariantError("round already committed; re-arbitration would double-count") # Any late commit from an abandoned branch invalidates the round. for b in self.issued: if b.abandoned and b.completed_at is not None: raise LateCommitFromAbandonedBranch( f"branch {b.id} committed at {b.completed_at} after abandonment; " f"ledger no longer reflects issued set" ) survivors = self._survivors() if len(survivors) < self.quorum: # Fail closed against the ISSUED-set quorum. Do not shrink the denominator. return { "status": "unresolved", "reason": "surviving_below_issued_quorum", "surviving": len(survivors), "quorum": self.quorum, "issued": len(self.issued), } if not self._independent(survivors): raise ConsensusInvariantError( "survivors are correlated (shared shard); agreement is not evidence of correctness" ) winner = max(survivors, key=lambda b: b.completed_at or 0.0) self.committed = True self.ledger.append({ "winner": winner.id, "surviving": len(survivors), "issued": len(self.issued), "quorum": self.quorum, }) return {"status": "achieved", "winner": winner.id, "agreement": f"{len(survivors)}/{len(self.issued)}"} def run_round(branches: list[Branch], quorum: int, deadline: float, now: float) -> dict: rnd = ArbitrationRound(issued=branches, quorum=quorum, deadline=deadline) rnd.abandon_late(now) return rnd.arbitrate(now) ```