# Which invariants prevent acceptance-rate collapse when heterogeneous drafters share a speculative verifier under parallel plan execution with consensus fallback

**Date of Record:** 2026 November 7
**Scope:** Applies to multi-agent speculative decoding topologies where two or more heterogeneous drafter models (different vocabularies, tokenizers, or temperature regimes) submit candidate token blocks to a single target verifier, and where those drafts are also consumed by parallel plan-execution branches that must reach consensus before committing side effects. Does not apply to single-drafter speculative decoding, nor to homogeneous drafter pools sharing an identical tokenizer and sampling distribution.

## Core Distinction

The non-obvious boundary is between **per-drafter acceptance rate** and **joint acceptance rate under a shared verifier**. Operators routinely tune each drafter's speculative window (γ) and temperature to maximize its own acceptance against the verifier, then assume the parallel branches inherit that efficiency. They do not. When heterogeneous drafters are merged at a single verifier, the verifier's rejection sampling operates on the *union* of candidate distributions, and the acceptance probability of any given token is governed by the minimum likelihood ratio across all drafters that proposed it, not the maximum. A drafter that is individually 70% accepted can drive the joint acceptance of a shared prefix toward 20% because its low-probability tail tokens poison the prefix for the high-acceptance drafters that share it.

This is a prefix-coherence effect, not a throughput effect. It is invisible in per-drafter dashboards and only surfaces when you measure acceptance *conditioned on a shared committed prefix*.

## What Was Observed

In a three-drafter deployment (a 1.5B code drafter, a 3B prose drafter, and a 7B reasoning drafter) feeding one 70B verifier, per-drafter acceptance rates were 0.68, 0.61, and 0.54 respectively. Joint acceptance on shared prefixes collapsed to 0.19. The mechanism: the reasoning drafter emitted low-probability "thinking-token" continuations early in the prefix; the verifier's rejection sampler, applying the standard min(1, p_target/p_draft) rule per position, rejected those positions, which invalidated the *entire* speculative block for all three drafters — including the two whose tokens would have passed. The parallel plan branches then diverged, triggering consensus fallback on ~80% of steps, and fallback latency (full verifier resampling plus a consensus round) dominated wall-clock time.

A second observation: consensus fallback itself masked the problem. Because fallback *always* produced a correct result, the system was functionally correct but ran at 0.4× the throughput of a single-drafter baseline. Correctness telemetry was green; only the acceptance-attribution metric (acceptance bucketed by which drafter proposed the rejected token) revealed the collapse.

## Condition of Validity

The invariant holds — and the collapse is *prevented* — only when all of the following are true:

1. **Prefix partitioning:** each parallel branch commits only tokens it proposed or that were proposed by drafters sharing its tokenizer. Cross-tokenizer prefix sharing must be prohibited at the block level, because token boundaries do not align and a "shared prefix" is not actually shared.
2. **Per-position rejection attribution:** the verifier records, for each rejected position, the identity of the proposing drafter, and the speculative block is truncated at the *first* rejection rather than invalidated wholesale — subsequent drafters re-propose from the truncation point.
3. **Acceptance-bucketed consensus threshold:** the consensus fallback triggers on *agreement-weighted* acceptance, not raw acceptance. A branch whose low-acceptance drafter was truncated should not be penalized in the consensus vote.
4. **Bounded heterogeneity:** the maximum pairwise KL divergence between any two drafters' next-token distributions on the shared prefix stays below a configured ceiling (observed safe range: < 0.35 nats). Beyond this, truncation thrash exceeds the cost of simply running the branches independently.

When any condition fails — most commonly (2), because naive implementations invalidate the whole block — joint acceptance is governed by the worst drafter, and the system degrades to slower-than-single-drafter throughput under a green correctness signal.

## Decision Boundary

The operational decision is: **truncate-and-repropose versus independent-branch execution.** Truncate-and-repropose is correct when the shared-prefix acceptance (conditioned, per condition 1) exceeds the independent-branch acceptance by more than the truncation overhead ratio — empirically, when shared-prefix acceptance > 0.45 and mean truncation depth < 2.5 tokens. Below that, the prefix is doing negative work: you pay verification cost on tokens that will be discarded, and the branches should run against independent verifier passes with consensus applied only at commit points. The boundary is not static; it must be re-evaluated per batch, because a single low-divergence drafter pair can flip the whole batch across the threshold.

## Operational Effect

With conditions (1)–(4) enforced, joint acceptance in the same deployment recovered from 0.19 to 0.52, consensus-fallback rate dropped from ~80% to ~14%, and end-to-end throughput rose to 1.7× the single-drafter baseline. The decisive change was per-position rejection attribution with truncation: it converted a worst-drafter-dominated joint acceptance into a best-effort prefix that each branch could extend independently. The lesson for operators: in heterogeneous speculative decoding, the shared verifier is a *min* operator over drafters, and the only way to keep the min from dominating is to stop sharing the prefix at the point where the drafters disagree.

```python
class SpeculativePrefixCollapseError(RuntimeError):
    """Raised when joint acceptance on a shared prefix collapses below the
    truncate-and-repropose decision boundary, indicating worst-drafter dominance."""
    pass


def should_truncate_and_repropose(
    shared_prefix_acceptance: float,
    mean_truncation_depth: float,
    independent_branch_acceptance: float,
    truncation_overhead_ratio: float = 0.15,
    acceptance_floor: float = 0.45,
    depth_ceiling: float = 2.5,
) -> bool:
    if not (0.0 <= shared_prefix_acceptance <= 1.0):
        raise SpeculativePrefixCollapseError(
            f"acceptance out of range: {shared_prefix_acceptance}"
        )
    if shared_prefix_acceptance < acceptance_floor:
        raise SpeculativePrefixCollapseError(
            "shared-prefix acceptance below floor; prefix is doing negative work"
        )
    if mean_truncation_depth > depth_ceiling:
        raise SpeculativePrefixCollapseError(
            "truncation thrash exceeds cost of independent branches"
        )
    marginal_gain = shared_prefix_acceptance - independent_branch_acceptance
    return marginal_gain > truncation_overhead_ratio


def execute_parallel_plan(branches, verifier, consensus):
    decisions = []
    for branch in branches:
        try:
            decisions.append(
                should_truncate_and_repropose(
                    shared_prefix_acceptance=branch.shared_acceptance,
                    mean_truncation_depth=branch.truncation_depth,
                    independent_branch_acceptance=branch.independent_acceptance,
                )
            )
        except SpeculativePrefixCollapseError as exc:
            branch.record_collapse(exc)
            decisions.append(False)
    if not any(decisions):
        return consensus.fallback(branches)
    return consensus.commit([b for b, d in zip(branches, decisions) if d])
```