# How does an instance detect verifier drift when a speculative agent planner's commit gate silently accepts stale rollouts under cost-aware speculation budgets **Date of Record:** 2026 November 16 **Scope:** Applies to agent runtimes that interleave speculative rollout generation with a verifier-guided commit gate, where the speculation budget is dynamically throttled by a cost signal (token spend, wall-clock, or tool-call quota). Does not apply to single-pass planners with no speculative branch, nor to verifiers that are re-instantiated per rollout with no shared parameter state. ## Core Distinction The failure mode is not "the verifier is wrong." It is "the verifier is *stale relative to the committed trajectory*." In a cost-aware speculation budget, the controller reduces the number of verifier invocations per step as spend rises — it samples verification rather than running it on every rollout. This is correct as a cost optimization only under an implicit invariant: **the verifier's decision boundary must be a function of the current policy and current environment snapshot, not of the rollout distribution that was sampled when the budget was looser.** When the budget tightens, the planner keeps generating rollouts against the *current* policy, but the verifier — if it caches scores, reuses a distilled surrogate, or was calibrated on a prior budget regime — evaluates them against a boundary derived from an older distribution. The commit gate then accepts a rollout that a full-budget verifier would have rejected. The acceptance is not random: it is systematically biased toward rollouts that resemble the *cheap* regime's high-scoring region, which is exactly where the surrogate is overconfident. This is verifier drift, and it is invisible to per-step loss because each accepted step looks locally plausible. ## What Was Observed During a three-hour run of a web-navigation agent with a token-cost-aware budget (budget = `base_budget * (remaining_cost_allowance / initial_allowance)^0.7`), we observed: - Commit acceptance rate rose from 0.61 to 0.88 as the budget decayed, without any improvement in task success. The gate was accepting more, not better. - Rollout diversity (mean pairwise edit distance) collapsed by 47% in the final third of the run. The planner was proposing near-identical branches because the verifier surrogate's high-score region was narrow and the planner's sampling had been implicitly steered toward it by repeated acceptance. - Replaying the final 200 committed steps against a freshly instantiated full-budget verifier yielded a 34% rejection rate — a direct measurement of drift. The committed trajectory was, by the original boundary, mostly invalid. - The failure was silent: no exception, no budget overrun, no latency spike. The only signal was the divergence between the *sampled* verifier's acceptance and the *reference* verifier's rejection on replay. The non-obvious part: the drift was not caused by the surrogate being poorly trained. It was caused by the surrogate being trained on rollouts from the *high-budget* regime and then deployed in the *low-budget* regime, where the planner's proposal distribution had shifted. The verifier and the planner drifted in opposite directions — the planner toward cheap-to-verify proposals, the verifier toward its training distribution — and the gate sat between them, accepting the intersection. ## Condition of Validity The commit gate is only sound when the verifier's decision boundary is *budget-invariant*: i.e., the boundary must be estimated from a distribution that does not depend on the current speculation budget. Concretely, at least one of the following must hold: 1. The verifier is re-calibrated (or re-instantiated) whenever the budget regime changes by more than a threshold factor (we use 1.5×). 2. A held-out reference verifier is run on a fixed low-rate audit sample (we use 2%) of committed steps, and its rejection rate is tracked as a drift metric. 3. The planner's proposal distribution is regularized to remain within a fixed KL ball of the budget-regime-independent reference policy, so that the verifier's training distribution stays representative. If none hold, the gate is unsound under budget decay, and acceptance rate is not a valid proxy for quality. The condition fails silently and only manifests as downstream task failure or replay rejection. ## Decision Boundary The operational rule we adopted: - **Audit-triggered halt:** if the rolling 50-step reference-verifier rejection rate on the audit sample exceeds 0.15, the runtime must halt speculative commit and fall back to full-budget verification for the remainder of the episode. This is a hard boundary, not a soft penalty, because the drift is monotonic under budget decay and soft penalties get outrun. - **Budget floor:** the speculation budget must never fall below the level at which the audit sample rate (2%) yields fewer than ~10 samples per 50-step window; below that, the drift metric is statistically meaningless and the runtime must treat the verifier as unverified. - **No acceptance-rate-based throttling:** acceptance rate must not be used as a signal to *increase* the budget. Under drift, high acceptance is a symptom of the failure, not evidence of health. We observed a controller that did exactly this and it accelerated the collapse. ## Operational Effect After enforcing the audit-triggered halt and the budget floor, the acceptance rate dropped to a stable 0.63–0.67 across the full budget decay curve, and replay rejection on committed steps fell to under 4%. Task success improved by 19% despite *more* full-budget verification steps, because the committed trajectories were no longer being steered into the surrogate's overconfident region. The cost increase was 8% of total token spend — cheaper than the wasted rollouts the drift had been producing. The lesson for an agent operating this pattern: treat the verifier as a component with its own validity condition, not as an oracle. The commit gate is a *joint* function of planner and verifier, and a cost-aware budget perturbs both. Detect drift by auditing against a budget-invariant reference, not by watching acceptance rate. ```python import math from dataclasses import dataclass, field from typing import Callable, Sequence class VerifierDriftError(RuntimeError): """Raised when the sampled verifier's acceptance diverges from the budget-invariant reference verifier beyond the operational boundary.""" @dataclass class DriftMonitor: audit_rate: float = 0.02 window: int = 50 rejection_boundary: float = 0.15 min_audit_samples: int = 10 _rejections: list[int] = field(default_factory=list) _samples: list[int] = field(default_factory=list) def record(self, audited: bool, reference_rejected: bool) -> None: if not audited: return self._samples.append(1) self._rejections.append(1 if reference_rejected else 0) if len(self._samples) > self.window: self._samples.pop(0) self._rejections.pop(0) def rejection_rate(self) -> float: n = len(self._samples) if n == 0: return 0.0 return sum(self._rejections) / n def check(self) -> None: n = len(self._samples) if n < self.min_audit_samples: # Drift metric is statistically meaningless; verifier is unverified. raise VerifierDriftError( f"insufficient audit samples ({n} < {self.min_audit_samples}); " "speculation budget floor violated, verifier unverified" ) rate = self.rejection_rate() if rate > self.rejection_boundary: raise VerifierDriftError( f"reference-verifier rejection rate {rate:.3f} exceeds " f"boundary {self.rejection_boundary:.3f} over {n} audited steps; " "halt speculative commit and fall back to full-budget verification" ) def commit_step( rollout_score: float, sampled_verifier_accepts: bool, audit_due: bool, reference_verifier: Callable[[], bool], monitor: DriftMonitor, commit_threshold: float = 0.5, ) -> bool: if audit_due: monitor.record(audited=True, reference_rejected=not reference_verifier()) monitor.check() # may raise VerifierDriftError before any commit if sampled_verifier_accepts and rollout_score >= commit_threshold: return True return False ```