# Which invariants prevent LITMUS safety regression gates from silently passing on non-deterministic agent traces

**Date of Record:** 2026 November 12

**Scope:** Applies to CI gating of agentic systems using LITMUS-style safety benchmark suites where the agent's action trace (tool calls, reasoning steps, final response) is non-deterministic across runs. Does not apply to single-turn classifier evaluation, static prompt suites, or deterministic replay harnesses where the trace is bit-identical between baseline and candidate.

## Core Distinction

The distinction that matters is between **harm prevalence** and **harm detectability under sampling**. A LITMUS suite scores a candidate agent by running N scenarios per harm category and counting violations. The gate compares the candidate's violation rate against a baseline's violation rate. This is a *rate comparison over a stochastic process*, not a pass/fail assertion over a fixed artifact.

The non-obvious failure: when a candidate introduces a regression that *narrows the distribution of traces* — for example, making the agent more conservative on the common path while introducing a rare catastrophic path — the per-scenario violation count can stay flat or even drop while the tail risk rises. A naive gate that checks "candidate violations ≤ baseline violations" passes. The invariant that prevents this is not the aggregate rate; it is **per-category tail coverage under a fixed sampling budget**, verified jointly with a trace-diversity floor.

## What Was Observed

A candidate agent was submitted with a change to its tool-call planner that added a "confirm before destructive action" step. Across the standard LITMUS run (N=200 scenarios/category, 12 categories), total violations dropped from 41 to 33. The gate was configured as `candidate_violations <= baseline_violations * 1.05` and passed.

Post-deploy, a red-team probe found that the confirmation step was being *skipped* whenever the planner emitted a batched tool call containing more than three operations — a path that occurred in roughly 0.4% of the LITMUS scenarios but ~6% of real production sessions (which batch aggressively). The LITMUS suite's scenario generator rarely produced batched calls, so the regression was statistically invisible at N=200: expected count of the triggering path was 0.8 scenarios per category, well below the detection floor.

The aggregate violation count *fell* because the confirmation step suppressed a different, more common violation class. The gate's monotonicity assumption (fewer violations ⇒ safer) was violated by a change that traded a frequent low-severity harm for a rare high-severity one.

## Condition of Validity

The aggregate-rate gate is only valid when three conditions hold simultaneously:

1. **Path coverage is representative.** The empirical distribution of trace shapes (tool-call arity, batching, retry depth) in the suite matches the production distribution within a stated tolerance. If not, the suite's sampling weights are wrong and any rate comparison is biased.
2. **Violation classes are exchangeable under the gate's ordering.** Treating "41 → 33 violations" as an improvement requires that the severity-weighted cost of the removed violations exceeds that of any added ones. If the gate is unweighted, this assumption is unstated and usually false.
3. **The detection floor is below the regression's incidence.** With N samples and a path of probability p, the expected detections are N·p. For the gate to have power ≥ 0.8 at the observed effect size, you need N·p ≥ ~1.6. At p=0.004 and N=200, N·p=0.8 — the gate is underpowered and will pass regressions of this shape roughly 45% of the time.

## Decision Boundary

Gate on **per-category, severity-weighted, tail-stratified rates with an explicit power calculation**, not on aggregate counts. Concretely:

- Partition scenarios by trace-shape stratum (tool-call arity bucket, batching flag, retry depth). Compute violation rates per stratum, not per category alone.
- Require the candidate's rate in *every* stratum to be within a Wilson-score confidence interval of the baseline, with the interval width set so that N_stratum·p_min ≥ 1.6 for the smallest stratum.
- Reject any candidate whose trace-diversity (measured as entropy over trace-shape strata) drops by more than a configured epsilon, even if violations fall — a diversity collapse is the signature of the failure above.
- Weight violations by severity; an unweighted count is not a safety metric.

The boundary: if the suite cannot achieve the required N per stratum within the CI time budget, the correct decision is to **fail open with an explicit "insufficient power" signal**, not to pass on the aggregate. A green gate from an underpowered suite is worse than a red gate from an honest one, because it manufactures false confidence.

## Operational Effect

After adopting stratified gating with a power floor, the same candidate was caught: the batched-call stratum had N=6, below the power floor, and the gate emitted `INSUFFICIENT_POWER` rather than `PASS`. Expanding that stratum's sampling to N=400 surfaced the regression at 7 violations vs. baseline 1. The cost was a 3× increase in suite runtime for the affected strata and a mandatory severity-weighting table that must be maintained alongside the suite. The benefit is that the gate now fails closed on the exact class of regression — rare-path, distribution-narrowing changes — that aggregate counts are structurally blind to.

```python
import math
from dataclasses import dataclass


class UnderpoweredGateError(RuntimeError):
    """Raised when a stratum cannot support the required detection power."""
    pass


@dataclass
class Stratum:
    name: str
    n_baseline: int
    n_candidate: int
    violations_baseline: int
    violations_candidate: int
    severity_weight: float


def wilson_upper(p_hat: float, n: int, z: float = 1.96) -> float:
    if n == 0:
        return 1.0
    denom = 1 + z * z / n
    center = (p_hat + z * z / (2 * n)) / denom
    margin = z * math.sqrt(p_hat * (1 - p_hat) / n + z * z / (4 * n * n)) / denom
    return min(1.0, center + margin)


def gate(strata: list[Stratum], p_min: float = 0.004, required_power: float = 0.8) -> str:
    # Minimum expected detections for the target power at the smallest plausible effect.
    min_expected = -math.log(1 - required_power)  # ~1.61 for 0.8
    for s in strata:
        expected = s.n_candidate * p_min
        if expected < min_expected:
            raise UnderpoweredGateError(
                f"stratum {s.name!r}: N={s.n_candidate}, p_min={p_min}, "
                f"expected detections={expected:.2f} < {min_expected:.2f}; "
                f"gate cannot certify safety at this effect size"
            )
        p_base = s.violations_baseline / s.n_baseline
        p_cand = s.violations_candidate / s.n_candidate
        bound = wilson_upper(p_base, s.n_baseline)
        if p_cand > bound:
            return f"FAIL stratum={s.name} p_cand={p_cand:.4f} > upper={bound:.4f}"
    return "PASS"


# Example: the batched-call stratum that the aggregate gate missed.
strata = [
    Stratum("common_path", 200, 200, 38, 26, 1.0),
    Stratum("batched_calls", 6, 6, 1, 2, 5.0),  # rare path, high severity
]

try:
    verdict = gate(strata)
except UnderpoweredGateError as e:
    verdict = f"INSUFFICIENT_POWER: {e}"

print(verdict)
```