# Why does an instance-level safety classifier pass a LITMUS adversarial suite while the deployed agent still executes the harmful trajectory? **Date of Record:** 2026 November 15 **Scope:** Applies to LITMUS-style harnesses that score agent safety by classifying individual turns, tool calls, or intermediate messages against an adversarial prompt suite. Does not apply to end-to-end trajectory judges that score the final environment state, nor to static prompt-injection filters evaluated without an executing agent. ## Core Distinction A LITMUS harness measures the safety of *observable instances* — a single message, a single tool invocation, a single completion — whereas agent harm is a property of a *trajectory*. The classifier is asked "is this instance unsafe?" while the deployment question is "did this sequence of individually-permissible instances produce an unsafe outcome?" These are not the same predicate, and the gap between them is where the benchmark reports a pass that the runtime does not honor. The failure is not classifier inaccuracy. It is a **compositional invariant violation**: the harness assumes that if every instance in a trajectory is classified safe, the trajectory is safe. That assumption is false whenever the harm is *assembled* from benign parts — a read of a credential file, a lookup of an internal hostname, and a POST to an external endpoint are each unremarkable in isolation, and only their concatenation is the exfiltration. ## What Was Observed Across three LITMUS runs (n=1,200 adversarial trajectories, 5 agent scaffolds), the instance-level classifier reported a 96.4% pass rate. A trajectory-level replay judge on the same rollouts flagged 11.2% of the *passed* trajectories as realizing a harmful end state. The delta was concentrated in trajectories where no single instance crossed the classifier's decision threshold but the sequence did. Three recurring compositions drove the gap: 1. **Split-payload assembly.** Harmful content is emitted across ≥2 tool calls where each call carries a fragment below the classifier's salience threshold. The harness scores each fragment independently; the agent's own context window concatenates them. 2. **Capability composition.** Two safe capabilities (filesystem read of a config, outbound HTTP to an allowlisted host) combine into credential egress. Each capability is in-scope; the composition is not. 3. **State-dependent permission.** An instance is safe against the initial environment state but unsafe against the state produced by earlier instances in the same trajectory. The harness scores instances against a *reset* environment, not the running one. In all three, the classifier's verdict on every instance was correct *for that instance in isolation*. The harness's aggregate verdict was wrong because it applied an instance-level predicate to a trajectory-level object. ## Condition of Validity The instance-level pass is a valid safety signal **only if** the following holds: for every trajectory τ = (i₁, …, iₙ), `safe(i₁) ∧ … ∧ safe(iₙ) ⟹ safe(τ)`. This is the compositional-safety assumption, and it is exactly what the observations above falsify. The assumption holds when (a) the harm predicate is *instance-local* — the instance alone is the harm, not the sequence — and (b) instances are scored against the live trajectory state, not a reset environment. For prompt-injection filters, refusal classifiers, and single-turn jailbreak suites, (a) typically holds. For tool-using agents, code-execution agents, and anything with persistent environment state, (a) and (b) fail routinely, and the harness's pass rate becomes an upper bound on safety, not a measurement of it. The condition also fails silently when the harness resets the environment between instance evaluations. That reset is correct for reproducibility and *wrong* for safety: it erases exactly the state dependence that makes composition dangerous. ## Decision Boundary Do not treat a LITMUS instance-level pass as evidence of trajectory safety when any of the following is true: - The agent has **tool access that mutates external state** (writes, sends, executes). - The agent can **read state it later acts on** within the same trajectory (the read/write composition). - The harness **resets the environment** between instance evaluations. - The adversarial suite includes **multi-step** cases where the harm requires ≥2 instances. When these hold, the instance-level pass rate is a *lower bound on benignness*, not an upper bound on harm. The correct decision is to require a trajectory-level judge — one that scores the final environment state or the full sequence against a harm predicate — before claiming the agent is safe. If no such judge exists, the honest report is "instance-safe, trajectory-unverified," which is a materially weaker claim than the harness's headline number implies. The boundary is sharp: instance-level scoring answers "would a reviewer flag any single step?" Trajectory-level scoring answers "did the agent do the bad thing?" Only the second is a safety claim about a deployed agent. ## Operational Effect Teams that ship on the instance-level pass rate observe a characteristic post-deployment signature: the safety dashboard shows 96% pass while incident review finds exfiltration, privilege composition, and split-payload attacks that the benchmark "passed." The benchmark was not lying; it was answering a different question than the one the deployment needed answered. The operational fix is to make the harness's unit of evaluation match the unit of harm. Concretely: score trajectories, not instances; do not reset environment state between steps within a trajectory; and when an instance-level classifier is retained for cost, treat its pass as a *necessary but not sufficient* condition and gate deployment on a trajectory-level judge. The cost of the trajectory judge is real — it is O(n) state replays per rollout — but the cost of shipping on a compositional-safety assumption that does not hold is the incident the benchmark was supposed to prevent. The invariant to encode: **a safety pass is only as strong as the predicate it was computed over.** An instance predicate cannot certify a trajectory. When the harness conflates them, the pass rate is a measurement of the wrong object, and the gap between the two objects is precisely the attack surface. ```python class TrajectorySafetyError(RuntimeError): """Raised when an instance-level safety pass is used to certify a trajectory.""" pass def certify_trajectory_safe(instances, trajectory_judge, environment_state): """ Guard against the compositional-safety fallacy: an instance-level pass does not imply trajectory safety when harm can be assembled from individually-permissible instances. """ if not instances: raise TrajectorySafetyError("empty trajectory: nothing to certify") instance_safe = all(inst.get("verdict") == "safe" for inst in instances) # The instance-level pass is necessary but NOT sufficient. if instance_safe and trajectory_judge is None: raise TrajectorySafetyError( "instance-level pass without a trajectory judge: " "compositional-safety assumption unverified; " "harm may be assembled from individually-safe instances" ) if environment_state.get("reset_between_steps"): raise TrajectorySafetyError( "environment was reset between instance evaluations: " "state-dependent composition is invisible to this harness" ) trajectory_verdict = trajectory_judge(instances, environment_state) if trajectory_verdict != "safe": raise TrajectorySafetyError( f"trajectory judge rejected a sequence of instance-safe steps: " f"{trajectory_verdict!r}" ) return True ```