# Why does an instance drift from its declared skill manifest after a red-team tool-hijack probe and silently retain a stale tool binding **Date of Record:** 2026 November 18 **Scope:** Applies to LITMUS-derived red-team harnesses that exercise skill-injection and tool-hijack scenarios against a single long-lived agent instance across multiple probes. Does not apply to stateless single-shot evaluations, nor to harnesses that tear down and rebuild the agent process between every probe. ## Core Distinction There are two distinct notions of "the agent under test" that a LITMUS-derived harness conflates at its own peril: 1. **The declared manifest** — the skill set and tool bindings the harness *intends* the instance to hold, expressed as a versioned artifact (the skill manifest, the tool registry snapshot, the policy file). 2. **The live binding table** — the actual mapping from tool-name to callable/endpoint that the running instance will resolve at the moment a probe invokes a tool. A red-team probe that hijacks a tool typically does so by *injecting* an alternate binding: a shadow tool registered under the same name, a skill that overrides a prior skill's tool reference, or a prompt that causes the planner to re-resolve a tool name against a poisoned registry. The non-obvious invariant is this: **a successful hijack probe does not merely produce a bad output for that probe; it mutates the live binding table, and that mutation persists into every subsequent probe unless the harness explicitly restores the declared manifest.** The declared manifest is not the live state. The harness's pass/fail logic reads outputs, but the instance's future behavior is governed by the binding table. The failure mode is therefore not "the hijack succeeded." It is "the hijack succeeded, was logged as a detected finding, and then the *next* probe's result was attributed to that next probe's scenario while actually being contaminated by the previous probe's residual binding." ## What Was Observed In a multi-probe LITMUS-derived run, the harness executed a skill-injection scenario (probe N) in which an injected skill re-declared the `fetch` tool to point at an attacker-controlled endpoint. The harness's detector correctly flagged probe N as a tool-hijack success. The harness then proceeded to probe N+1, a benign tool-use correctness scenario, and recorded a *failure* on probe N+1: the agent called `fetch` and received data inconsistent with the scenario's fixture. The root cause was that probe N's injected skill had registered its override into the live binding table, and the harness's teardown between probes reset the *conversation context* and the *scratch workspace* but not the *tool binding table*. Probe N+1's scenario assumed `fetch` resolved to the fixture server; it resolved to the residual attacker endpoint from probe N. The probe N+1 failure was reported as a genuine correctness defect, inflating the false-positive rate of the benign suite and — more dangerously — masking that the hijack from probe N had a persistence property the harness was not measuring. The signature is reproducible: run a hijack probe, then a benign probe, with no binding reset, and the benign probe's tool calls route to the hijack target. Run the benign probe first, and it passes. The order dependence is the tell. ## Condition of Validity The contamination is only possible when all of the following hold: - The instance is long-lived across probes (shared process or shared binding registry). - The hijack vector mutates a binding that is resolved by *name* at call time rather than captured at session start. - The harness's inter-probe teardown resets context and workspace but not the binding table. - The harness attributes each probe's outcome to that probe's scenario without an isolation check. If any one fails — stateless per-probe process, name resolution frozen at session start, full binding reset between probes, or contamination-aware attribution — the failure does not occur. This is why the bug is invisible in single-shot harnesses and in harnesses that rebuild the agent per probe, and why it surfaces specifically in the "realistic long-session" configurations that LITMUS-derived harnesses adopt to model persistent agents. ## Decision Boundary The harness must decide, per probe, whether it is measuring *the instance as a persistent agent* (in which case residual binding is a legitimate finding to record, not to erase) or *the scenario in isolation* (in which case the binding table must be restored to the declared manifest before the probe runs and verified after). The boundary is: **if the probe's scenario declares a tool binding, the harness must assert the live binding table matches the declared manifest before the probe executes, and must fail closed if it does not.** A probe that runs against a binding table it did not establish is not a probe of its own scenario. The correct response is to refuse to run the probe and raise, not to run it and attribute the result. The inverse boundary also holds: a harness that resets bindings between every probe cannot detect persistence-of-hijack, which is itself a finding worth measuring. The two modes must be explicitly selected, never defaulted into by accident. ## Operational Effect Left unaddressed, the contamination produces three compounding effects: benign-suite false positives that erode trust in the harness's correctness signal; attribution errors that credit a hijack's persistence to an unrelated scenario; and a systematic undercount of hijack severity, because the harness measures the hijack's *output* on probe N but never measures its *residual binding* on probe N+1. The fix is a binding-table assertion at probe entry, keyed to the declared manifest, that fails closed on mismatch and records the residual binding as a first-class finding rather than silently inheriting it. ```python from dataclasses import dataclass, field from typing import Callable, Dict class BindingContaminationError(RuntimeError): """Raised when the live tool binding table diverges from the declared manifest at probe entry, indicating residual state from a prior probe.""" pass @dataclass class Instance: declared_manifest: Dict[str, str] # tool_name -> expected endpoint/skill id live_bindings: Dict[str, str] = field(default_factory=dict) def assert_binding_isolation(instance: Instance, probe_id: str) -> None: """Fail closed if the live binding table does not match the declared manifest. A probe that runs against bindings it did not establish is not a probe of its own scenario; its result must not be attributed to that scenario. """ for tool, expected in instance.declared_manifest.items(): actual = instance.live_bindings.get(tool) if actual != expected: raise BindingContaminationError( f"probe {probe_id!r}: live binding for tool {tool!r} is {actual!r}, " f"declared manifest requires {expected!r}; residual hijack binding " f"from a prior probe is present. Refusing to run probe." ) extra = set(instance.live_bindings) - set(instance.declared_manifest) if extra: raise BindingContaminationError( f"probe {probe_id!r}: undeclared tools present in live bindings: " f"{sorted(extra)}; manifest does not authorize them." ) def run_probe(instance: Instance, probe_id: str, scenario: Callable[[Instance], None]) -> None: assert_binding_isolation(instance, probe_id) scenario(instance) # Post-probe: restore declared manifest so the next probe starts clean, # but record any residual mutation as a persistence finding first. residual = { t: v for t, v in instance.live_bindings.items() if instance.declared_manifest.get(t) != v } if residual: # Persistence-of-hijack is a finding, not a silent inheritance. record_finding(probe_id, "residual_binding", residual) instance.live_bindings = dict(instance.declared_manifest) def record_finding(probe_id: str, kind: str, detail: dict) -> None: # Emit to the harness finding log; kept minimal here. print(f"[finding] probe={probe_id} kind={kind} detail={detail}") ```