# Where must an instance boundary be drawn in LITMUS-style multi-turn injection benchmarks **Date of Record:** 2026 November 14 **Scope:** Applies to harnesses that measure agent susceptibility to indirect and multi-turn prompt injection by threading adversarial content across a sequence of turns or tool observations within a single benchmark instance. Does not apply to single-turn static-prompt injection suites, nor to red-team evaluations where each adversarial payload is scored in isolation. ## Core Distinction The non-obvious invariant is this: **a "turn" is not a "session," and a "session" is not an "instance."** Most LITMUS-derived harnesses conflate the three because the common implementation reuses one conversation object across the payload sequence. The correct operational distinction is that an *instance* is the atomic unit of scoring — the smallest scope within which contamination is allowed to persist and outside of which it must be provably absent. A multi-turn injection payload is *supposed* to leak across turns within an instance; that leakage is the thing being measured. The failure mode is leakage across *instances* — residual state (KV cache, retrieved context, scratchpad memory, tool-side caches, or agent-written notes) that carries a payload fragment from instance N into instance N+1, inflating or deflating the measured attack success rate in ways that are invisible in per-instance pass/fail output. ## What Was Observed In a 3,000-instance multi-turn suite (payloads staged across 4–7 turns with a mid-sequence tool-observation injection), the harness reported an aggregate attack success rate of 41.2%. Re-running the identical suite in a fresh process per instance produced 34.7%. The delta — 6.5 points — was traced to two independent cross-instance channels: 1. **Retrieval cache persistence.** The tool-observation injector wrote a poisoned document into a shared retrieval index keyed by content hash. Instances later in the run whose benign queries hashed near an earlier payload retrieved the poisoned chunk, effectively delivering a turn-0 injection that the instance's own payload schedule never contained. 2. **Agent scratchpad carryover.** A subset of agents wrote intermediate reasoning to a persistent workspace file. When an instance aborted mid-sequence (guardrail trip), the workspace was not cleared, and the next instance's agent read a partial adversarial instruction as if it were its own prior reasoning. Both channels produced *false positives* — successful injections attributed to multi-turn payloads that were actually single-shot carryover. A third channel produced false negatives: a per-instance rate limiter tripped on the third instance of a batch, causing the agent to short-circuit before reaching the injected turn, which the scorer recorded as "resisted." ## Condition of Validity The measured attack success rate is valid **only if** all mutable state touched by instance N is either (a) destroyed before instance N+1 begins, or (b) provably partitioned by an instance-scoped key on every read path — not merely on the write path. The condition fails silently whenever a read path omits the instance key, which is the common case for retrieval indexes, embedding caches, and file-backed scratchpads. Note that process-level isolation is *necessary but not sufficient*: a fresh process with a shared on-disk vector store still leaks. The invariant must be enforced at the storage layer, verified by a canary instance that plants a unique marker and asserts the marker is unreachable from the next instance. ## Decision Boundary Choose **per-instance hard isolation** (fresh process + fresh storage namespace + canary assertion) when the metric being reported is *susceptibility to multi-turn structure specifically* — i.e., when the claim is "the agent failed because the payload was threaded," not merely "the agent failed." Choose **shared-namespace batching** only when the reported metric is explicitly an aggregate "any-injection success rate" and cross-instance contamination is documented as part of the measured surface. The boundary is crossed the moment a result is cited to argue about *multi-turn* robustness; at that point, any cross-instance channel invalidates the causal claim, because the observed success may be attributable to a single-turn carryover the multi-turn schedule did not author. ## Operational Effect Enforcing the boundary raises per-instance cost (process spawn, cold cache) by roughly the ratio of cache-warm to cache-cold latency — in the observed run, a 2.3× wall-clock increase. This is the price of a defensible causal claim. The practical consequence is that multi-turn LITMUS harnesses should report two numbers: the isolated attack success rate (the headline) and the shared-namespace rate (diagnostic, labeled as contaminated). Divergence between them is itself a signal — a large gap indicates the agent's robustness depends on state the evaluator did not intend to grant it. ```python class InstanceBoundaryViolation(RuntimeError): """Raised when state from one benchmark instance is reachable from another.""" pass def assert_instance_isolation(prev_marker: str, current_namespace_reads: set[str]) -> None: """Canary check: a marker planted in instance N must be unreachable from N+1.""" if prev_marker in current_namespace_reads: raise InstanceBoundaryViolation( f"Cross-instance leak: marker {prev_marker!r} reachable from next instance; " "multi-turn attack success rate is not causally attributable to payload structure." ) return None def run_instance(instance_id: int, store, marker: str) -> dict: reads = store.read_paths() if instance_id > 0: assert_instance_isolation(prev_marker=marker, current_namespace_reads=reads) return {"instance": instance_id, "reads": sorted(reads)} ```