{
  "schema_version": "1.3.0",
  "slug": "under-what-conditions-must-an-instance-of-a-litmus-safety-harness-be-treated-as-a-stateful-authority-rather-than-a-stateless-test-oracle",
  "title": "Under what conditions must an instance of a LITMUS safety harness be treated as a stateful authority rather than a stateless test oracle",
  "date_of_record": "2026 November 16",
  "scope": "Applies to continuous red-teaming pipelines where a LITMUS safety benchmark harness is invoked repeatedly against a persistent agent across a session, and where harness verdicts gate deployment or escalation. Does not apply to one-shot, single-turn benchmark scoring where each probe is independent and the agent under test holds no cross-probe memory.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/under-what-conditions-must-an-instance-of-a-litmus-safety-harness-be-treated-as-a-stateful-authority-rather-than-a-stateless-test-oracle.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/under-what-conditions-must-an-instance-of-a-litmus-safety-harness-be-treated-as-a-stateful-authority-rather-than-a-stateless-test-oracle.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/under-what-conditions-must-an-instance-of-a-litmus-safety-harness-be-treated-as-a-stateful-authority-rather-than-a-stateless-test-oracle.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1255,
    "python_slice_tokens": 256,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "A LITMUS harness is commonly integrated as a *stateless oracle*: feed it a probe, receive a `PASS`/`FAIL`/`REFUSE` verdict, discard the harness instance. This is correct only when the agent under test is itself stateless across probes. Under continuous red-teaming, the agent accumulates conversational state, tool-call caches, and in-context policy inferences...",
    "what_was_observed": "In a deployment scoring a 40-probe escalating ladder, the harness emitted `PASS` for probe 33 (a prompt-injection escalation) when replayed from a fresh instance against the *same* agent process that had already been red-teamed through probes 1–32. The fresh harness instance had no record that the agent had been hardened in-context, so it scheduled probe 33 at a severity the agent had already learned to refuse. The o...",
    "condition_of_validity": "The harness may be treated as a stateless oracle **if and only if** all three hold: 1. The agent under test is reset to a byte-identical initial state between probes (no transcript, no cache, no learned policy). 2. The harness's internal scheduler state is reset in lockstep with the agent reset. 3. The probe set is fixed and non-adaptive (no severity escalat...",
    "decision_boundary": "The decision to escalate a finding to a deployment gate depends on which regime the verdict was produced under: - **Stateless-oracle verdict:** reproducible, order-independent, safe to aggregate across runs. May gate deployment on a per-probe basis. - **Stateful-authority verdict:** valid only for the producing session. Must be recorded with a session token ...",
    "operational_effect": "Pipelines that upcast stateful verdicts to stateless ones exhibit a characteristic pathology: intermittent `PASS` on re-run of known `FAIL` probes, correlated with how far into the escalation ladder the probe sits. The operational fix is to pin the harness instance to the session (never re-instantiate mid-session), hash the probe order into the verdict recor..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "3a1b65734c3d2431f03c8c6a50c4e3cd9646c433afd94a12fd67e00fe3ecfe6f",
    "verified_classes": [
      "HarnessStateBoundaryError",
      "Verdict"
    ],
    "verified_functions": [
      "probe_order_hash",
      "assert_verdict_valid_for_session",
      "gate_deployment"
    ],
    "verified_functions_count": 3,
    "defensive_guards_count": 2,
    "raised_exceptions": [
      "HarnessStateBoundaryError"
    ],
    "sample_fail_closed_invariants": [],
    "stdlib_dependencies": [
      "dataclasses",
      "hashlib"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}
