{
  "schema_version": "1.3.0",
  "slug": "what-structural-signals-indicate-a-litmus-safety-benchmark-harness-has-silently-desynchronized-from-the-agent-under-test-in-ci",
  "title": "What structural signals indicate a LITMUS safety benchmark harness has silently desynchronized from the agent under test in CI",
  "date_of_record": "2026 November 11",
  "scope": "Applies to continuous red-teaming pipelines where a LITMUS-style safety harness drives an agent through adversarial probe suites inside CI and gates merges on aggregate safety scores. Does not apply to one-shot offline evaluations, human-in-the-loop red teams, or harnesses that re-instantiate the agent from a pinned artifact per probe.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/what-structural-signals-indicate-a-litmus-safety-benchmark-harness-has-silently-desynchronized-from-the-agent-under-test-in-ci.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/what-structural-signals-indicate-a-litmus-safety-benchmark-harness-has-silently-desynchronized-from-the-agent-under-test-in-ci.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/what-structural-signals-indicate-a-litmus-safety-benchmark-harness-has-silently-desynchronized-from-the-agent-under-test-in-ci.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1783,
    "python_slice_tokens": 414,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "The failure that matters is not a failing safety score. A failing score is signal: the harness observed the agent and the agent behaved unsafely. The boundary failure is **harness–agent desynchronization**, where the harness reports a *passing* (or stably unchanged) score while no longer exercising the agent that will actually ship. Desynchronization is sile...",
    "what_was_observed": "Across several CI integrations, the following structural signals preceded a confirmed desynchronization event. None of them is a score change; all of them are properties of the harness run itself. 1. **Probe-to-agent-version binding is absent from the artifact.** The harness emits a report keyed by probe-suite hash but not by the agent build digest. When the agent binary/prompt bundle changes, the report hash is iden...",
    "condition_of_validity": "These signals are valid *only* when the harness is intended to evaluate the live agent build produced by the current CI job. They are **not** valid when: - The pipeline deliberately evaluates a pinned, frozen agent artifact (regression baseline), in which case a stable report hash is correct and expected. - The probe suite is a deterministic conformance suit...",
    "decision_boundary": "Treat a run as **desynchronized and therefore invalid as a safety gate** when *any* of the following hold simultaneously with a passing score: - Report hash unchanged while the agent build digest changed, **and** the harness does not record the agent digest in its report. - Refusal rate on at least one category is identical to the harness fixture baseline to...",
    "operational_effect": "The practical consequence is that safety gates must validate their own coupling to the agent before trusting their verdict. A green safety score is evidence of safety only if the harness can prove, structurally, that it exercised the artifact under test. Pipelines that gate on score alone will ship unsafe agents through a harness that has quietly decoupled —..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "81a2af626cafa1017614ea705d0b8b5d1d45efac1eb8a26f875d0a77180be260",
    "verified_classes": [
      "HarnessDesyncError",
      "HarnessRun"
    ],
    "verified_functions": [
      "assert_coupled",
      "gate"
    ],
    "verified_functions_count": 2,
    "defensive_guards_count": 1,
    "raised_exceptions": [
      "HarnessDesyncError"
    ],
    "sample_fail_closed_invariants": [],
    "stdlib_dependencies": [
      "dataclasses",
      "hashlib"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}