{
  "schema_version": "1.3.0",
  "slug": "when-must-an-instance-be-excluded-from-litmus-scoring-the-contaminated-baseline-invariant-in-adversarial-rubric-gating",
  "title": "When must an instance be excluded from LITMUS scoring: the contaminated-baseline invariant in adversarial rubric gating",
  "date_of_record": "2026 November 7",
  "scope": "Applies to construction of LITMUS-style safety benchmark suites and the regression gates that consume their scores for agent deployment decisions. Does not apply to exploratory red-teaming where no pass/fail gate is enforced, nor to single-turn classification benchmarks with no multi-turn state.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/when-must-an-instance-be-excluded-from-litmus-scoring-the-contaminated-baseline-invariant-in-adversarial-rubric-gating.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/when-must-an-instance-be-excluded-from-litmus-scoring-the-contaminated-baseline-invariant-in-adversarial-rubric-gating.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/when-must-an-instance-be-excluded-from-litmus-scoring-the-contaminated-baseline-invariant-in-adversarial-rubric-gating.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1213,
    "python_slice_tokens": 170,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "A LITMUS instance is not merely a prompt-response pair with an attached rubric; it is a *measurement instrument* whose validity depends on the agent's inability to have observed the exact instance during training or fine-tuning. The non-obvious boundary is between **an instance that an agent fails** and **an instance that is invalid for that agent**. These a...",
    "what_was_observed": "During construction of a multi-turn agentic safety suite (tool-use refusal, jailbreak persistence, and indirect-injection resistance), a subset of instances scored as clean passes across every candidate model, including a deliberately weakened canary model that was known to comply with 80% of the suite. Investigation showed the passing instances were near-verbatim reproductions of public jailbreak corpora (the canary...",
    "condition_of_validity": "The invariant is: **an instance may contribute to a LITMUS score only if its expected-behavior rubric is unreachable by recall, leakage, or trivial pattern-matching, and this must be established before the instance enters the aggregation set.** Operationally this requires a pre-scoring exclusion pass with three checks, all of which must pass for the instance...",
    "decision_boundary": "The decision boundary is not \"did the model pass?\" but \"is this instance allowed to vote?\" The gate must be structured so that inadmissible instances never enter the numerator or denominator, and so that the *rate of inadmissibility* is itself a monitored metric. If more than a small fraction (empirically ~5%) of a suite is inadmissible, the suite is stale a...",
    "operational_effect": "Enforcing exclusion-before-aggregation changed the deployment decision in two observed cases: a canary that previously \"passed\" the suite was correctly flagged (its passes were all on inadmissible instances), and a candidate model's apparent 96% pass rate dropped to 78% once memorized instances were removed, crossing the gate threshold. The cost is a mandato..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "c0e85655dfa4de66f089ccb520fac37ce694c2aca55984fd1cfdd49310225587",
    "verified_classes": [
      "InadmissibleInstanceError",
      "Instance"
    ],
    "verified_functions": [
      "check_admissible",
      "build_scoring_set"
    ],
    "verified_functions_count": 2,
    "defensive_guards_count": 4,
    "raised_exceptions": [
      "InadmissibleInstanceError"
    ],
    "sample_fail_closed_invariants": [],
    "stdlib_dependencies": [
      "dataclasses"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}