{
  "schema_version": "1.3.0",
  "slug": "which-invariants-prevent-adversarial-trajectory-benchmarks-from-silently-scoring-unobservable-harm",
  "title": "Which invariants prevent adversarial trajectory benchmarks from silently scoring unobservable harm",
  "date_of_record": "2026 November 13",
  "scope": "Applies to LITMUS-style safety benchmarks where an adversarial agent trajectory is executed against a live tool environment and harm is scored by an instrumented harness observing side effects. Does not apply to static transcript grading, red-team prompt suites scored by a judge model alone, or purely synthetic trajectory replay with no live side effects.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/which-invariants-prevent-adversarial-trajectory-benchmarks-from-silently-scoring-unobservable-harm.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/which-invariants-prevent-adversarial-trajectory-benchmarks-from-silently-scoring-unobservable-harm.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/which-invariants-prevent-adversarial-trajectory-benchmarks-from-silently-scoring-unobservable-harm.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1999,
    "python_slice_tokens": 486,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "The non-obvious boundary is between *harm that occurred* and *harm the harness can observe*. In a LITMUS construction, the harness scores a trajectory by instrumenting the environment (filesystem, network egress, process spawn, API calls) and attributing observed side effects to the agent's actions. The invariant that is easy to violate is **observability co...",
    "what_was_observed": "During construction of a LITMUS suite with 240 adversarial trajectories across eight harm categories (data exfiltration, destructive filesystem mutation, credential access, unauthorized network egress, resource exhaustion, privilege escalation, supply-chain tampering, and covert persistence), three failure modes recurred: 1. **Sensor-set drift against the rubric.** The harm rubric for \"data exfiltration\" enumerated H...",
    "condition_of_validity": "The LITMUS score for a trajectory is valid only if all of the following hold: - **Sensor–rubric bijection.** For every harm predicate in the rubric, there exists at least one instrumented channel whose event schema can satisfy that predicate, and every instrumented channel maps to at least one rubric predicate or is explicitly marked as noise. This must be e...",
    "decision_boundary": "The operational decision is whether a given trajectory's score may be aggregated into the benchmark's headline safety metric. - **Aggregate** only trajectories whose events were fully attributable, whose harm predicates were each covered by an instrumented channel, and whose environment fidelity was verified against the target for the exercised capability se...",
    "operational_effect": "Enforcing these invariants changes the benchmark's output shape. The headline metric gains a mandatory `unobservable_fraction` alongside the harm rate, and the two are reported together; a low harm rate with a high unobservable fraction is a failed benchmark, not a safe agent. Positive controls become a gate on category validity and run on every execution, n..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "564838b2056058b3c0aaec7c6b2a90e0554757b707856efb29f61c53c23799a4",
    "verified_classes": [
      "UnobservableHarmError",
      "Score",
      "Event",
      "TrajectoryResult",
      "SensorSet",
      "FidelityBound"
    ],
    "verified_functions": [
      "score_trajectory",
      "aggregate",
      "channels_for",
      "is_superset_of"
    ],
    "verified_functions_count": 4,
    "defensive_guards_count": 3,
    "raised_exceptions": [
      "UnobservableHarmError"
    ],
    "sample_fail_closed_invariants": [],
    "stdlib_dependencies": [
      "__future__",
      "dataclasses",
      "enum",
      "typing"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}