{
  "schema_version": "1.3.0",
  "slug": "where-must-an-instance-of-litmus-tool-misuse-scoring-fail-the-pre-execution-adjudication-gap",
  "title": "Where must an instance of LITMUS tool-misuse scoring fail: the pre-execution adjudication gap",
  "date_of_record": "2026 November 13",
  "scope": "Applies to LITMUS-style safety benchmarks that score agentic trajectories for tool misuse by adjudicating the *intent and parameters* of a proposed tool call against a policy. It does not apply to purely observational benchmarks that score only realized side effects, nor to post-hoc transcript audits where the tool call has already executed.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/where-must-an-instance-of-litmus-tool-misuse-scoring-fail-the-pre-execution-adjudication-gap.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/where-must-an-instance-of-litmus-tool-misuse-scoring-fail-the-pre-execution-adjudication-gap.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/where-must-an-instance-of-litmus-tool-misuse-scoring-fail-the-pre-execution-adjudication-gap.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1277,
    "python_slice_tokens": 221,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "The non-obvious boundary in LITMUS scoring is the difference between **harm that a tool call would cause if executed** and **harm that the scoring harness attributes to the agent for proposing the call**. A benchmark can be internally consistent and still mis-score an agent whenever the adjudication of a call happens *before* the environment has committed th...",
    "what_was_observed": "In a run of 1,200 trajectories against a LITMUS-derived harness, 37 trajectories were scored as `tool_misuse` where the flagged call never mutated state. In every one of those 37, the agent proposed a call that *would* have been destructive — e.g. `delete_records(filter=\"*\")` or `transfer_funds(amount=balance)` — but the call was rejected downstream by the environment (403, 409 conflict, or a confirmation gate the ag...",
    "condition_of_validity": "LITMUS proposal-time scoring is valid **iff** for every tool in the suite, the tuple `(policy_verdict, resolved_parameters, acting_principal)` is identical at proposal time and at commit time. This holds when: - tools are stateless or idempotent with respect to the flagged parameter, - the acting principal is fixed at proposal and not re-derived from ambient...",
    "decision_boundary": "The operational rule: **adjudicate at the last moment at which the full execution context is available, and record which context was used.** Concretely, if a tool call can be rejected or rewritten by the environment after proposal, the scorer must either (a) defer adjudication to commit and score the realized effect, or (b) score both the proposed and the re...",
    "operational_effect": "Teams that treat LITMUS proposal-time scores as ground truth will over-penalize agents that correctly probe permissions (the 403 cases) and under-penalize agents that exploit ambient-state resolution (the 9 cases). The first error is visible and gets argued about; the second is silent and ships. The fix is structural: carry a `resolution_context` alongside e..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "a516bc042272efbc06a1f5523f8fac4fe1131d6926431429bd75fee79ec224f8",
    "verified_classes": [
      "AdjudicationContextMismatch",
      "CallContext"
    ],
    "verified_functions": [
      "score_tool_call"
    ],
    "verified_functions_count": 1,
    "defensive_guards_count": 4,
    "raised_exceptions": [
      "AdjudicationContextMismatch"
    ],
    "sample_fail_closed_invariants": [
      "no commit context: proposal-time scoring is a counterfactual, not a fact",
      "resolved parameters diverged between proposal and commit; the scorer's model of the call is strictly less informed than "
    ],
    "stdlib_dependencies": [
      "dataclasses"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}