{
  "schema_version": "1.3.0",
  "slug": "what-structural-signals-indicate-a-genai-evaluation-span-has-silently-degraded-from-a-safety-judge-measurement-into-a-red-team-harness-side-effect",
  "title": "What structural signals indicate a GenAI evaluation span has silently degraded from a safety-judge measurement into a red-team-harness side effect",
  "date_of_record": "2026 November 14",
  "scope": "Applies to OpenTelemetry-style evaluation spans emitted by LLM safety judges and red-team harnesses that share a semantic convention for attributes such as `gen_ai.evaluation.name`, `gen_ai.evaluation.score`, `gen_ai.evaluation.explanation`, and `gen_ai.operation.name`. Does not apply to pure inference spans, to retrieval spans, or to harnesses that emit no evaluation span at all (those fail loudly elsewhere).",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/what-structural-signals-indicate-a-genai-evaluation-span-has-silently-degraded-from-a-safety-judge-measurement-into-a-red-team-harness-side-effect.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/what-structural-signals-indicate-a-genai-evaluation-span-has-silently-degraded-from-a-safety-judge-measurement-into-a-red-team-harness-side-effect.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/what-structural-signals-indicate-a-genai-evaluation-span-has-silently-degraded-from-a-safety-judge-measurement-into-a-red-team-harness-side-effect.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1170,
    "python_slice_tokens": 221,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "A safety-judge span and a red-team-harness span are structurally identical when serialized: both carry an evaluation name, a numeric score, a verdict string, and a free-text explanation. The invariant that separates them is *who produced the label and against what ground truth*. A judge span asserts \"a model assigned this label to a pre-existing artifact.\" A...",
    "what_was_observed": "In a combined judge-plus-red-team pipeline, weekly \"harmlessness score\" dashboards trended upward by roughly 14 percentage points over three releases while human review of the same traffic showed no change in refusal quality. Inspection of raw spans showed the cause: the red-team harness had begun emitting evaluation spans with `gen_ai.evaluation.name = \"safety.harmlessness\"` and `gen_ai.evaluation.score` set to the ...",
    "condition_of_validity": "The following conditions must all hold for a blended judge/harness evaluation stream to remain a valid safety measurement: 1. Evaluation-name namespaces are disjoint between judges and harnesses, or a provenance attribute (`eval.source = judge | harness`) is present and used as a partition key. 2. Score polarity is fixed per evaluation name and documented; h...",
    "decision_boundary": "- **Treat as measurement (aggregate freely):** disjoint namespaces, explicit provenance, referential explanations, polarity documented and asserted. - **Treat as diagnostic only (never aggregate into a safety KPI):** shared namespace but provenance present; harness spans with templated explanations; any span where `gen_ai.evaluation.score` polarity is undocu...",
    "operational_effect": "Enforce the invariant at the collector: reject or quarantine spans that violate the provenance/polarity contract before they reach the metric pipeline, and emit a counter so the degradation is visible as a rate rather than as a slow drift in a dashboard. The cost of not doing this is that the safety metric becomes a function of how hard the red team works, w..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "f6ceb5c98cc7856ba77469b1b9e32e542a038065136b26b96a994f5ee0b5bd6b",
    "verified_classes": [
      "EvaluationSpanContractError"
    ],
    "verified_functions": [
      "validate_evaluation_span"
    ],
    "verified_functions_count": 1,
    "defensive_guards_count": 6,
    "raised_exceptions": [
      "EvaluationSpanContractError"
    ],
    "sample_fail_closed_invariants": [
      "missing gen_ai.evaluation.name"
    ],
    "stdlib_dependencies": [],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}