# In what scenarios must an instance treat span-level safety scores as non-aggregable across an agent trace boundary

**Date of Record:** 2026 November 7

**Scope:** Applies to GenAI observability pipelines that compute per-span safety scores and then roll them up to trace-, session-, or agent-level evaluation metrics. Does not apply to pipelines that emit only per-span scores with no aggregation, nor to batch offline eval jobs where spans are replayed in isolation without parent linkage.

## Core Distinction

The failure is not "a low safety score was missed." It is the distinction between **span-local safety semantics** and **trace-global safety semantics**. A span-level classifier observes exactly one unit of model output (a completion, a tool-call argument block, a retrieved chunk echoed back). Its score is a probability that *this unit*, in isolation, violates a policy. That probability is well-calibrated only against the span's own input distribution.

Aggregating these scores — mean, max, p95, or a weighted average — implicitly asserts that safety is a monotone, decomposable property of the trace. It is not. Safety of an agent trace is a property of the *composition*: a benign retrieval span, a benign planning span, and a benign tool-call span can compose into an exfiltration path that no individual span scores highly. Conversely, a single high-severity span (a refusal bypass) can be diluted to below threshold by a long tail of benign utility spans in a mean aggregation. The invariant that actually holds is: **trace safety is bounded below by the max span score only when the spans are causally independent and the policy is a per-unit predicate.** Agent spans are neither.

## What Was Observed

In a pipeline scoring a multi-step tool-using agent, a trace containing 41 spans produced a mean safety score of 0.07 (well under the 0.30 incident threshold) and a max of 0.91. The 0.91 span was a single tool-call argument that concatenated a user-controlled string into a shell invocation. The 40 remaining spans were retrieval and summarization steps with scores in the 0.02–0.09 range. The mean-based trace scorer classified the trace as clean; the max-based scorer flagged it. Both were "correct" relative to their own definitions, which is precisely the problem: the aggregation choice, made once at pipeline configuration time, silently defined the safety semantics.

Second observation: drift detection operated on the distribution of per-span scores per model version. When the agent's planner was upgraded, the *number* of spans per trace increased by roughly 3x (more ReAct iterations), which shifted the per-span score distribution toward the benign mode purely by adding more low-risk spans. The drift detector fired a false "safety improvement" alert while the trace-level incident rate was unchanged. The drift signal was measuring span count, not model behavior — a Simpson's-paradox-style composition effect.

Third observation, forensics: reconstructing an incident required joining spans by `trace_id`, but two sub-agents in the same trace emitted spans with the same `span_id` namespace (the sub-agent runtime reused a local counter). The forensic join silently cross-linked spans from different sub-agents, producing a plausible but wrong causal chain. The safety score of the "incident span" was attached to the wrong parent, and the root-cause attribution pointed at the retrieval tool rather than the planner.

## Condition of Validity

Span-level aggregation to a trace-level safety verdict is valid only when all of the following hold:

1. **Policy decomposability.** The safety policy is expressible as a per-unit predicate (e.g., "this completion must not contain PII"), not as a relational predicate over the trace (e.g., "the agent must not combine retrieved PII with an outbound tool call").
2. **Causal independence of span scores.** No span's score is conditioned on a sibling span's content in a way that changes its interpretation. Tool-call argument spans violate this routinely: the argument is only dangerous *given* the retrieved context.
3. **Stable span cardinality per trace.** The per-trace span count distribution is stationary across the comparison window, so that mean-based rollups are not dominated by count drift.
4. **Globally unique span identity.** `span_id` is unique within at least the trace scope, and parent linkage is unambiguous across sub-agent runtimes.

When any condition fails — and in agent traces, conditions 1 and 2 fail almost always — the pipeline must not emit a single scalar trace safety score without also emitting the aggregation method and the span-count denominator, and must not allow that scalar to drive automated incident thresholds.

## Decision Boundary

The operational decision is: **does the pipeline aggregate, or does it surface the reduction as a structured object?**

- If the policy is per-unit and span cardinality is stationary → aggregate to a scalar; a mean is acceptable; alert on the scalar.
- If the policy is relational (compositional exfiltration, multi-step jailbreak, tool-chaining) → do **not** aggregate. Emit `{max, argmax_span_id, span_count, policy_class}` and require the incident classifier to consume the argmax span plus its causal neighborhood. The scalar mean must be suppressed from the alerting path entirely.
- If span cardinality is non-stationary across model versions → drift detection must operate on **per-span score conditioned on span role** (planner / retrieval / tool-call), not on the pooled per-span distribution, or it will report composition drift as behavior drift.
- If span identity is not globally unique → forensics joins must be rejected, not best-effort repaired. A wrong causal chain is worse than an admitted gap.

## Operational Effect

Pipelines that aggregate span safety with a mean and alert on the mean exhibit a characteristic failure profile: **long benign traces mask single high-severity spans**, and the miss rate scales with agent verbosity. Pipelines that alert on max without role conditioning exhibit the inverse: **retrieval-heavy traces generate false positives** because a single echoed chunk can score high in isolation. The stable configuration is role-conditioned scoring with max-plus-argmax surfacing and a separate composition-policy detector that consumes the causal neighborhood of the argmax span. Forensics must treat missing or duplicated span identity as a hard pipeline error, because the incident record is the artifact of record and a silently mis-joined trace cannot be distinguished from a correct one after the fact.

```python
"""Guard: refuse to emit an aggregable trace-level safety scalar when the
trace's safety policy is relational or span identity is not unique."""

from dataclasses import dataclass
from typing import Sequence


class NonAggregableTraceError(RuntimeError):
    """Raised when a trace cannot be reduced to a single safety scalar."""


@dataclass(frozen=True)
class Span:
    span_id: str
    role: str          # "planner" | "retrieval" | "tool_call" | "completion"
    safety_score: float


@dataclass(frozen=True)
class TraceVerdict:
    max_score: float
    argmax_span_id: str
    span_count: int
    policy_class: str


RELATIONAL_POLICIES = {"compositional_exfiltration", "multi_step_jailbreak", "tool_chaining"}


def reduce_trace(
    spans: Sequence[Span],
    policy_class: str,
    *,
    allow_scalar: bool,
) -> TraceVerdict:
    if not spans:
        raise NonAggregableTraceError("empty trace: no spans to score")

    ids = [s.span_id for s in spans]
    if len(ids) != len(set(ids)):
        # Forensics join would cross-link sub-agent spans. Refuse.
        raise NonAggregableTraceError(
            f"duplicate span_id within trace scope: {sorted(ids)}"
        )

    if policy_class in RELATIONAL_POLICIES and allow_scalar:
        raise NonAggregableTraceError(
            f"policy {policy_class!r} is relational; a scalar mean would "
            "dilute the argmax span and must not be emitted"
        )

    for s in spans:
        if not (0.0 <= s.safety_score <= 1.0):
            raise NonAggregableTraceError(
                f"span {s.span_id} score out of range: {s.safety_score}"
            )

    top = max(spans, key=lambda s: s.safety_score)
    return TraceVerdict(
        max_score=top.safety_score,
        argmax_span_id=top.span_id,
        span_count=len(spans),
        policy_class=policy_class,
    )
```