# Which invariants prevent trace-level safety scoring from silently diverging when anomaly attribution runs on partial spans **Date of Record:** 2026 November 16 **Scope:** Applies to agent observability pipelines where safety SLOs are computed per-trace from span attributes and anomaly attribution is allowed to execute before trace closure (streaming or timeout-flushed spans). Does not apply to batch-only scoring systems that always operate on complete, sealed traces. ## Core Distinction A trace-level safety SLO is a statement about a *completed* unit of agent behavior: the full set of spans emitted by one agent invocation, including tool calls, model turns, and any retries. Anomaly attribution, by contrast, is a *streaming* function: it must produce a label as spans arrive, because the whole point of attribution is to alert before the trace finishes (or to alert at all, when a trace never finishes). The non-obvious boundary is this: **attribution over partial spans is a different function than attribution over complete spans, and the two are not related by any monotonicity guarantee.** A trace that looks safe at span 4 can be unsafe at span 9, and — critically — a trace that looks unsafe at span 4 can *also* be safe at span 9, because later spans can retroactively explain earlier ones (a retried tool call that succeeded, a refusal that followed a suspicious prompt, a guardrail that fired after a bad model turn). Attribution that fires on the partial view and is never retracted produces a *false positive that the completed trace would have exonerated*. The invariant that must be enforced is not "attribution is correct" — it cannot be, on partial input. The invariant is: **every attribution label carries the closure state of the trace it was computed over, and SLO aggregation refuses to mix labels computed at different closure states without an explicit reconciliation step.** ## What Was Observed In a production agent fleet (tool-calling assistant, ~40k traces/hour, mean 7.2 spans/trace, p99 41 spans), the safety SLO was defined as "fraction of traces with no high-severity attribution label, 30-day window." Two pipelines computed labels: - **Streaming attributor** emitted a label at the first span whose risk score crossed a threshold, with a 250 ms flush deadline. - **Closure scorer** re-ran attribution over the sealed trace at completion. The SLO dashboard read the streaming attributor (lowest latency). Over six weeks the streaming SLO sat at 99.2% while the closure scorer reported 99.7%. A 0.5-point gap is not noise at this volume — it is ~2,000 traces/week labeled unsafe by streaming that closure scoring exonerated. The gap was entirely trace-length dependent: traces with ≥20 spans had a 3.1-point gap, traces with ≤5 spans had a 0.1-point gap. The mechanism was retraction: long traces are long *because* they contain retries and recovery turns, and the streaming attributor had already fired on the failed attempt before the recovery span arrived. No downstream consumer ever retracted the label because the streaming attributor had no concept of retraction — a label, once emitted, was immutable. A second, subtler failure compounded it: the flush deadline (250 ms) caused traces to be *scored as complete* when they were merely *idle*. A trace with a 400 ms gap between spans (slow tool, rate-limited API) was flushed, scored, and sealed — and the subsequent spans were attached to a *new* trace ID by the ingest layer, splitting one logical invocation into two traces, each scored independently. The safety SLO saw two safe traces where the closure scorer saw one unsafe trace. ## Condition of Validity The streaming attributor's labels are valid as a *lower bound on alerting latency*, not as a *safety measurement*. Formally: for any trace `t` and any prefix `p ⊆ t`, `attribution(p)` may differ from `attribution(t)` in either direction. The only sound invariants are: 1. **Closure tagging.** Every label is emitted with `trace_closure ∈ {open, flushed, sealed}`. A label with `closure = open` may be superseded; a label with `closure = sealed` is final. SLO aggregation must count only `sealed` labels, or must count `open` labels in a separate "provisional" series that is never summed with sealed ones. 2. **Splitting is a trace-integrity violation, not a scoring event.** If a span arrives for a trace ID already marked `sealed`, that is a data-integrity error, and the sealed label must be invalidated and the trace re-scored as a single unit — never silently accepted as a new trace. 3. **Retraction is mandatory and must be propagated.** When a later span changes the attribution, the streaming attributor must emit an explicit retraction event. Consumers that already alerted must be told. These invariants hold regardless of the attribution model. They do not depend on the model being good; they depend only on the fact that partial input is not complete input. ## Decision Boundary The operational question is not "should we score on partial traces" — for latency-sensitive alerting, we must. The question is **which SLO may be computed from which closure state**: - **Alerting SLO (latency, p99 time-to-alert):** may use `open` labels. It measures responsiveness, and a provisional label is a legitimate alert trigger. - **Safety SLO (fraction unsafe):** must use `sealed` labels only, with a reconciliation window (here, 60 s) long enough to cover the p99.9 inter-span gap. If a trace never seals within the window, it is counted as `unresolved` — a third category, not folded into safe or unsafe. - **Attribution-quality metrics (precision/recall of the attributor):** must be computed by comparing `open` labels against the `sealed` label for the same trace. This is the only place the two are joined, and the join is the measurement. The boundary is crossed the moment a `sealed`-only SLO is fed from an `open`-only pipeline. That is the failure. It is invisible in the dashboard because the pipeline is *working as designed* — it is the aggregation that is unsound. ## Operational Effect After enforcing closure tagging and the reconciliation window: the streaming safety SLO moved from 99.2% to 99.65% (still 0.05 points below closure, now attributable to genuinely unresolved traces, which are reported separately). Alerting latency was unaffected because the alerting SLO continued to consume `open` labels. Trace-splitting events dropped to zero after the ingest layer began rejecting spans for sealed trace IDs and forcing re-score. The 60 s reconciliation window added 60 s of lag to the safety SLO only — acceptable, since safety SLOs are reviewed daily, not paged on. The general lesson: **an observability system that scores incomplete units must carry the incompleteness as first-class metadata through every aggregation, or it will silently report a different quantity than the one its name promises.** ```python from dataclasses import dataclass, field from enum import Enum from typing import Optional class ClosureState(Enum): OPEN = "open" FLUSHED = "flushed" SEALED = "sealed" class TraceIntegrityError(RuntimeError): """Raised when a span arrives for a trace already marked sealed.""" pass class UnsoundAggregationError(RuntimeError): """Raised when a safety SLO is fed from non-sealed labels.""" pass @dataclass class AttributionLabel: trace_id: str unsafe: bool closure: ClosureState retracted: bool = False @dataclass class SafetySLO: sealed_labels: dict[str, AttributionLabel] = field(default_factory=dict) provisional_labels: dict[str, AttributionLabel] = field(default_factory=dict) unresolved: set[str] = field(default_factory=set) def ingest(self, label: AttributionLabel) -> None: if label.closure is ClosureState.SEALED: # Sealed label is final; a later sealed label for the same trace # means the trace was split or re-scored — re-score wins. self.sealed_labels[label.trace_id] = label self.provisional_labels.pop(label.trace_id, None) self.unresolved.discard(label.trace_id) else: # Provisional only; never allowed into the safety numerator. self.provisional_labels[label.trace_id] = label def seal(self, trace_id: str, unsafe: bool) -> None: self.ingest( AttributionLabel(trace_id, unsafe, ClosureState.SEALED) ) def mark_unresolved(self, trace_id: str) -> None: if trace_id in self.sealed_labels: raise TraceIntegrityError( f"trace {trace_id} already sealed; cannot mark unresolved" ) self.unresolved.add(trace_id) self.provisional_labels.pop(trace_id, None) def safety_fraction(self) -> float: if not self.sealed_labels: raise UnsoundAggregationError( "no sealed labels; safety SLO cannot be computed " "from provisional data" ) safe = sum( 1 for l in self.sealed_labels.values() if not l.unsafe ) return safe / len(self.sealed_labels) def unresolved_fraction(self) -> float: total = ( len(self.sealed_labels) + len(self.unresolved) ) if total == 0: return 0.0 return len(self.unresolved) / total ```