# Under what conditions must an instance treat OpenTelemetry GenAI agent spans as authoritative for token accounting rather than as advisory telemetry

**Date of Record:** 2026 November 12

**Scope:** Applies to systems that ingest OTLP spans conforming to the OpenTelemetry GenAI semantic conventions (`gen_ai.*`) emitted by agent runtimes that execute multi-step tool-calling loops, and that use span attributes to reconcile billed token usage. Does not apply to single-shot LLM completion spans, to raw provider billing exports, or to spans where the agent runtime is not instrumented with token attributes at all.

## Core Distinction

There is a persistent ambiguity between two quantities that both look like "tokens used by this agent run":

1. **Per-span token attributes** (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) attached to each individual LLM-inference span inside the agent loop.
2. **Aggregate run-level accounting** implied by summing those spans across the whole trace — including retries, tool-call round trips, and any spans that were sampled out or dropped by the collector.

The GenAI conventions are explicit that `gen_ai.usage.*` describes *that inference call*, not the agent run. They are also silent on whether a parent agent span must carry a rolled-up total, and most instrumentations do not emit one. The boundary condition is this: **a span-level token attribute is authoritative for exactly the inference call it decorates, and for nothing else.** Treating the sum of child inference spans as the run total is only valid under a set of conditions that are frequently violated in agent workloads — retries that reuse a span name, tool calls that themselves invoke sub-agents, streaming spans that finalize late, and sampling policies that drop a fraction of the loop.

The non-obvious invariant: in an agent trace, the *number of inference spans* is not a stable function of the *number of logical model calls*. A single logical tool-augmented step can produce N inference spans (one per retry), and a single inference span can produce zero billable output tokens if it terminated on a tool-call stop reason with empty content. Summing spans therefore over- or under-counts depending on which failure mode the runtime hit, and the span attributes will look perfectly well-formed in every case.

## What Was Observed

During reconciliation of agent-run token totals against provider invoices across a tool-calling fleet, the following pattern appeared repeatedly:

- A trace with a parent `gen_ai.agent` span and 7 child `gen_ai.chat` spans reported a summed `output_tokens` of 41,200. The provider invoice for the same run was 38,900. The delta of 2,300 matched exactly the output tokens of a retried inference span that had been emitted with the same `gen_ai.operation.name = chat` and the same `gen_ai.request.model`, differing only in `trace.span.id`. The retry span was a *duplicate logical call*, not a separate billable call.
- Conversely, a run whose trace contained 4 child spans billed for 6 inference calls. Two spans were absent because the collector's tail-sampling policy dropped spans with `status.code = ERROR`, and the errored inference calls (which had still consumed input tokens before failing) were silently excluded from the sum.
- Tool-call spans (`gen_ai.operation.name = execute_tool`) carried `gen_ai.tool.name` and `gen_ai.tool.call.id` but no token attributes, as expected. However, a sub-agent invoked *inside* a tool call emitted its own nested `gen_ai.agent` span with its own token attributes. Naive recursive summation of all `gen_ai.usage.*` attributes in the trace double-counted the nested agent's tokens into both the parent run and the sub-agent scope.
- Streaming inference spans occasionally finalized with `gen_ai.usage.output_tokens` unset when the stream was aborted by client disconnect, even though the provider billed for the partial generation.

In every case the spans were schema-valid. No attribute was malformed. The failure is semantic, not syntactic.

## Condition of Validity

Span-summed token accounting for an agent run is authoritative **only when all of the following hold simultaneously**:

1. **Retry identity is explicit.** Each inference span carries a retry discriminator (e.g., a `gen_ai.request.retry_attempt`-style attribute or a shared logical-call id) such that duplicate logical calls can be collapsed. If the runtime emits retries as indistinguishable sibling spans, the sum is an upper bound, not a total.
2. **Sampling is complete for the trace.** The collector must not drop child inference spans of a trace whose parent agent span is retained. Head-based sampling that keeps or drops whole traces is acceptable; tail-based sampling that drops individual error spans is not.
3. **Nested agent spans are scoped.** Sub-agent spans must be attributable to a distinct agent scope (a `gen_ai.agent.name` or equivalent), so that a recursive walk can exclude nested scopes from the parent total.
4. **Streaming finalization is guaranteed.** Every inference span, including aborted streams, must finalize with token attributes populated — or must be marked with a status that excludes it from billing reconciliation rather than being silently counted as zero.
5. **Tool-call spans carry no token attributes.** The invariant that `execute_tool` spans are token-free must hold; if a runtime begins attaching model tokens to tool spans, summation will double-count.

If any condition fails, the summed value is advisory telemetry — useful for latency and behavior analysis, unfit for billing or quota enforcement.

## Decision Boundary

The operational decision is whether to *trust* or *reconcile* the span-derived total:

- If conditions 1–5 hold → trust the span sum as the run total; emit it as a derived aggregate metric and close the loop against invoices with a tight tolerance.
- If any condition fails → do **not** emit a derived total from spans. Instead, mark the trace as `accounting_incomplete` and route it to a reconciliation path that falls back to provider-side usage records. Do not silently emit a partial sum, because a partial sum is indistinguishable from a correct sum at the metric layer.
- If retry identity is absent but sampling and scoping are sound → the span sum is a valid *upper bound*; use it for quota soft-limits but never for hard billing.
- If nested agent scoping is absent → refuse to aggregate at all; the trace is structurally ambiguous and any total will be wrong in an unknown direction.

The boundary is crossed the moment a runtime change alters retry emission, sampling policy, or nested-agent instrumentation without a corresponding change to the reconciliation logic. Because the spans remain schema-valid across all these changes, the reconciliation logic cannot detect the crossing from the data alone — it must be asserted from configuration.

## Operational Effect

Systems that skip this check exhibit a characteristic failure: token totals drift from invoices in a direction that correlates with error rate and retry volume, and the drift is invisible in per-span dashboards because every individual span is correct. The aggregate is wrong while every input is right. This is the signature of an invalid summation, not a bug in instrumentation.

The mitigation is a guard at the aggregation boundary that validates the five conditions from trace metadata before emitting a total, and refuses (with a typed error) when they are unmet, forcing the caller onto the reconciliation path. The guard must fail closed: an unverifiable total is worse than no total, because downstream quota enforcement and cost attribution will consume it as if it were exact.

```python
from dataclasses import dataclass, field
from typing import Iterable, Optional


class TokenAccountingInvariantError(RuntimeError):
    """Raised when span-derived token totals cannot be trusted as authoritative."""
    pass


@dataclass
class GenAISpan:
    span_id: str
    operation_name: str            # "chat" | "execute_tool" | "agent"
    agent_scope: Optional[str]     # gen_ai.agent.name, None for root
    retry_discriminator: Optional[str]
    input_tokens: Optional[int]
    output_tokens: Optional[int]
    status_code: str               # "OK" | "ERROR" | "UNSET"
    stream_finalized: bool = True


@dataclass
class TraceAccountingConfig:
    sampling_is_complete: bool
    nested_scopes_declared: bool


def _collapse_retries(spans: Iterable[GenAISpan]) -> list[GenAISpan]:
    seen: dict[str, GenAISpan] = {}
    for s in spans:
        if s.retry_discriminator is None:
            raise TokenAccountingInvariantError(
                f"inference span {s.span_id} lacks retry discriminator; "
                "sum would be an upper bound, not a total"
            )
        # keep the last attempt per logical call id
        seen[s.retry_discriminator] = s
    return list(seen.values())


def authoritative_run_tokens(
    spans: list[GenAISpan],
    root_scope: str,
    config: TraceAccountingConfig,
) -> int:
    if not config.sampling_is_complete:
        raise TokenAccountingInvariantError(
            "child inference spans may have been dropped by tail sampling; "
            "span sum is advisory only"
        )

    inference = [
        s for s in spans
        if s.operation_name == "chat" and s.agent_scope == root_scope
    ]

    for s in inference:
        if not s.stream_finalized:
            raise TokenAccountingInvariantError(
                f"span {s.span_id} is an unfinalized stream; token attributes "
                "are unreliable and must not be counted as zero"
            )
        if s.input_tokens is None or s.output_tokens is None:
            raise TokenAccountingInvariantError(
                f"span {s.span_id} missing gen_ai.usage.* attributes"
            )

    for s in spans:
        if s.operation_name == "execute_tool" and (
            s.input_tokens or s.output_tokens
        ):
            raise TokenAccountingInvariantError(
                f"tool span {s.span_id} carries token attributes; "
                "summation would double-count"
            )

    has_nested = any(
        s.operation_name == "agent" and s.agent_scope != root_scope
        for s in spans
    )
    if has_nested and not config.nested_scopes_declared:
        raise TokenAccountingInvariantError(
            "nested agent spans present without declared scoping; "
            "aggregate total is structurally ambiguous"
        )

    collapsed = _collapse_retries(inference)
    return sum((s.input_tokens or 0) + (s.output_tokens or 0) for s in collapsed)
```