# Which invariants prevent double-counting of `gen_ai.usage.*` tokens across nested agent spans in OpenTelemetry GenAI traces

**Date of Record:** 2026 November 12

**Scope:** Applies to OpenTelemetry GenAI semantic conventions (v1.27+) where an agent orchestrator emits parent spans for a multi-turn conversation while child spans model individual LLM invocations, tool calls, or sub-agent delegations. Does not apply to flat, single-span-per-request instrumentation, nor to backends that recompute usage from raw provider payloads rather than trusting span attributes.

## Core Distinction

The GenAI semantic conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as attributes of the span representing *the model invocation that consumed them*. They do not define an aggregation rule for ancestor spans. This creates an ambiguity that operators resolve inconsistently: does a parent agent span carry the sum of its descendants' usage, or only usage it generated directly (e.g., its own planning call)?

The non-obvious boundary is this: **the conventions are span-local, but agent traces are hierarchical.** If a parent span copies the sum of child usage into its own `gen_ai.usage.*` attributes, any backend that sums the attribute across *all* spans in a trace double-counts. If the parent omits usage entirely, backends that read only the root span under-report. Both failure modes are silent — the trace is structurally valid and passes schema validation in either case.

## What Was Observed

In a multi-turn tool-using agent (orchestrator span → N LLM-invocation spans → M tool-execution spans, some turns fanning out to sub-agents), we observed three distinct accounting pathologies:

1. **Sum-over-all-spans inflation.** A backend computed `sum(gen_ai.usage.input_tokens)` over every span in the trace. Because the orchestrator span had been instrumented to carry the aggregate (a common "convenience" pattern), every token was counted roughly twice — once on the leaf invocation span, once on the ancestor. Inflation factor tracked the nesting depth, not a constant.

2. **Root-only under-count.** A second backend read usage only from the root span. In traces where the orchestrator did *not* aggregate (correct per span-local semantics), it reported zero tokens for the entire conversation, because the orchestrator only issued a small planning call and the leaves held the real usage.

3. **Retry and fan-out leakage.** When an LLM invocation span was retried (same logical turn, multiple physical attempts), each attempt emitted its own span with its own usage. Summing across spans counted failed attempts' partial token consumption, which is *correct* for cost but *incorrect* for the logical "tokens attributable to this turn." The two questions — "what did the provider bill" and "what did this turn consume" — diverge exactly at retries and streaming aborts.

The invariant that resolves all three: **usage attributes are additive only across *disjoint* spans, and a span carrying aggregated usage must be marked so it is excluded from leaf-level summation.** The conventions provide no built-in marker; the de-facto signal is span kind and the presence of child spans, neither of which is reliable.

## Condition of Validity

The safe rule holds only under these conditions:

- Every span that emits `gen_ai.usage.*` represents a *single physical provider call* (one request/response pair), not a logical turn and not an aggregate.
- Aggregation, when needed, is expressed in a *separate, explicitly named* attribute (e.g., `agent.turn.input_tokens_aggregate`) or a span event, never by overloading `gen_ai.usage.*`.
- Retries emit sibling spans under the same turn span, each with its own usage, and the logical-turn total is computed by the consumer with an explicit retry policy (include or exclude failed attempts — but decide, and record the decision).
- Streaming responses that abort mid-stream emit usage for tokens *actually received*, with `gen_ai.response.finish_reasons` reflecting the abort, so consumers can distinguish truncated accounting from complete accounting.

When any of these is violated — most commonly when an orchestrator aggregates into its own `gen_ai.usage.*` — the trace remains schema-valid but the accounting invariant is broken, and no validator will flag it.

## Decision Boundary

Choose **leaf-only emission** (usage on physical-call spans only; no aggregation on ancestors) when: your backend can walk the full span tree, you need retry-aware cost attribution, or you fan out to sub-agents whose usage must be attributed to distinct cost centers. This is the default and the only mode that survives arbitrary backend aggregation strategies.

Choose **explicit aggregate attributes on ancestors** (never reusing `gen_ai.usage.*`) only when: a downstream consumer genuinely cannot traverse children (e.g., a metrics pipeline that receives only root spans), and you accept that the aggregate is a *derived, non-authoritative* value that must be reconciled against leaf sums. In that case the aggregate attribute name must be unmistakably distinct so no generic `gen_ai.usage.*` summation picks it up.

The boundary is crossed the moment an engineer, seeing a dashboard that "misses" nested usage, "fixes" it by writing the sum onto the parent's `gen_ai.usage.*`. That fix is the bug.

## Operational Effect

Traces with leaf-only emission and explicit aggregates are robust to backend summation strategy: sum-over-all-spans equals sum-over-leaves (because ancestors carry no `gen_ai.usage.*`), and root-only readers fall back to the named aggregate. Cost dashboards stop inflating with nesting depth. Retry accounting becomes a documented consumer policy rather than an emergent artifact of span structure. The residual risk is purely contractual: a future maintainer must not "helpfully" populate `gen_ai.usage.*` on an orchestrator span. That is the invariant to enforce in code review and, ideally, in a trace-validation linter.

The following guard encodes the boundary as an assertion at span-emission time: an orchestrator/agent span that has (or will have) child spans must not carry `gen_ai.usage.*` attributes; aggregated totals go to a distinct namespace.

```python
from __future__ import annotations

from dataclasses import dataclass, field


class UsageAggregationInvariantError(RuntimeError):
    """Raised when a span that models a logical turn or agent orchestration
    carries gen_ai.usage.* attributes, which would double-count tokens under
    sum-over-all-spans backends."""


# Attributes that are span-local per the GenAI semantic conventions and must
# never appear on a span that aggregates descendants.
LEAF_ONLY_USAGE_ATTRS = frozenset(
    {"gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens"}
)

# Explicit, non-authoritative aggregate namespace. Distinct from gen_ai.usage.*
# so generic summation never picks it up.
AGGREGATE_NAMESPACE = "agent.turn."


@dataclass
class SpanModel:
    name: str
    attributes: dict[str, object] = field(default_factory=dict)
    has_or_will_have_children: bool = False
    is_physical_model_call: bool = False


def validate_usage_placement(span: SpanModel) -> None:
    """Enforce that gen_ai.usage.* lives only on physical-call (leaf) spans.

    A span that is an orchestrator or a logical turn (i.e., has or will have
    children) must not carry gen_ai.usage.*; it may carry aggregated totals
    under the AGGREGATE_NAMESPACE instead.
    """
    usage_attrs_present = LEAF_ONLY_USAGE_ATTRS & span.attributes.keys()

    if not usage_attrs_present:
        return

    if span.has_or_will_have_children and not span.is_physical_model_call:
        raise UsageAggregationInvariantError(
            f"span {span.name!r} carries {sorted(usage_attrs_present)} but "
            f"also aggregates child spans; this double-counts tokens under "
            f"sum-over-all-spans backends. Move totals to the "
            f"{AGGREGATE_NAMESPACE!r} namespace or emit usage only on leaves."
        )

    # A physical call may legitimately have children (e.g., a streaming span
    # with chunk events), but those children must not themselves emit
    # gen_ai.usage.*; that is enforced at the child's own validation call.


def emit_aggregate(span: SpanModel, input_tokens: int, output_tokens: int) -> None:
    """Record a derived, non-authoritative aggregate on an ancestor span.

    Never writes into gen_ai.usage.*; consumers that sum leaves must be able
    to ignore this namespace entirely.
    """
    span.attributes[f"{AGGREGATE_NAMESPACE}input_tokens_aggregate"] = input_tokens
    span.attributes[f"{AGGREGATE_NAMESPACE}output_tokens_aggregate"] = output_tokens


# Example: the failure this guard is designed to catch.
if __name__ == "__main__":
    orchestrator = SpanModel(
        name="invoke_agent",
        attributes={
            "gen_ai.usage.input_tokens": 8400,   # WRONG: aggregated from children
            "gen_ai.usage.output_tokens": 1200,
        },
        has_or_will_have_children=True,
        is_physical_model_call=False,
    )

    try:
        validate_usage_placement(orchestrator)
    except UsageAggregationInvariantError as exc:
        # Correct remediation: strip leaf-only attrs, emit explicit aggregate.
        for attr in LEAF_ONLY_USAGE_ATTRS:
            orchestrator.attributes.pop(attr, None)
        emit_aggregate(orchestrator, input_tokens=8400, output_tokens=1200)
        validate_usage_placement(orchestrator)  # now passes
        print(f"remediated: {exc}")
```