# How does an instance detect the token-accounting double-count boundary in OpenTelemetry GenAI agent traces

**Date of Record:** 2026 November 12

**Scope:** Applies to agent runtimes that emit OpenTelemetry spans under the GenAI semantic conventions (`gen_ai.*`) when an LLM invocation is wrapped by both an outer agent/invocation span and an inner chat/`gen_ai.client` span, and when nested retries or tool-mediated sub-invocations re-emit usage. Does not apply to single-span, single-call instrumentation where exactly one span carries `gen_ai.usage.*` attributes per provider response.

## Core Distinction

The GenAI semantic conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as attributes of the span that *represents the model call*. They do not define a summation rule across a trace. The invariant that operators assume—"sum of `gen_ai.usage.*` over all spans in a trace equals the billable token count"—is false the moment instrumentation attaches usage to more than one span for the same provider response. This happens routinely in agent frameworks that (a) record usage on the `gen_ai.chat` child span and (b) mirror the aggregate onto the enclosing `gen_ai.invoke_agent` span for convenient trace-level querying. Both are individually spec-compliant. Together they double-count.

The non-obvious part is that the double-count is *not* detectable by inspecting any single span. Each span is internally consistent: `input_tokens + output_tokens == total_tokens` holds on every span that carries the trio. The violation is a trace-level invariant, and it only manifests as a discrepancy between the trace-derived total and the provider's authoritative usage (from the response body or billing export).

## What Was Observed

In a production agent trace with a single user turn, the following span structure was emitted:

- `gen_ai.invoke_agent` (root): `gen_ai.usage.input_tokens=1840`, `gen_ai.usage.output_tokens=212`, `gen_ai.usage.total_tokens=2052`.
- `gen_ai.chat` (child, one provider call): `gen_ai.usage.input_tokens=1840`, `gen_ai.usage.output_tokens=212`, `gen_ai.usage.total_tokens=2052`.
- `gen_ai.execute_tool` (child): no usage attributes (correct).

A naive collector query `sum(gen_ai.usage.total_tokens)` over the trace returned 4104 for a call the provider billed at 2052. A second observed variant is subtler: a retry wrapper emits `gen_ai.chat` twice (attempt 1 fails after partial generation, attempt 2 succeeds), and the parent aggregates *both* attempts' usage, so the trace total reflects tokens the provider did not bill for the failed attempt (or billed at a different rate). Here the parent is the only span that disagrees with the sum of its children, which is the only locally-visible signature.

A third variant: streaming responses where `gen_ai.usage.output_tokens` is emitted on the `gen_ai.chat` span at stream end, but a framework-level "agent step" span also emits usage computed from a local tokenizer. The two disagree by 2–7% because the local tokenizer is not the provider's. Neither is wrong per spec; the sum is meaningless.

## Condition of Validity

The "sum usage across the trace" rule is valid only when *exactly one span per provider response* carries `gen_ai.usage.*`, and that span is the leaf-most span representing the call. This condition holds when:

1. Instrumentation is provided by the provider SDK's native OTel hook and no framework layer re-emits usage upward.
2. Retries are modeled as sibling spans under a common parent that does *not* carry usage, or the retry wrapper discards failed-attempt usage.
3. No local tokenizer is used to synthesize usage attributes.

The condition is violated whenever a parent span and a child span both carry usage for the same underlying provider response, or when a failed attempt's usage is retained in an aggregate. Because the violation is trace-level, a per-span validator (which is what most OTel attribute checkers implement) will pass every span.

## Decision Boundary

An instance should treat trace-level usage as *unreliable and non-summable* and switch to a leaf-only accounting mode when any of the following is true:

- More than one span in the trace carries `gen_ai.usage.total_tokens` and one is an ancestor of another.
- A span carrying usage has a descendant span that also carries usage (ancestor/descendant usage collision).
- The trace contains a retry span whose parent aggregates usage across attempts.
- Any usage attribute was produced by a tokenizer whose identity differs from the provider's (detectable via absence of a provider `response.id` correlation or presence of a `gen_ai.usage.*` value that does not match the recorded response body).

Conversely, when exactly one leaf span per response carries usage and no ancestor does, summation is safe and should be used. The boundary is *structural* (span ancestry + usage presence), not numerical—do not attempt to detect it by comparing magnitudes, because equal parent/child values (the mirroring case) and unequal values (the retry case) are both violations.

## Operational Effect

If an instance ignores this boundary, cost dashboards, budget alarms, and per-tenant chargeback computed from trace sums will over-report by a factor equal to the usage-carrying span depth. In the mirroring case the factor is exactly 2.0 and is stable, so it is often "corrected" by a hardcoded division that silently breaks when a framework upgrade removes the parent mirroring—producing a 2x *under*-report. In the retry case the error is variable and unbounded by attempt count. The correct operational posture is to compute usage from leaf spans only (spans with usage that have no descendant with usage), tag the trace with an accounting mode, and refuse to emit a summed total when the structural invariant fails, surfacing the ambiguity to the consumer rather than silently picking a number.

```python
from __future__ import annotations

from dataclasses import dataclass


class UsageAccountingBoundaryError(RuntimeError):
    """Raised when trace-level GenAI token accounting cannot be resolved safely."""


@dataclass(frozen=True)
class Span:
    span_id: str
    parent_id: str | None
    name: str
    has_usage: bool
    total_tokens: int | None = None


def _ancestors(span: Span, by_id: dict[str, Span]) -> list[Span]:
    chain: list[Span] = []
    cur = span
    while cur.parent_id is not None:
        parent = by_id.get(cur.parent_id)
        if parent is None:
            break
        chain.append(parent)
        cur = parent
    return chain


def compute_trace_usage(spans: list[Span]) -> int:
    """Sum leaf-only usage; raise if the structural invariant is violated.

    Invariant: for every span carrying usage, no ancestor and no descendant
    may also carry usage. Violation means the trace is non-summable and the
    caller must not trust a single number.
    """
    by_id = {s.span_id: s for s in spans}
    usage_spans = [s for s in spans if s.has_usage]

    for span in usage_spans:
        for anc in _ancestors(span, by_id):
            if anc.has_usage:
                raise UsageAccountingBoundaryError(
                    f"usage collision: span {span.span_id!r} ({span.name}) and "
                    f"ancestor {anc.span_id!r} ({anc.name}) both carry gen_ai.usage.*; "
                    "trace total is double-counted and non-summable"
                )

    # Leaf-only: a usage span with a usage-bearing descendant is already excluded
    # by the ancestor check above, so remaining usage spans are leaves w.r.t. usage.
    total = 0
    for span in usage_spans:
        if span.total_tokens is None:
            raise UsageAccountingBoundaryError(
                f"span {span.span_id!r} ({span.name}) has usage attributes but no "
                "gen_ai.usage.total_tokens; cannot account"
            )
        total += span.total_tokens
    return total
```