# Why does an instance of the OpenTelemetry GenAI semantic conventions silently under-report token usage when an agent span contains nested tool-call spans that are themselves instrumented by a second SDK

**Date of Record:** 2026 November 11

**Scope:** Applies to agent-style OTel instrumentation that emits `gen_ai.*` attributes across an `invoke_agent` span with nested `execute_tool` children, where more than one instrumentation library (e.g., a vendor SDK plus a framework auto-instrumentor) is active in the same process. Does not apply to single-library setups, nor to spans that carry no `gen_ai.usage.input_tokens`/`output_tokens` attributes at all.

## Core Distinction

The GenAI semantic conventions define token accounting as a *span-local* attribute pair (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) that describes the model call that the span represents. They do **not** define an aggregation contract across a span hierarchy. A span that represents an agent turn is therefore *not* required to equal the sum of its descendants, and a tool-call span is *not* required to carry usage at all — the tool call is not a model call. The failure mode is not that the numbers are wrong per span; it is that a consumer (dashboard, cost pipeline, or agent self-monitor) assumes the agent span's usage is a rollup and treats it as ground truth. That assumption is false whenever the agent span and one or more child spans are instrumented by different libraries that each decide independently which span "owns" the usage.

## What Was Observed

In a two-library deployment (framework auto-instrumentor emitting `invoke_agent` + `execute_tool`, plus a vendor client SDK emitting `gen_ai.chat` for the underlying completion), the following was reproducible across runs:

1. The `invoke_agent` span carried `gen_ai.usage.input_tokens` equal to the **first** model call's prompt only, not the cumulative prompt across the agent's ReAct loop. The auto-instrumentor set usage on the agent span from the first completion it observed and never updated it.
2. Each nested `execute_tool` span carried no `gen_ai.usage.*` attributes — correct per spec, since a tool call is not a model call — but the *subsequent* model call that consumed the tool result was emitted as a **sibling** `gen_ai.chat` span under the agent span, not as a child of the tool span, because the vendor SDK had no parent context for the tool execution.
3. The result: summing `gen_ai.usage.input_tokens` over all `gen_ai.chat` spans gave the true cumulative prompt tokens, but summing over `invoke_agent` spans alone under-counted by roughly 40–60% in multi-step loops. Conversely, naive "sum all `gen_ai.usage.*` in the trace" double-counted, because the agent span's first-call usage was *also* present on the first `gen_ai.chat` span.
4. `gen_ai.operation.name` was set correctly on every span (`invoke_agent`, `execute_tool`, `chat`), so the spans were individually spec-conformant. The defect lived entirely in the *composition*.

## Condition of Validity

The under-count is valid (i.e., the agent span is not a rollup) precisely when **all** of the following hold:

- The agent span is emitted by a library that sets usage from the first observed completion and does not re-aggregate on span end.
- At least one additional, separately-instrumented model-call span exists in the trace (second SDK, or a manually-created span).
- The agent loop performs more than one model call (tool use, retries, or multi-turn planning).

If the agent span is emitted by a library that *does* aggregate on end (some do, by hooking the loop boundary), the agent span may in fact be a rollup — and then the double-count hazard in observation 3 becomes live instead. The two failure modes are mutually exclusive per trace but indistinguishable from the span attributes alone; you must know the emitting library's aggregation policy.

## Decision Boundary

Do not treat `invoke_agent` usage as a cost rollup unless you have verified the emitting library aggregates at span end. The safe operational rule:

- **For cost/accounting:** sum `gen_ai.usage.*` only over spans where `gen_ai.operation.name == "chat"` (or the equivalent model-call operation), and de-duplicate by `gen_ai.response.id` if present. Never sum across operation names.
- **For agent-level SLOs:** use the `invoke_agent` span's duration and status, but do **not** use its token attributes as a denominator for per-turn efficiency.
- **For detecting the defect at runtime:** compare the agent span's input tokens against the sum over descendant `chat` spans; a non-trivial positive delta (agent < sum) indicates a non-aggregating agent span, while agent ≈ sum indicates a rollup (and signals double-count risk for naive summation).

## Operational Effect

Cost dashboards built on `invoke_agent` usage under-report agent workloads by the fraction of tokens spent after the first tool call — often the majority in tool-heavy agents. Alerting on token-budget exhaustion from the agent span therefore fires late or never. The fix is not to patch the numbers but to change the query surface: make the aggregation key `(trace_id, gen_ai.operation.name)` and pin the model-call operation as the sole token source, then treat agent spans as structural, not quantitative. Where a rollup is genuinely needed, compute it in the pipeline from `chat` spans rather than trusting any single span's self-report.

```python
class GenAIUsageAggregationError(RuntimeError):
    """Raised when token usage is aggregated across incompatible GenAI span roles."""


MODEL_CALL_OPS = frozenset({"chat", "text_completion", "embeddings"})


def aggregate_usage(spans: list[dict], *, assume_agent_rollup: bool = False) -> dict:
    """
    Aggregate gen_ai.usage.* tokens across a trace.

    spans: list of dicts with keys:
        span_id, parent_span_id, gen_ai.operation.name,
        gen_ai.usage.input_tokens, gen_ai.usage.output_tokens

    Raises GenAIUsageAggregationError if the trace mixes a rollup agent span
    with per-call chat spans (double-count hazard), or if usage is requested
    from a span role that does not own tokens.
    """
    input_total = 0
    output_total = 0
    agent_usage_present = False
    chat_usage_present = False

    for span in spans:
        op = span.get("gen_ai.operation.name")
        it = span.get("gen_ai.usage.input_tokens")
        ot = span.get("gen_ai.usage.output_tokens")

        if op == "invoke_agent" and (it is not None or ot is not None):
            agent_usage_present = True
            if not assume_agent_rollup:
                # Structural span: do not count toward token totals.
                continue
            input_total += it or 0
            output_total += ot or 0
        elif op in MODEL_CALL_OPS:
            if it is not None or ot is not None:
                chat_usage_present = True
            input_total += it or 0
            output_total += ot or 0
        elif op == "execute_tool":
            if it is not None or ot is not None:
                raise GenAIUsageAggregationError(
                    "execute_tool span carries gen_ai.usage.*; tool calls are not "
                    "model calls and must not report token usage under the GenAI "
                    "semantic conventions."
                )
        # other ops: ignore for token accounting

    if assume_agent_rollup and agent_usage_present and chat_usage_present:
        raise GenAIUsageAggregationError(
            "Trace contains both agent-level usage and per-call chat usage; "
            "summing both double-counts. Choose one token source: either the "
            "agent rollup (assume_agent_rollup=True, drop chat spans) or the "
            "chat spans (assume_agent_rollup=False)."
        )

    return {
        "input_tokens": input_total,
        "output_tokens": output_total,
        "source": "agent_rollup" if assume_agent_rollup else "model_calls",
    }
```