# Where must an instance of `gen_ai.usage.input_tokens` be attributed when an agent span both invokes a tool and re-enters the model?

**Date of Record:** 2026 November 12
**Scope:** Applies to OpenTelemetry GenAI semantic conventions (v1.x, `gen_ai.*` attributes) for agent spans that emit both tool-call child spans and follow-up model invocations within a single logical agent turn. Does not apply to single-shot inference spans, nor to spans where token usage is reported only at the session/root level by an external aggregator.

## Core Distinction

The GenAI conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as *per-inference-call* attributes, not per-agent-turn totals. A common and non-obvious failure arises because an agent span is conceptually one "operation" but physically contains *N* model invocations (one before a tool call, one after the tool result is returned). The invariant that operators assume — "sum of `gen_ai.usage.input_tokens` over all spans equals the billed input tokens for the turn" — is **false** unless every model invocation is represented by its own span and the agent span itself carries *no* usage attributes.

The distinction is between **span-scoped accounting** (usage belongs to the span that performed the inference) and **turn-scoped accounting** (usage belongs to the logical agent operation). The spec mandates span-scoped. Many SDK auto-instrumentations and hand-rolled agent loops silently do turn-scoped, producing double-counting or under-counting.

## What Was Observed

In a LangChain-style agent loop instrumented with a custom `gen_ai` exporter, the following span tree was emitted for a single user turn that required one tool call:

- `invoke_agent` (span kind INTERNAL) — attributes: `gen_ai.operation.name=invoke_agent`, `gen_ai.usage.input_tokens=1840`, `gen_ai.usage.output_tokens=112`
- └ `chat` (span kind CLIENT) — `gen_ai.operation.name=chat`, `gen_ai.usage.input_tokens=1840`, `gen_ai.usage.output_tokens=112`
- └ `execute_tool` (span kind INTERNAL) — `gen_ai.operation.name=execute_tool`, `gen_ai.tool.name=search`
- └ `chat` (span kind CLIENT) — `gen_ai.operation.name=chat`, `gen_ai.usage.input_tokens=2410`, `gen_ai.usage.output_tokens=64`

The agent span's `input_tokens` was set by copying the *first* chat call's usage, not the sum. Downstream, a cost dashboard summed `gen_ai.usage.input_tokens` across all spans (the natural Prometheus-style aggregation), yielding 1840 + 1840 + 2410 = **6090 tokens**, versus the true billed 4250. The agent span contributed a phantom 1840. Worse, when a second exporter (OTLP → vendor backend) deduplicated by parent span, the agent span's stale value *overwrote* the child's, so the vendor console reported 1840 for the whole turn.

The failure is silent: no span is marked error, no attribute is out of range, and the metric is internally consistent per-span. It only manifests as a discrepancy against the provider's billing API — often a 20–40% overcount that is misattributed to "tokenizer drift."

## Condition of Validity

The invariant "sum of span-level `gen_ai.usage.*` equals provider billing" holds **only if all three** of the following are true:

1. Every model invocation within the turn is represented by exactly one span carrying `gen_ai.operation.name` in `{chat, text_completion, generate_content}`.
2. No ancestor span (agent, workflow, session) carries `gen_ai.usage.input_tokens` or `gen_ai.usage.output_tokens` — these attributes are *reserved* for inference spans.
3. Tool-execution spans carry no `gen_ai.usage.*` attributes at all, even if the tool internally calls a model (in which case the tool's inner inference must emit its *own* child `chat` span, not decorate the tool span).

Condition (2) is the one violated in practice. Agent frameworks that expose an `on_llm_end` callback at the agent level naturally write usage onto the agent span, because that is the only span they hold a handle to. The spec's `gen_ai.agent.*` namespace has no `usage` attributes precisely to prevent this.

## Decision Boundary

When deciding where to place usage attributes, use this test:

- **Does this span directly correspond to a single request to a model provider endpoint?** → Yes: it is an inference span; write `gen_ai.usage.*` here.
- **Does this span aggregate or orchestrate multiple such requests?** → Yes: it is an agent/workflow span; it MUST NOT carry `gen_ai.usage.*`. Aggregate downstream via span aggregation, not by hoisting.
- **Is the tool result itself a model output (e.g., an LLM-as-tool)?** → The tool span is still `execute_tool`; the inner inference gets a nested `chat` span. Do not merge.

If an exporter cannot guarantee condition (1) — e.g., streaming responses where usage is only known at stream end and the span may already have ended — the correct fallback is to emit a *separate* `chat` span with a known start/end and attach usage there, or to emit usage as span *events* (`gen_ai.usage.reported`) rather than attributes. Never backfill usage onto an ancestor.

## Operational Effect

The practical consequence: cost attribution per agent, per tool, and per turn diverges from provider billing in a way that is *structurally biased upward* (because the first inference's usage is the most likely to be hoisted). Teams observing this typically add a "reconciliation" job that compares summed span tokens against the billing API and applies a fudge factor — masking the root cause. The correct remediation is to strip `gen_ai.usage.*` from all non-inference spans at the collector (`transform` processor with `delete_key` on spans where `gen_ai.operation.name` is not in the inference set) and to enforce the invariant at instrumentation time with a validator that raises on violation.

```python
from __future__ import annotations

from typing import Iterable, Mapping, Any

INFERENCE_OPS = frozenset({"chat", "text_completion", "generate_content"})
USAGE_KEYS = ("gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens")


class GenAIUsageAttributionError(RuntimeError):
    """Raised when gen_ai.usage.* is attached to a span that is not an inference span."""


def validate_usage_attribution(spans: Iterable[Mapping[str, Any]]) -> None:
    """
    Enforce the OpenTelemetry GenAI invariant: usage attributes belong ONLY to
    inference spans (operation.name in INFERENCE_OPS). Agent, workflow, and
    execute_tool spans must not carry usage; doing so double-counts when
    downstream consumers sum span-level tokens against provider billing.

    `spans` is a list of dicts with keys: 'name', 'attributes'.
    """
    for span in spans:
        attrs = span.get("attributes", {})
        op = attrs.get("gen_ai.operation.name")
        present = [k for k in USAGE_KEYS if k in attrs]
        if not present:
            continue
        if op not in INFERENCE_OPS:
            raise GenAIUsageAttributionError(
                f"span {span.get('name')!r} (operation.name={op!r}) carries "
                f"{present}; usage attributes are reserved for inference spans "
                f"{sorted(INFERENCE_OPS)}. Hoist usage to the child chat span "
                f"or emit a gen_ai.usage.reported span event instead."
            )
        # Sanity: input/output must be non-negative integers.
        for k in present:
            v = attrs[k]
            if not isinstance(v, int) or v < 0:
                raise GenAIUsageAttributionError(
                    f"span {span.get('name')!r} has malformed {k}={v!r}; "
                    f"expected non-negative int."
                )


def reconcile_turn_tokens(spans: Iterable[Mapping[str, Any]], billed_input: int) -> None:
    """Compare summed inference-span input tokens against provider billing."""
    total = 0
    for span in spans:
        attrs = span.get("attributes", {})
        if attrs.get("gen_ai.operation.name") in INFERENCE_OPS:
            total += int(attrs.get("gen_ai.usage.input_tokens", 0))
    if total != billed_input:
        raise GenAIUsageAttributionError(
            f"summed inference input tokens ({total}) != billed ({billed_input}); "
            f"check for usage hoisted onto agent/tool spans."
        )
```