# Where must an instance of gen_ai.usage.input_tokens be attributed when an agent span fans out to multiple tool calls **Date of Record:** 2026 November 16 **Scope:** Applies to OpenTelemetry instrumentation of LLM agent spans (`gen_ai.operation.name = "invoke_agent"` or `"chat"`) that emit child `execute_tool` spans and per-turn model calls. Does not apply to single-shot inference spans with no tool fan-out, nor to embeddings/reranking operations where token accounting is per-request only. ## Core Distinction The GenAI semantic conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as attributes of the *model inference* span. When an agent span encloses multiple model calls (a reasoning loop), there are two structurally different quantities that both look like "input tokens": 1. **Cumulative billed tokens** — the sum across every underlying model call, including re-sent conversation history on each turn. 2. **Unique context tokens** — the size of the prompt as materialized at the final turn (or the union of distinct content). These diverge by a factor proportional to the number of turns, because each turn re-bills the accumulated history. A five-turn agent loop with a 10k-token system prompt and growing history can easily report 50k cumulative input tokens while the unique context is 18k. Attributing the cumulative number to the parent agent span *and* to each child chat span double-counts the same tokens in any downstream aggregation that sums across the trace tree. ## What Was Observed Instrumenting a tool-using agent with a vendored OTel SDK, the parent `invoke_agent` span carried `gen_ai.usage.input_tokens` set to the sum of all child `chat` spans. A cost dashboard that aggregated "tokens per trace" by summing all spans carrying the attribute produced a 3.7x overcount versus the provider invoice. The overcount matched exactly `sum(turns) / 1` — the parent's total plus the sum of children's totals, each child already being part of the parent's sum. A second, subtler observation: child `execute_tool` spans were being stamped with the token attributes of the model call that *decided* to invoke the tool. This is semantically wrong — a tool execution consumes no model tokens — and it caused the tool span to be counted as an inference event by samplers keyed on `gen_ai.usage.*` presence. ## Condition of Validity The invariant holds only when the following are simultaneously true: - The parent agent span is a *container* (its duration encloses ≥1 child model call). - Children emit their own `gen_ai.usage.*` attributes. - Any consumer aggregates token attributes by summing across spans in a trace without deduplication. If the parent span intentionally carries a rollup and children are marked with a convention (e.g., a `gen_ai.usage.aggregate = true` marker or the parent is the *only* span carrying usage), summation is safe. The failure is specifically the *unmarked dual emission*. ## Decision Boundary The correct placement is: **token usage belongs to the span that represents the actual model inference request, never to a container span and never to a tool span.** For agent fan-out: - Each `chat` / `text_completion` child span carries its own `gen_ai.usage.input_tokens` / `output_tokens`. - The `invoke_agent` parent carries *no* `gen_ai.usage.*` attributes, OR carries them with an explicit `gen_ai.usage.aggregate = "sum"` marker that consumers must honor by deduplicating against children. - `execute_tool` spans carry zero token attributes; tool cost, if tracked, uses a separate namespace. When an agent makes a single model call and no tool fan-out, the parent and child collapse and the attribute is unambiguous — this is the case the convention was written for, and the case where the boundary silently does not apply. ## Operational Effect Enforcing single-owner attribution reduced the dashboard's reported token count to within 2% of the provider invoice and eliminated a class of sampling bugs where tool-heavy traces were over-retained because every span looked like an inference event. The cost of the fix is that consumers wanting per-agent totals must now sum children explicitly rather than reading the parent — a small aggregation change that trades a silent correctness bug for a visible query. ```python from __future__ import annotations from dataclasses import dataclass, field class TokenAttributionError(RuntimeError): """Raised when gen_ai.usage.* attributes are placed on a span that must not own them.""" @dataclass class Span: name: str kind: str # "agent" | "chat" | "tool" attributes: dict[str, object] = field(default_factory=dict) children: list["Span"] = field(default_factory=list) def has_usage(self) -> bool: return any(k.startswith("gen_ai.usage.") for k in self.attributes) def validate_attribution(span: Span) -> None: """Enforce single-owner token attribution across an agent span tree.""" if span.kind == "tool" and span.has_usage(): raise TokenAttributionError( f"tool span {span.name!r} carries gen_ai.usage.* attributes; " "tool execution consumes no model tokens" ) if span.kind == "agent" and span.has_usage(): aggregate_marker = span.attributes.get("gen_ai.usage.aggregate") if aggregate_marker != "sum": raise TokenAttributionError( f"agent container span {span.name!r} carries gen_ai.usage.* without " "gen_ai.usage.aggregate='sum'; this double-counts child chat spans" ) if not any(c.kind == "chat" and c.has_usage() for c in span.children): raise TokenAttributionError( f"agent span {span.name!r} declares aggregate usage but has no " "child chat span carrying usage; rollup is unverifiable" ) for child in span.children: validate_attribution(child) def trace_total_tokens(root: Span) -> int: """Sum tokens exactly once: prefer leaf chat spans, honor aggregate marker.""" total = 0 for span in walk(root): if span.kind == "chat" and span.has_usage(): total += int(span.attributes.get("gen_ai.usage.input_tokens", 0)) total += int(span.attributes.get("gen_ai.usage.output_tokens", 0)) return total def walk(span: Span): yield span for child in span.children: yield from walk(child) if __name__ == "__main__": good = Span( name="invoke_agent", kind="agent", children=[ Span("chat.turn1", "chat", {"gen_ai.usage.input_tokens": 1000}), Span("execute_tool.search", "tool", {}), Span("chat.turn2", "chat", {"gen_ai.usage.input_tokens": 1400}), ], ) validate_attribution(good) assert trace_total_tokens(good) == 2400 bad = Span( name="invoke_agent", kind="agent", attributes={"gen_ai.usage.input_tokens": 2400}, children=[Span("chat.turn1", "chat", {"gen_ai.usage.input_tokens": 1000})], ) validate_attribution(bad) # raises TokenAttributionError ```