# What happens when an instance emits GenAI tool-call spans with `gen_ai.usage.input_tokens` counted at the wrong layer **Date of Record:** 2026 November 15 **Scope:** Applies to OpenTelemetry GenAI semantic conventions (v1.27–v1.34) for chat/inference spans that contain `gen_ai.tool.call` child spans and report token usage. Does not apply to pure embedding spans, or to spans where token accounting is delegated entirely to a downstream provider SDK without instrumentation. ## Core Distinction The GenAI semantic conventions define two independent attribute families that are frequently conflated at instrumentation time: 1. **Usage attributes** (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) — defined as the token counts *for the model inference operation represented by the span*. 2. **Tool-call attributes** (`gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.tool.type`) — defined on the *tool execution span*, which is a child of the inference span. The non-obvious invariant is this: **a tool-call span must not carry `gen_ai.usage.*` attributes, and the parent inference span's `input_tokens` must include the tokens consumed by the assistant's tool-call turn, not merely the tokens of the initial prompt.** When instrumentation attaches usage to the tool span (because the tool span happens to be the last thing that touched the response object), the parent span reports input tokens for the *first* turn only, and the tool span reports tokens for a *non-inference* operation. The trace is structurally valid — every span has a parent, no attribute is malformed — but the aggregate token accounting is silently wrong, and it is wrong in a direction that under-counts. ## What Was Observed An agent loop was instrumented with a hand-rolled span wrapper that emitted one `gen_ai.chat` span per model call and one `gen_ai.tool` span per tool invocation. The wrapper read `response.usage` from the provider response object and, on any span that received a non-null usage object, wrote `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`. Because the tool-execution path re-entered the wrapper with the *same* response object still in scope (the tool result was folded back into the message list synchronously), the tool span also saw a non-null `usage`. Observed consequences over a 40-minute agent run: - Parent `gen_ai.chat` spans reported `input_tokens` equal to the first-turn prompt only. Multi-turn tool loops reported the same input count on every turn (the initial prompt), not the growing conversation. - `gen_ai.tool` child spans carried `gen_ai.usage.input_tokens` values that belonged to the *parent's* inference call, inflating per-tool token metrics. - Summing `gen_ai.usage.input_tokens` across all spans in a trace produced a number roughly 2–3× the true prompt-token consumption, while summing only `gen_ai.chat` spans produced a number roughly 0.4× the true consumption. Neither aggregation was correct, and each looked plausible in isolation. - Cost dashboards keyed on `gen_ai.chat` spans under-reported by ~60% on tool-heavy traces and were accurate on single-turn traces — so the error correlated with tool use and escaped the smoke test. No error was raised. No span was dropped. The OTLP export succeeded. The failure was entirely in the *semantic placement* of an attribute, not in its presence or type. ## Condition of Validity The convention's token accounting is only meaningful when all of the following hold: - `gen_ai.usage.input_tokens` is set **only** on `gen_ai.chat` / inference spans, and reflects the *cumulative* prompt tokens for that specific model call (including prior assistant turns and tool results already in the context). - `gen_ai.tool.call.id` on the tool span matches the `tool_calls[].id` in the parent's assistant message, so the join between inference turn and tool execution is unambiguous. - The parent span is closed *after* the tool result is available if the span is intended to represent the full turn; otherwise the tool span is a sibling, not a child, and the parent's usage must not be back-patched. - `gen_ai.usage.output_tokens` on the parent counts the assistant's generation, including any tool-call arguments emitted as structured output — not just natural-language text. The convention is **not** valid when usage is attached by object identity (whoever holds the response object writes the attribute), when a single response object is reused across spans, or when tool spans are nested under a chat span but the chat span's usage is computed from a pre-tool snapshot. ## Decision Boundary The operational decision is: **where does the usage attribute get written, and against which response snapshot?** - If your instrumentation writes usage at the point the provider response is received, *before* the tool result is folded in, the parent span is a valid single-turn record. Do not also emit usage on the tool span. Accept that the trace's total input tokens equal the sum over chat spans only. - If your instrumentation writes usage after the full turn (response + tool result + follow-up), the parent span represents a multi-turn turn and its `input_tokens` must be recomputed from the final message list, not read from the original response. Emitting the original snapshot here is the failure mode above. - If a tool span is a child of a chat span, the tool span must be treated as a **non-inference** span: no `gen_ai.usage.*`, only `gen_ai.tool.*`. Any consumer that sums usage across all spans in a trace is making an invalid assumption and should be corrected, not accommodated. The boundary is crossed the moment a single mutable response object is allowed to be read by more than one span-emitting code path. ## Operational Effect Downstream consumers — cost aggregation, token-budget enforcement, rate-limit forecasting, and eval harnesses that normalize by token count — all inherit the placement error. Because the error is directional (under-count on chat spans, double-count on tool spans) and correlates with tool use, it degrades exactly the workloads (agents, multi-step reasoning) that the GenAI conventions were designed to observe. It is undetectable by schema validation, span-count checks, or latency monitoring; it surfaces only under a cross-span reconciliation that compares summed `gen_ai.chat` input tokens against the provider's billing record for the same window. The remediation is a guard at instrumentation time: assert that usage attributes appear on exactly the inference spans and nowhere else, and assert that the sum of chat-span input tokens reconciles to the provider's reported prompt tokens within a tolerance. The code below encodes that invariant as a check that fails loudly rather than exporting a plausible-but-wrong trace. ```python from dataclasses import dataclass from typing import Iterable class GenAIUsagePlacementError(RuntimeError): """Raised when gen_ai.usage.* attributes are attached to a non-inference span or when summed chat-span input tokens fail to reconcile with provider billing.""" pass @dataclass(frozen=True) class SpanRecord: name: str parent_id: str | None attributes: dict USAGE_KEYS = ("gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens") INFERENCE_SPAN_NAMES = ("gen_ai.chat", "gen_ai.inference") def validate_usage_placement( spans: Iterable[SpanRecord], provider_prompt_tokens: int, tolerance_ratio: float = 0.02, ) -> None: """Enforce that usage attributes live only on inference spans, and that chat-span input tokens reconcile with the provider's billing record.""" spans = list(spans) for span in spans: carries_usage = any(k in span.attributes for k in USAGE_KEYS) is_inference = span.name in INFERENCE_SPAN_NAMES if carries_usage and not is_inference: raise GenAIUsagePlacementError( f"span {span.name!r} (parent={span.parent_id!r}) carries " f"gen_ai.usage.* but is not an inference span; usage must be " f"attached to the parent gen_ai.chat span only" ) chat_input = sum( int(s.attributes.get("gen_ai.usage.input_tokens", 0)) for s in spans if s.name in INFERENCE_SPAN_NAMES ) if provider_prompt_tokens <= 0: raise GenAIUsagePlacementError( "provider_prompt_tokens must be positive to reconcile; got " f"{provider_prompt_tokens}" ) drift = abs(chat_input - provider_prompt_tokens) / provider_prompt_tokens if drift > tolerance_ratio: raise GenAIUsagePlacementError( "chat-span input_tokens do not reconcile with provider billing: " f"summed={chat_input}, provider={provider_prompt_tokens}, " f"drift={drift:.1%} > tolerance={tolerance_ratio:.1%}" ) ```