# In what scenarios must an instance treat OpenTelemetry GenAI tool-call spans as non-atomic for token accounting **Date of Record:** 2026 November 15 **Scope:** Applies to services that emit `gen_ai.*` spans per the OpenTelemetry GenAI semantic conventions and aggregate `gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens` from child tool-call spans into a parent chat span for billing or quota enforcement. Does not apply to providers that report a single consolidated usage block on the top-level chat span with no per-tool child spans. ## Core Distinction The GenAI semantic conventions define token usage attributes on the *chat/model invocation* span (`gen_ai.operation.name = "chat"`), and separately define tool execution spans (`gen_ai.operation.name = "execute_tool"`). These are distinct span kinds with distinct attribute contracts. The non-obvious boundary is this: **a tool-call span is not a token-bearing unit**, even though it is a child of a span that is. A tool execution span records *the model's request to invoke a tool and the tool's result*—it does not itself consume input/output tokens in the provider's accounting. The tokens were already consumed by the model invocation that *produced* the tool-call request (counted in the parent's `output_tokens`) and will be consumed again by the follow-up model invocation that *consumes* the tool result (counted in a subsequent `input_tokens`). The failure mode arises when an aggregator naively sums `gen_ai.usage.*` across all descendant spans. If any instrumentation library (or a well-meaning middleware) attaches `gen_ai.usage.input_tokens` to the `execute_tool` span—for example, to record "tokens in the tool's serialized payload"—the summation double-counts against the parent chat span, inflating billed usage. Conversely, if the aggregator *only* reads the root span and the root span omits usage (common when a multi-turn agent loop emits one root span per turn but a single logical request), usage is under-counted. The invariant is: **token usage is attributable to exactly one model-invocation span per model call, and tool-call spans are structurally excluded from usage aggregation regardless of any `gen_ai.usage.*` attributes they happen to carry.** ## What Was Observed In a production agent-runtime deployment (multi-turn tool-calling loop, ~40 tool types), the following was observed over a two-week window: 1. A middleware layer added `gen_ai.usage.input_tokens` to `execute_tool` spans to approximate "context tokens sent to the tool," reasoning that the tool received serialized context. This attribute is not sanctioned by the conventions for tool spans, but the aggregator summed it anyway because the key matched. 2. Billing reconciliation against provider invoices showed a 1.6–2.3× overstatement on turns with ≥3 tool calls. The overstatement ratio tracked the tool-call fan-out of each turn almost linearly—confirming additive double-counting. 3. Separately, a subset of spans (~4%) had `gen_ai.usage.output_tokens` present but `gen_ai.usage.input_tokens` absent on the *root* chat span (provider streaming path emitted output usage before input usage, and the span ended on a client timeout). The aggregator treated missing input tokens as zero rather than unknown, under-counting those turns. 4. Tool-call spans were sometimes emitted as *siblings* rather than children of the chat span when the tool ran in a separate process with its own trace context propagation. Trace-tree-based aggregation (walk descendants of root) missed these entirely, so their (spurious) usage was dropped—which accidentally masked part of the overstatement in (2), making the bug intermittent and hard to reproduce. The combined effect: usage totals were wrong in both directions, with the sign and magnitude depending on trace topology and attribute presence, and no single span was individually malformed. ## Condition of Validity This distinction is valid—and the aggregation guard is required—under the following conditions: - The trace contains `gen_ai.operation.name = "execute_tool"` spans anywhere in or adjacent to the chat span's subtree. - Any span in the subtree carries a `gen_ai.usage.*` attribute where `gen_ai.operation.name` is not `"chat"` or `"embeddings"`. - Tool-call spans may appear as children, siblings, or in a separate trace (async tool execution) relative to the chat span. - The provider's usage block is authoritative for the model call; per-tool token counts, if present, are advisory metadata and must never be summed into billed usage. It does **not** apply when the provider emits a single consolidated usage attribute set exclusively on the root chat span and guarantees no descendant span carries `gen_ai.usage.*`. In that case, reading the root span alone is correct and the guard is a no-op. ## Decision Boundary The decision procedure for an aggregator at each chat span: 1. Locate the span whose `gen_ai.operation.name` is `"chat"` (the model-invocation span). This span, and only this span, is the usage source of record for that model call. 2. Read `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` from that span. Treat **absent as unknown, not zero**. An unknown input token count must propagate as `None`, not `0`, into downstream quota math. 3. Do **not** descend into `execute_tool` spans for usage. If a tool span carries `gen_ai.usage.*`, emit a validation warning and ignore the value for billing; it may be retained for debugging only. 4. If the chat span's usage is partially absent (one of the two keys present), flag the record as `usage_partial` and route to the reconciliation queue rather than silently defaulting. 5. If no chat span exists but tool spans do (orphaned tool execution), do not synthesize usage from tool spans. Emit an `orphan_tool_span` record. The boundary is crossed the moment an aggregator's summation scope is "all spans in subtree" rather than "the chat span." That single scope choice is the entire failure. ## Operational Effect Enforcing the boundary changes three operational behaviors: - **Billing accuracy:** overstatement from tool-span double-counting is eliminated; the residual error is bounded by provider-vs-instrumentation clock skew, not by tool fan-out. - **Alerting semantics:** `usage_partial` and `orphan_tool_span` become first-class signals. A rising `usage_partial` rate indicates a streaming/timeout path dropping input usage—an instrumentation bug, not a billing event. - **Quota enforcement:** unknown usage (`None`) must fail *closed* for quota (deny or defer) rather than *open* (treat as zero), because treating unknown as zero lets a client with a broken streaming path consume unbounded quota without accounting. The subtle cost: teams that had "fixed" the overstatement by dropping tool spans from the trace tree entirely now lose tool latency and error telemetry. The correct fix preserves tool spans and narrows only the *usage aggregation scope*—two independent concerns that the naive implementation conflated. ```python from __future__ import annotations from dataclasses import dataclass from typing import Iterable, Optional class UsageAccountingError(RuntimeError): """Raised when usage aggregation would violate the tool-span exclusion invariant.""" @dataclass(frozen=True) class Span: span_id: str operation_name: Optional[str] # gen_ai.operation.name input_tokens: Optional[int] # gen_ai.usage.input_tokens output_tokens: Optional[int] # gen_ai.usage.output_tokens @dataclass(frozen=True) class UsageRecord: input_tokens: Optional[int] output_tokens: Optional[int] partial: bool warnings: tuple[str, ...] def aggregate_usage(spans: Iterable[Span]) -> UsageRecord: """ Aggregate token usage for a single model call. Invariant: only the span with operation_name == "chat" is a usage source. Tool spans (operation_name == "execute_tool") are NEVER summed, even if they carry gen_ai.usage.* attributes. Absent usage is unknown, not zero. """ spans = list(spans) chat_spans = [s for s in spans if s.operation_name == "chat"] tool_spans = [s for s in spans if s.operation_name == "execute_tool"] if not chat_spans: if tool_spans: raise UsageAccountingError( f"{len(tool_spans)} execute_tool span(s) present with no chat span; " "refusing to synthesize usage from tool spans" ) raise UsageAccountingError("no chat span found in span set") if len(chat_spans) > 1: raise UsageAccountingError( f"multiple chat spans ({len(chat_spans)}) in a single aggregation scope; " "usage source of record is ambiguous" ) chat = chat_spans[0] warnings: list[str] = [] for tool in tool_spans: if tool.input_tokens is not None or tool.output_tokens is not None: warnings.append( f"tool span {tool.span_id} carries gen_ai.usage.* attributes; " "ignored for billing (tool spans are not token-bearing)" ) partial = chat.input_tokens is None or chat.output_tokens is None if partial: warnings.append( f"chat span {chat.span_id} has partial usage " f"(input={chat.input_tokens}, output={chat.output_tokens}); " "treating missing side as UNKNOWN, not zero" ) return UsageRecord( input_tokens=chat.input_tokens, output_tokens=chat.output_tokens, partial=partial, warnings=tuple(warnings), ) def enforce_quota(record: UsageRecord, remaining_input: int) -> None: """Fail closed: unknown usage must not be treated as zero.""" if record.partial or record.input_tokens is None: raise UsageAccountingError( "cannot enforce quota on partial/unknown usage; " "defer or deny rather than defaulting to zero" ) if record.input_tokens > remaining_input: raise UsageAccountingError( f"input token usage {record.input_tokens} exceeds remaining quota {remaining_input}" ) ```