# Under what conditions must an instance treat `gen_ai.usage.input_tokens` on an agent span as non-authoritative for cost accounting **Date of Record:** 2026 November 17 **Scope:** Applies to OpenTelemetry GenAI semantic conventions (v1.27+ / experimental `gen_ai.*` namespace) emitted by agent runtimes that produce `invoke_agent`, `execute_tool`, and nested chat spans. Does not apply to single-shot chat completions without tool loops, nor to provider-native billing exports used as a source of truth. ## Core Distinction The GenAI conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as *per-span* counters describing the token volume attributable to that span's own model interaction. They are **not** defined as additive across a span tree, and they are **not** defined as exclusive of prompt-cache reads or of re-transmitted conversation history. For a single LLM call this is unambiguous. For an agent span that fans out into N tool-call spans and M nested chat spans, the attribute on the parent `invoke_agent` span is a *summary of the model interaction that the agent runtime itself performed* — typically the final synthesis call — and it does **not** include the input tokens consumed by the nested chat spans that the agent orchestrated. The boundary condition is this: any aggregator that sums `gen_ai.usage.input_tokens` over all spans sharing a `trace_id` double-counts on some runtimes and under-counts on others, and there is no attribute in the convention that disambiguates which behavior a given emitter chose. The distinction that matters operationally: **token attributes are observability signals, not billing ledgers.** They are sampled, they are subject to span-dropping under `ParentBased(TraceIdRatioBased)`, and they are emitted by the runtime author's choice, not by the model provider. ## What Was Observed On a production agent trace (LangGraph-style orchestrator emitting OTel GenAI spans, provider = a major hosted API), a single user turn produced the following span tree: - `invoke_agent` (root): `gen_ai.usage.input_tokens = 1840`, `gen_ai.usage.output_tokens = 212` - `chat` (child, planning): `input_tokens = 41200`, `output_tokens = 640` - `execute_tool` × 6 (children): no usage attributes at all - `chat` (child, per-tool synthesis) × 6: `input_tokens` ranged 38000–44000 each - `chat` (child, final): `input_tokens = 52100`, `output_tokens = 388` The provider's billing dashboard for the same window reported ~410k input tokens for that turn. The sum of `gen_ai.usage.input_tokens` across the trace was ~330k. The root span alone reported 1.8k. Three plausible aggregations of the *same trace* disagreed by more than an order of magnitude, and none matched billing. Two mechanisms explained the gap. First, the orchestrator re-sent the full conversation history on every nested chat call (no prompt caching), so each nested span's `input_tokens` legitimately included prior turns — summing them is correct only if you intend to measure *provider-side billed input*, and incorrect if you intend to measure *unique context size*. Second, the root `invoke_agent` span's usage attributes reflected only the final synthesis call, because the runtime emitted the child chat spans with their own usage and did not roll up. The convention permits both; it does not mandate either. A second trace on a different runtime (a vendor SDK) showed the opposite: the root `invoke_agent` span carried the *rolled-up* total, and child `chat` spans carried per-call usage. Summing that tree double-counted by roughly 2×. ## Condition of Validity The following invariants hold, and only these: 1. `gen_ai.usage.input_tokens` on a `chat` span is valid as the input token count *for that single model call*, provided the span was not dropped and the emitter is not a caching proxy that reports cache hits as zero. 2. `gen_ai.usage.input_tokens` on an `invoke_agent` span is valid **only** as "tokens attributable to the agent's own direct model call," and **only** if the runtime documents that it does not roll up. There is no attribute distinguishing rolled-up from non-rolled-up; you must know your emitter. 3. Summing usage across a trace is valid **only** if every span in the tree is a leaf `chat` span with no `invoke_agent` ancestor also reporting usage, and only if the runtime does not re-send history. Neither is guaranteed by the convention. 4. `execute_tool` spans carry no usage attributes by convention; any tool that internally calls an LLM must emit its own `chat` child span, and if it does not, its tokens are invisible to trace-based accounting entirely. 5. Token attributes are subject to sampling. A `TraceIdRatioBased(0.1)` sampler makes trace-derived totals a 10% sample; extrapolating them to a billing figure is a category error unless the sampler is `AlwaysOn` for the agent service. ## Decision Boundary Treat trace-derived token sums as **authoritative for cost** if and only if *all* of the following hold: the agent service uses `AlwaysOn` sampling; the emitter is documented as non-rolling-up; every LLM call (including inside tools) is wrapped in a `chat` span; and prompt caching is either disabled or the emitter reports cache-read tokens in a separate attribute. If any condition fails, downgrade trace token sums to **directional only** and reconcile against provider billing exports. Treat trace token sums as **authoritative for context-window pressure** (a different question) only if you compute the maximum single-span `input_tokens` along any root-to-leaf path, never the sum. Context pressure is a max, not a sum; conflating them is the most common misread. ## Operational Effect The practical consequence is that a cost dashboard built on `sum(gen_ai.usage.input_tokens) by trace_id` will silently misreport by 2× on roll-up emitters and by 0.5× on fan-out emitters, and the error is invisible because the numbers are plausible. The correct posture is to tag each emitter with a known roll-up policy at instrumentation time, compute cost from leaf `chat` spans only, and run a nightly reconciliation delta against the provider bill with an alert threshold. When the delta exceeds 15%, the emitter's policy has drifted or a tool is calling an LLM without a span, and the trace pipeline is no longer a valid cost signal. The invariant to encode: *a token attribute is a claim by the emitter about one span, never a fact about a trace.* ```python from dataclasses import dataclass class TokenAccountingInvariantError(RuntimeError): """Raised when a trace's token attributes cannot be trusted for cost.""" @dataclass(frozen=True) class EmitterPolicy: rolls_up_usage: bool # invoke_agent span includes child chat usage always_on_sampling: bool wraps_tool_llm_calls: bool # every in-tool LLM call emits a chat span reports_cache_reads_separately: bool def cost_authoritative_tokens( spans: list[dict], policy: EmitterPolicy, *, provider_billed_tokens: int | None = None, tolerance: float = 0.15, ) -> int: """Return leaf-chat token sum, or raise if the trace cannot support cost use.""" if not policy.always_on_sampling: raise TokenAccountingInvariantError( "sampled trace: token sums are a sample, not a ledger" ) if not policy.wraps_tool_llm_calls: raise TokenAccountingInvariantError( "tool-internal LLM calls lack chat spans; tokens are invisible" ) if not policy.reports_cache_reads_separately: raise TokenAccountingInvariantError( "cache reads not separated; input_tokens understates billed input" ) leaf_chat = [ s for s in spans if s.get("name") == "chat" and s.get("parent_kind") != "chat" ] if policy.rolls_up_usage: # Root already includes children; summing would double-count. roots = [s for s in spans if s.get("name") == "invoke_agent"] if len(roots) != 1: raise TokenAccountingInvariantError( f"roll-up emitter but {len(roots)} invoke_agent roots" ) total = roots[0]["gen_ai.usage.input_tokens"] else: total = sum(s["gen_ai.usage.input_tokens"] for s in leaf_chat) if provider_billed_tokens is not None: delta = abs(total - provider_billed_tokens) / max(provider_billed_tokens, 1) if delta > tolerance: raise TokenAccountingInvariantError( f"trace total {total} vs billed {provider_billed_tokens} " f"delta {delta:.0%} exceeds {tolerance:.0%}; emitter policy drifted" ) return total ```