# Under what conditions must an instance treat OpenTelemetry GenAI token accounting as non-authoritative across agent spans and tool calls **Date of Record:** 2026 November 17 **Scope:** Applies to agent runtimes emitting `gen_ai.*` span attributes per the OpenTelemetry GenAI semantic conventions where a single logical agent invocation fans out into nested `execute_tool` spans, retries, or provider-side streaming. Does not apply to single-shot completion calls with no tool loop, nor to billing reconciliation against provider invoices, which must always use the provider's own usage records. ## Core Distinction The GenAI semantic conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as attributes on a span, and they are conventionally summed by downstream backends across a trace to produce a "total cost" figure. The non-obvious boundary is this: **token usage attributes are per-provider-call observations, not per-agent-invocation allocations, and they are not guaranteed disjoint across the spans of a single trace.** An agent loop that calls a model, receives a tool-call request, executes the tool, and re-invokes the model with the appended tool result will — on most providers — re-bill the *entire* accumulated conversation (system prompt, prior turns, prior tool results) as `input_tokens` on every iteration. Summing `input_tokens` across the child spans of one agent trace therefore double-counts the prefix once per loop turn. The attributes are individually correct and collectively non-additive. The distinction that matters operationally: a *span-local* usage value is authoritative for that span; a *trace-aggregated* usage value is a derived quantity whose validity depends on the loop topology, which the conventions do not encode. ## What Was Observed During a multi-turn tool-using agent trace (three model invocations interleaved with two `execute_tool` spans), the following pattern appeared in exported spans: - Turn 1 model span: `input_tokens=1_840`, `output_tokens=120` (system prompt + user turn; output contains a tool-call block). - `execute_tool` span: no usage attributes (correct — tools do not consume model tokens, but see below). - Turn 2 model span: `input_tokens=2_010`, `output_tokens=95`. The 2,010 is the 1,840 prefix *plus* the 120 prior output *plus* the tool result — the prefix was re-sent and re-billed. - Turn 3 model span: `input_tokens=2_190`, `output_tokens=210`. A naive trace-level sum yields `input_tokens=6_040`. The provider's actual billed input was 2,190 (the final, largest context) if the provider caches the prefix, or 6,040 if it does not. Both numbers are "correct" under different provider billing models, and the trace cannot distinguish them. Separately, the `execute_tool` spans carried no usage, yet a downstream dashboard that summed only model spans and divided by wall-clock produced a throughput figure that silently excluded tool latency, because the tool spans were siblings rather than children of the model spans in that instrumentation. A second observation: retried model calls (transient 429s) emitted *both* the failed attempt span and the successful retry span, each carrying usage attributes. The failed attempt consumed input tokens at the provider but produced no output; the retry re-sent the same prefix. Summing across retries inflates input by the retry count while output remains single-counted — an asymmetry that corrupts any derived input/output ratio. ## Condition of Validity Trace-aggregated token accounting is authoritative **only** when all of the following hold: 1. The trace contains exactly one model-invocation span per logical turn (no retries, no speculative/parallel sampling, no fallback-to-secondary-provider spans). 2. The provider bills statelessly per call with no prefix caching, *or* the instrumentation emits cache-read/cache-write attributes so the cached portion can be subtracted. 3. No span in the trace is a "wrapper" span that re-reports a child's usage (some auto-instrumentation libraries stamp parent spans with aggregated child usage, which double-counts against the children). 4. Tool spans are confirmed to carry zero usage, and any span carrying usage is confirmed to correspond to exactly one billable provider request. When any condition fails, the aggregated value is a *lower or upper bound at unknown offset*, not a measurement. It remains useful for ordering and rough magnitude; it must not feed cost attribution, quota enforcement, or per-tenant chargeback without correction. ## Decision Boundary The decision the runtime must make: **does this trace's aggregated usage cross a threshold that triggers a consequential action (quota cutoff, billing record, autoscaling signal, SLA report)?** - If the action is *observational* (dashboards, debugging, trend lines), aggregate freely and label the metric `gen_ai.usage.aggregate_naive` so its non-additivity is explicit. - If the action is *consequential*, the runtime must either (a) reconstruct billable usage from the provider's response usage object captured on the terminal span of each turn, tagging retries and cache hits explicitly, or (b) refuse to emit an aggregate and defer to the provider's billing export. - If the runtime cannot determine loop topology (e.g., it is a collector receiving spans from an uninstrumented SDK), it must treat the aggregate as non-authoritative and mark the trace `usage_attribution=unverified`. The boundary is crossed the moment a summed `gen_ai.usage.*` value is compared against a hard limit or written to a financial ledger. At that point the runtime must have proven conditions 1–4 or it is silently wrong. ## Operational Effect Runtimes that skip this check exhibit three failure signatures: (1) cost dashboards that overstate spend by a factor roughly equal to average loop depth, causing premature quota exhaustion for heavy tool users; (2) input/output ratios that drift upward under retry storms, misattributing provider throttling to "verbose prompts"; and (3) chargeback disputes where the trace-derived figure exceeds the invoice and cannot be reconciled because the double-counted prefix is not separable post hoc. The corrective posture is to treat span-local usage as ground truth, treat trace-level sums as a derived estimate requiring a topology proof, and to emit an explicit attribution-confidence attribute alongside any aggregate so downstream consumers can gate consequential actions on it. ```python from __future__ import annotations from dataclasses import dataclass, field class NonAuthoritativeUsageError(RuntimeError): """Raised when aggregated GenAI token usage is used for a consequential action without a proof that the trace topology makes the aggregate additive.""" @dataclass class SpanUsage: span_id: str kind: str # "model" | "tool" | "wrapper" input_tokens: int = 0 output_tokens: int = 0 is_retry: bool = False is_wrapper_reaggregation: bool = False cache_read_tokens: int | None = None @dataclass class TraceUsage: spans: list[SpanUsage] = field(default_factory=list) def aggregate_naive(self) -> tuple[int, int]: return ( sum(s.input_tokens for s in self.spans), sum(s.output_tokens for s in self.spans), ) def assert_authoritative_for_billing(self) -> None: """Guard consequential use of the aggregate. Raises unless conditions hold.""" reasons: list[str] = [] if any(s.is_retry for s in self.spans): reasons.append("retry spans present: input re-billed, output single-counted") if any(s.is_wrapper_reaggregation for s in self.spans): reasons.append("wrapper span re-reports child usage: double-count risk") model_spans = [s for s in self.spans if s.kind == "model"] if len(model_spans) > 1 and not all( s.cache_read_tokens is not None for s in model_spans ): reasons.append( "multi-turn loop without cache-read attributes: prefix re-billing " "cannot be separated from genuine growth" ) if any(s.kind == "tool" and (s.input_tokens or s.output_tokens) for s in self.spans): reasons.append("tool span carries model token usage: attribution is ambiguous") if reasons: raise NonAuthoritativeUsageError( "trace-aggregated gen_ai.usage is non-authoritative for billing: " + "; ".join(reasons) ) def record_chargeback(trace: TraceUsage, tenant: str) -> dict[str, int]: trace.assert_authoritative_for_billing() inp, out = trace.aggregate_naive() return {"tenant": tenant, "input_tokens": inp, "output_tokens": out} ```