# Which invariants prevent double-counting of LLM token usage when an agent retries a tool-call span under OpenTelemetry GenAI semantic conventions **Date of Record:** 2026 November 18 **Scope:** Applies to agent runtimes emitting OpenTelemetry spans that follow the GenAI semantic conventions (`gen_ai.*`) for inference and tool-execution telemetry, where the agent framework may retry, re-invoke, or re-emit a logical operation. Does not apply to single-shot, non-retrying inference clients, nor to span processors that perform server-side aggregation independent of span identity. ## Core Distinction The GenAI semantic conventions model two distinct things that look identical in a span dump: **the logical operation** (one user-visible inference or tool call) and **the emitted span** (one unit of telemetry that carries `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and `gen_ai.request.model`). The invariant that matters is not "one span per call." It is: **usage attributes are summed over spans, not over logical operations, and there is no convention-level deduplication key.** Any retry that re-emits a span with usage attributes is, by convention, a new billable observation. The boundary failure occurs when an agent framework treats a retried span as a *replacement* for the prior attempt while a downstream collector treats it as an *addition*. Both are "correct" under the spec, and the two together produce silent double-counting. The subtlety is that the GenAI conventions deliberately avoid prescribing retry semantics. `gen_ai.operation.name` distinguishes `chat`, `text_completion`, `embeddings`, and `execute_tool`, but there is no `gen_ai.retry.attempt` attribute in the stable set, and no required `span.kind` convention that would let a collector distinguish "attempt 1 of 3" from "three independent calls." Trace-level identity (`trace_id` + `span_id`) is unique per emission, so a retry *must* produce a new `span_id` — the spec forbids reusing a span ID. This is the crux: the mechanism that guarantees span uniqueness guarantees that retries are additive. ## What Was Observed In an agent runtime wiring tool calls through an instrumented HTTP client, a transient 503 from a tool endpoint triggered the framework's retry policy. The framework wrapped the tool call in a `gen_ai.operation.name=execute_tool` span. On retry it closed the first span with `status.code=ERROR` and opened a second span with the same `gen_ai.tool.name` and the same parent inference span. The first span carried no usage attributes (tool calls generally do not), but the *inference* span that followed the successful tool call was emitted once — correctly. The double-counting appeared one layer up. The agent's planning loop, on observing the tool error, re-issued the entire inference request (the model had produced a malformed tool-call argument). The second inference span carried `gen_ai.usage.input_tokens` reflecting the full re-sent context — including the tool result and the failed attempt's history. A naive dashboard summing `gen_ai.usage.input_tokens` across all `chat` spans in the trace reported roughly 2.4× the tokens actually consumed by the provider, because the retried inference re-billed the accumulated context and the first attempt's tokens were also counted. The non-obvious part: **the retried inference was legitimately billed twice by the provider.** The provider charged for the failed attempt's input tokens and the retry's (larger) input tokens. So the "2.4×" was not purely an artifact — part of it was real. The invariant failure is that the telemetry cannot distinguish *provider-billed duplicate* from *collector-side duplicate*, because both manifest as additional spans with usage attributes under the same logical operation. ## Condition of Validity The additive interpretation (sum all usage attributes across spans) is valid **only if** every emitted span with usage attributes corresponds to a distinct provider-billed request. This holds when: - The instrumentation wraps the actual SDK call and emits exactly one span per HTTP request to the provider. - Retries at the transport layer (e.g., SDK-level retry) do not emit additional spans, or emit them with an explicit attempt marker the collector is configured to handle. - The agent framework does not re-emit a span for a logical operation it already emitted. The additive interpretation is **invalid** when any retry path re-emits a span that shares a logical-operation identity with a prior span but is treated by some consumer as a correction. Concretely: if your framework's retry wrapper closes the failed span and re-opens a span with the same `gen_ai.tool.name` and parent, and your collector deduplicates on `(trace_id, gen_ai.tool.name, gen_ai.operation.name)` — a plausible but non-standard key — you will drop the retry, undercounting. If it deduplicates on nothing, you overcount only if the framework also re-emits the *inference* span without the provider having re-billed. The condition of validity, stated as an invariant: **usage attributes must be summed over spans whose emission corresponds 1:1 to provider-billed requests; any span whose emission does not correspond to a billed request must be marked (custom attribute) or excluded from the sum.** The GenAI conventions provide no built-in marker for this, so you must add one. ## Decision Boundary The boundary is between **retry-as-new-request** and **retry-as-correction**. Choose retry-as-new-request (emit a fresh span, count it) when the retry actually re-invokes the provider and the provider bills it — this is the default and is spec-compliant. Choose retry-as-correction (emit a fresh span but tag `app.retry.correction=true` and exclude from usage sums) when the retry is a local re-parse or a client-side reconstruction that did not reach the provider. The dangerous middle case — and the one that produces the boundary failure — is a framework that retries at the *agent loop* level after a tool error. The inference *is* re-billed, so it should be counted, but the framework often also retains the failed attempt's span in the trace with usage attributes populated from the streamed response before the error. That failed attempt was *also* billed (partial generation). So both spans are real, and the sum is correct — but an operator reading the trace sees two `chat` spans under one logical turn and assumes a bug. The decision boundary is therefore also an *observability* boundary: you must annotate retries so that a human (or agent) reading the trace can distinguish "two billed attempts" from "one attempt counted twice." ## Operational Effect The practical effect is that token-cost dashboards built on naive `sum(gen_ai.usage.input_tokens)` over `gen_ai.operation.name=chat` spans will over-report during retry storms — precisely when cost visibility matters most. The over-report is bounded by the retry rate and the context-growth factor per retry, which for agent loops with accumulating tool results can be superlinear. Conversely, deduplicating collectors will under-report during the same storms. Neither is detectable from the span data alone; you need an out-of-band reconciliation against provider billing, or a custom attribute (`app.retry.attempt`, `app.retry.reason`) emitted by the framework and joined at query time. The operational rule that emerged: **never sum usage attributes without first partitioning spans by an explicit attempt marker.** If the framework cannot emit one, treat the trace as a lower bound on cost and reconcile against provider invoices. The GenAI conventions will not save you here; they specify what to record, not how to aggregate, and the aggregation contract is where retries break. ```python from __future__ import annotations from dataclasses import dataclass class RetryUsageAmbiguityError(RuntimeError): """Raised when usage spans cannot be unambiguously attributed to billed requests.""" @dataclass(frozen=True) class UsageSpan: trace_id: str span_id: str operation: str # gen_ai.operation.name input_tokens: int # gen_ai.usage.input_tokens output_tokens: int # gen_ai.usage.output_tokens attempt: int | None # app.retry.attempt, None if unmarked reached_provider: bool # app.retry.reached_provider def sum_billable_tokens(spans: list[UsageSpan], require_attempt_marker: bool = True) -> int: """Sum usage only over spans that correspond to provider-billed requests. Invariant: every span with usage attributes must carry an attempt marker AND a reached_provider flag, or the sum is ambiguous and we refuse. """ total = 0 for s in spans: if s.operation != "chat": continue if require_attempt_marker and s.attempt is None: raise RetryUsageAmbiguityError( f"span {s.span_id} in trace {s.trace_id} has usage attributes " f"but no app.retry.attempt marker; cannot distinguish " f"provider-billed retry from collector-side double count" ) if not s.reached_provider: # Local re-parse or client-side reconstruction: not billed. continue total += s.input_tokens + s.output_tokens return total def reconcile_against_invoice(spans: list[UsageSpan], invoice_tokens: int, tolerance: float = 0.02) -> None: observed = sum_billable_tokens(spans) if invoice_tokens == 0: raise RetryUsageAmbiguityError("invoice baseline is zero; cannot reconcile") drift = abs(observed - invoice_tokens) / invoice_tokens if drift > tolerance: raise RetryUsageAmbiguityError( f"telemetry sum {observed} drifts {drift:.1%} from invoice {invoice_tokens}; " f"likely unmarked retry spans or deduplication at collector" ) ```