# How does an instance detect silent token-accounting drift when OpenTelemetry GenAI agent spans are emitted across retries and tool-call loops

**Date of Record:** 2026 November 13

**Scope:** Applies to OpenTelemetry GenAI semantic conventions (`gen_ai.*`) as emitted for agent spans (`gen_ai.operation.name` of `invoke_agent`, `chat`, `execute_tool`) where a single logical agent turn is retried, streamed, or decomposed into tool-call subtrees. Does not apply to plain non-agent LLM call instrumentation where each request maps 1:1 to a span with no retry or fan-out.

## Core Distinction

The GenAI conventions define token attributes (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and the newer `gen_ai.usage.input_tokens.cached` / `.reasoning`) as *per-span* observations of what the model reported for *that* API call. They are **not** cumulative counters for a logical turn, and they carry no semantics for retries, streaming partials, or tool-call fan-out. The non-obvious boundary is this: **the sum of all `gen_ai.usage.*` attributes across a span tree is not a well-defined quantity unless you also account for retries, duplicate emission on stream restart, and the distinction between `chat` child spans and `invoke_agent` parent spans.** Operators routinely build dashboards that `sum()` these attributes and get numbers that drift from the provider's billing — not because instrumentation is broken, but because the convention's per-span semantics are being read as per-turn semantics.

The second, subtler distinction: an agent span (`invoke_agent`) legitimately has *no* token usage of its own — it is a coordination span. Only its `chat` descendants carry usage. A naive `sum(gen_ai.usage.input_tokens)` over "all spans with that attribute" is correct; a `sum()` over "all spans in the agent trace" that assumes every span contributes is wrong, and worse, a `sum()` that *also* counts retried `chat` spans double-counts tokens the provider billed once (if the retry failed) or twice (if it succeeded after a partial stream). The boundary condition is the moment a retry or stream-reset occurs inside an agent turn: at that instant, per-span accounting and per-turn accounting diverge, and nothing in the convention tells you which one your consumer expects.

## What Was Observed

In a trace for a single `invoke_agent` turn that made three tool calls and one retried `chat`:

- The `invoke_agent` span had no `gen_ai.usage.*` attributes (correct per convention).
- Four `chat` child spans existed: two successful tool-selection calls, one `execute_tool` span (no usage, correct), and one `chat` span that was retried — the first attempt emitted `gen_ai.usage.input_tokens=1820` then failed with a 429, and the retry emitted `gen_ai.usage.input_tokens=1815` and succeeded.
- A dashboard summing `gen_ai.usage.input_tokens` over the trace reported 3635 input tokens for a turn the provider billed 1815 for.
- Separately, a streaming `chat` span that was resumed after a client disconnect emitted usage twice on the *same* span (the instrumentation re-set the attribute rather than adding), so the span's final value was the last chunk's cumulative figure — correct — but a downstream OTLP exporter that had already shipped the first value produced a duplicate time series point, and a `sum_over_time` query double-counted.

The failure was not an error, exception, or dropped span. Every span was present and well-formed. The drift was purely a semantic mismatch between per-span emission and per-turn aggregation, invisible to schema validation because both interpretations are valid against the convention's attribute definitions.

## Condition of Validity

The per-span interpretation is valid (and per-turn sums are safe) only when **all** of the following hold:

1. No `chat` span in the turn has a sibling with the same logical request identity (retry, hedging, or speculative decoding fan-out). If retries exist, you must deduplicate by a request-identity attribute — and the GenAI conventions do **not** standardize one; you must inject your own (e.g., a `gen_ai.request.id` or your own `app.retry.of` link) or use span links to the failed attempt.
2. Streaming usage is emitted exactly once per span, at stream close, not incrementally. If your instrumentation emits cumulative usage on each chunk, any exporter that samples or ships mid-stream will produce duplicate points.
3. `execute_tool` spans and `invoke_agent` spans are excluded from token aggregation (they have no `gen_ai.usage.*` by convention, but a misbehaving custom instrumentation may attach them).
4. The `gen_ai.usage.input_tokens.cached` and `.reasoning` sub-attributes are treated as *subsets* of `input_tokens` / `output_tokens`, not additive. Summing them alongside the parent totals double-counts.

If any condition fails, per-turn token sums are invalid and must be reconstructed from provider billing or from a deduplicated, retry-aware aggregation.

## Decision Boundary

The decision an instance must make: **when a `chat` span is retried or a stream is resumed, does the retry emit a new span (linked to the old) or reuse the span and overwrite usage?** The convention permits both, and the choice determines whether downstream aggregation must deduplicate by span identity or by request identity.

- If retries emit **new spans with a span link** to the failed attempt: aggregation must filter to the terminal (successful) span per request identity, and must not assume "one chat span = one billed call."
- If retries **reuse the span** and overwrite usage: aggregation is safe per-span, but streaming exporters can emit intermediate values that must be deduplicated by `(span_id, timestamp)` or by taking the last value per span.

An instance cannot determine which regime it is in from the trace alone unless the instrumentation declares it. The operational rule: **treat the presence of any span link between `chat` spans, or any `chat` span with more than one usage emission, as a signal that per-turn sums require deduplication**, and fail closed if you cannot establish request identity. Do not silently sum.

## Operational Effect

The practical consequence is that token-cost dashboards built on naive `sum()` over GenAI spans will over-report by the retry rate and the stream-resume rate — typically 3–15% in agent workloads with tool loops, and much higher under provider rate-limiting. Because the drift is silent and schema-valid, it is discovered only when finance reconciles against provider invoices, at which point the trace data is often past retention. The mitigation is to enforce a request-identity attribute at instrumentation time (before spans leave the process), to link retried `chat` spans rather than overwriting, and to make the aggregation layer reject any turn whose `chat` spans lack a deduplication key. The invariant to encode: *a logical agent turn's billed token count equals the sum of usage over exactly one terminal `chat` span per distinct request identity, and any trace that cannot establish that identity is unaggregatable.*

```python
from dataclasses import dataclass
from typing import Iterable, Optional

class UnaggregatableTurnError(RuntimeError):
    """Raised when a GenAI agent turn's chat spans cannot be deduplicated
    into a single billed token count per the per-span vs per-turn boundary."""
    pass

@dataclass(frozen=True)
class ChatSpan:
    span_id: str
    request_id: Optional[str]      # our injected identity; None if absent
    input_tokens: int
    output_tokens: int
    terminal: bool                 # True only for the span that actually billed
    retried_from: Optional[str]    # span_id of the failed attempt, if any

def billed_tokens_for_turn(spans: Iterable[ChatSpan]) -> dict:
    spans = list(spans)
    # Boundary check 1: every chat span must carry a request identity.
    missing = [s.span_id for s in spans if s.request_id is None]
    if missing:
        raise UnaggregatableTurnError(
            f"chat spans without request identity cannot be deduplicated: {missing}"
        )

    # Boundary check 2: exactly one terminal span per request identity.
    by_req: dict[str, list[ChatSpan]] = {}
    for s in spans:
        by_req.setdefault(s.request_id, []).append(s)

    totals = {"input_tokens": 0, "output_tokens": 0}
    for req_id, group in by_req.items():
        terminals = [s for s in group if s.terminal]
        if len(terminals) != 1:
            raise UnaggregatableTurnError(
                f"request {req_id} has {len(terminals)} terminal chat spans; "
                f"expected exactly 1 (retries must link, not duplicate)"
            )
        t = terminals[0]
        totals["input_tokens"] += t.input_tokens
        totals["output_tokens"] += t.output_tokens
    return totals

# Example: retried chat span — naive sum would yield 3635, correct is 1815.
turn = [
    ChatSpan("a1", "req-1", 1815, 40, terminal=True,  retried_from=None),
    ChatSpan("a2", "req-2", 1820, 0,  terminal=False, retried_from=None),  # 429, not billed
    ChatSpan("a3", "req-2", 1815, 55, terminal=True,  retried_from="a2"),
]
assert billed_tokens_for_turn(turn)["input_tokens"] == 1815
```