# Which invariants prevent silent token-accounting corruption in OpenTelemetry GenAI agent spans when tool calls re-enter the model loop

**Date of Record:** 2026 November 13

**Scope:** Applies to OpenTelemetry GenAI semantic conventions (`gen_ai.*`) as emitted by agent runtimes that interleave LLM invocations with tool executions inside a single logical agent span. Does not apply to single-shot chat completions with no tool round-trips, nor to vendor-specific proprietary tracing that does not claim OTel GenAI conformance.

## Core Distinction

The GenAI semantic conventions distinguish between an **agent span** (a logical unit of autonomous work, typically `invoke_agent`) and the **model-invocation spans** (`chat` / `text_completion`) nested beneath it. Token usage attributes (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) are defined on the *model invocation*, not on the agent. The agent span is a container. This is the distinction that gets violated in practice: teams frequently aggregate token counts onto the agent span and then *also* leave the per-invocation counts in place, producing double-counting when a backend sums descendants, or producing an undercount when a backend reads only the agent span and the runtime forgot to populate it on re-entry.

The non-obvious invariant is this: **within one agent span, the multiset of `chat` child spans must be a partition of the agent's total token consumption, and each child's `gen_ai.usage.*` must reflect exactly one provider round-trip.** A tool call does not consume tokens; it suspends the loop and the next model call is a *new* child span. An agent that loops N times over tools must emit N+1 `chat` spans (initial plan + one continuation per tool result batch), never one span with mutated counters.

## What Was Observed

A production agent runtime emitted a single `chat` span per user turn and mutated `gen_ai.usage.output_tokens` in place as each tool result triggered a follow-up completion, because the instrumentation wrapped the whole ReAct loop in one span. Two consequences surfaced downstream:

1. **Backends that deduplicate by `span_id`** (most OTLP collectors with `deltatocumulative` or any span-level upsert) kept only the final mutated value. Token cost was underreported by the sum of all intermediate completions — in one traced session, 4 tool round-trips collapsed from 11,204 output tokens to 2,891.
2. **Backends that treat each emitted span record as immutable** (streaming exporters, some Jaeger configurations) saw the same `span_id` emitted 5 times with different usage values and counted all of them, overreporting by roughly 3.9×.

The failure is silent in both directions. No attribute is malformed; `gen_ai.usage.output_tokens` is a valid integer throughout. The corruption lives in the *cardinality relationship* between span identity and provider round-trip, which the semantic conventions assume but do not enforce at the wire level.

A second, subtler observation: tool-call attributes (`gen_ai.tool.name`, `gen_ai.tool.call.id`) were attached to the *parent* `chat` span rather than to a dedicated tool-execution span or `gen_ai.tool.call` event. When the same tool was called twice in one loop, the second `gen_ai.tool.call.id` overwrote the first (last-write-wins on a single-valued attribute), severing the join key between the model's `tool_calls` response and the tool result that was fed back. Trace consumers could no longer reconstruct which result satisfied which call.

## Condition of Validity

The partition invariant holds only when all of the following are true:

- Each provider round-trip (one request/response pair to the model API) maps to exactly one span with a stable `span_id` assigned at span start and never reused.
- Tool executions are represented either as child spans or as `gen_ai.tool.call` events, each carrying a unique `gen_ai.tool.call.id` that matches the `id` returned in the model's tool-call payload.
- The agent span carries **no** `gen_ai.usage.*` attributes; if a rolled-up total is desired, it is a derived value computed by the backend over child spans, not an emitted attribute.
- Streaming responses are accumulated to a single final usage record before the span closes; partial usage is never emitted on a span that will later be updated.

When any of these is violated — particularly the in-place mutation of a span that has already been exported — the invariant is undefined and downstream aggregation is untrustworthy regardless of how correct the individual attribute values look.

## Decision Boundary

Instrumentors must choose one of two consistent postures and must not mix them:

- **Per-round-trip spans (conformant):** emit one `chat` span per model call, one tool span/event per tool execution, no usage attributes on the agent span. Backends sum descendants. This is the only posture that survives both deduplicating and append-only collectors.
- **Single aggregated span (non-conformant but sometimes unavoidable):** if the runtime genuinely cannot emit per-call spans (e.g., a synchronous SDK with no span lifecycle hooks), then usage must be accumulated in local variables and written **once** at span end, and the span must be marked with a non-standard attribute such as `gen_ai.aggregation.mode=summed` so consumers know not to sum descendants. Emitting both per-call and aggregate is the one forbidden combination.

The boundary is crossed the moment instrumentation mutates an already-exported span's usage counters. There is no safe amount of this.

## Operational Effect

Conformant instrumentation yields token totals that are stable under collector re-processing, replay, and sampling changes — because each span is immutable and the sum is associative. Non-conformant instrumentation yields totals that drift with collector configuration, which means cost dashboards, budget alarms, and rate-limit accounting all become configuration-dependent rather than workload-dependent. The practical symptom is a discrepancy between the provider's billing export and the trace-derived token count that changes when you upgrade the collector but not when you change the workload. When you see that, check span identity versus round-trip cardinality before checking attribute values.

```python
from __future__ import annotations

from dataclasses import dataclass, field


class TokenAccountingInvariantError(RuntimeError):
    """Raised when a GenAI agent span violates the round-trip partition invariant."""


@dataclass
class RoundTrip:
    span_id: str
    input_tokens: int
    output_tokens: int
    tool_call_ids: list[str] = field(default_factory=list)


@dataclass
class AgentSpan:
    span_id: str
    round_trips: list[RoundTrip] = field(default_factory=list)
    emitted_usage_on_agent: bool = False

    def record_round_trip(self, rt: RoundTrip) -> None:
        if any(existing.span_id == rt.span_id for existing in self.round_trips):
            raise TokenAccountingInvariantError(
                f"span_id {rt.span_id!r} reused across round-trips; "
                "each provider call must get a fresh span_id"
            )
        if rt.input_tokens < 0 or rt.output_tokens < 0:
            raise TokenAccountingInvariantError(
                f"negative usage on span {rt.span_id!r}: "
                f"in={rt.input_tokens} out={rt.output_tokens}"
            )
        self.round_trips.append(rt)

    def assert_conformant(self) -> None:
        if self.emitted_usage_on_agent:
            raise TokenAccountingInvariantError(
                "agent span carries gen_ai.usage.* attributes; usage must live "
                "only on child chat spans to keep descendant-summing correct"
            )
        seen_tool_ids: set[str] = set()
        for rt in self.round_trips:
            for call_id in rt.tool_call_ids:
                if call_id in seen_tool_ids:
                    raise TokenAccountingInvariantError(
                        f"duplicate gen_ai.tool.call.id {call_id!r}; "
                        "tool-call ids must be unique within an agent span"
                    )
                seen_tool_ids.add(call_id)

    def total_tokens(self) -> tuple[int, int]:
        self.assert_conformant()
        return (
            sum(rt.input_tokens for rt in self.round_trips),
            sum(rt.output_tokens for rt in self.round_trips),
        )


def reconcile(agent: AgentSpan, provider_reported_output: int) -> None:
    agent.assert_conformant()
    _, traced_out = agent.total_tokens()
    if traced_out != provider_reported_output:
        raise TokenAccountingInvariantError(
            f"trace-derived output tokens ({traced_out}) != provider report "
            f"({provider_reported_output}); likely a collapsed or double-counted "
            "round-trip in the agent span"
        )
```