# How does an instance detect token accounting drift when GenAI agent spans report tool-call latency but omit `gen_ai.usage.*` on nested tool invocations

**Date of Record:** 2026 November 9

**Scope:** Applies to OpenTelemetry-instrumented agent runtimes emitting `gen_ai.*` spans (per the GenAI semantic conventions, experimental status) where an agent span parents one or more tool-call child spans. Does not apply to single-shot LLM completion spans without tool dispatch, nor to runtimes that aggregate usage exclusively at the root trace level via a separate metrics pipeline.

## Core Distinction

The GenAI semantic conventions treat `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as attributes of the *generation* span — the span representing a single model inference round-trip. Tool-call spans (`gen_ai.operation.name = "execute_tool"`) are not generations; they carry `gen_ai.tool.name`, `gen_ai.tool.call.id`, and `gen_ai.tool.type`, but no usage attributes, because a tool execution does not itself consume model tokens. The agent span (`gen_ai.operation.name = "invoke_agent"`) is a *container*: it is expected to be the sum of its child generation spans, but the spec does not mandate that the agent span re-emit aggregated usage. The distinction that matters operationally: **an agent span with a non-zero duration and zero `gen_ai.usage.*` is not necessarily a bug — it is a bug only when its generation children exist and carry usage that the parent fails to reflect, or when no generation child exists at all for a turn that clearly invoked a model.** The non-obvious part is that the second case is silent: a tool-call span can have a valid parent, a valid `tool.call.id`, and a plausible duration while the generation span that produced the tool call was never emitted (sampler drop, exporter backpressure, or an instrumentation gap in the SDK's streaming path). The trace looks structurally complete at the agent level and structurally incomplete one level down.

## What Was Observed

In a production agent runtime (LangGraph-style state machine, OTel Python SDK 1.24, OTLP/HTTP exporter, tail-based sampling at the collector), we observed traces where:

- The `invoke_agent` span carried `gen_ai.usage.input_tokens` and `output_tokens` equal to the *first* generation turn only, not the sum across turns. The agent loop executed 3–7 model turns per invocation; the parent span's usage attributes were set once at span start from a cached counter and never updated on child completion.
- Tool-call child spans (`execute_tool`) appeared with correct `gen_ai.tool.name` and `gen_ai.tool.call.id`, but the *generation* span for the turn that decided to call the tool was absent in ~2.3% of traces. These were not sampler drops — the collector's tail sampler retained the trace because the root errored or exceeded latency threshold; the generation span was never exported by the SDK because the streaming response path returned before the span's `end()` ran under a `contextvars` reset that detached the span from the active context.
- Consequently, token accounting computed by summing `gen_ai.usage.*` over all spans in the trace undercounted by a median of 41% on affected traces, and the agent span's own attributes undercounted by a median of 68% across *all* multi-turn traces (not just the 2.3%).

The failure was invisible to span-count assertions and to "does the trace have a gen_ai span" checks. It was only visible when comparing the agent span's declared usage against the sum of its generation children, and separately when comparing tool-call count against generation count for turns that must have produced a tool call.

## Condition of Validity

The detection rule below is valid only under these conditions, and produces false positives outside them:

1. The runtime emits one generation span per model turn, with `gen_ai.operation.name = "chat"` (or the provider-specific equivalent) and usage attributes populated. Runtimes that batch multiple turns into one generation span (e.g., server-side agent loops) violate the premise.
2. Tool calls are dispatched as child spans of the generation span that requested them, or as siblings under the agent span with a trace-level ordering that lets you associate them. If tool calls are emitted asynchronously after the generation span closes, the parent-child count invariant does not hold.
3. The agent span is the trace's logical root for usage aggregation, and no separate `gen_ai.client.token.usage` histogram metric is the authoritative source. If a metrics pipeline exists and is trusted, reconcile against it rather than against span sums — the two can disagree for the same underlying reason.
4. Sampling is either head-based with a known rate or tail-based with retention that does not preferentially drop generation spans. Tail samplers that key on error status will systematically retain traces whose generation spans were lost, biasing the observed drift upward.

## Decision Boundary

The boundary between "acceptable accounting gap" and "instrumentation defect requiring remediation":

- **Gap ≤ 5% between agent-span declared usage and sum of generation children, across a rolling 1-hour window, with ≥ 100 traces:** treat as rounding and multi-turn caching effects (prompt caching attributes `gen_ai.usage.cache_read.input_tokens` are frequently omitted from the parent sum). No action.
- **Gap > 5% but every generation child is present and the discrepancy is explained by cache-read tokens:** correct the aggregation, do not page.
- **Any trace where a tool-call span exists and zero generation spans exist in the same trace:** hard defect. This is the silent case — the model must have run to produce a tool call, so a missing generation span means the span was dropped, not that no model was called. This is the boundary the naive check misses.
- **Agent span declares usage but has zero generation children, or declares usage less than any single child's usage:** impossible state, indicates the parent attribute was written from a stale or wrong counter. Escalate.
- **Tool-call spans present with `gen_ai.tool.call.id` values that do not appear in any generation span's tool-call attributes (if the runtime emits them):** the association is broken; usage attribution to the correct turn is unreliable.

The decision must be made per-trace and then aggregated, never on the aggregate alone: an aggregate 5% gap can hide a bimodal distribution of 0% and 40% traces.

## Operational Effect

When this boundary is crossed undetected, the downstream effects are concrete and compounding: cost attribution reports understate spend on multi-turn agents (the 68% median undercount above translated to a 2.9× understatement of per-invocation cost for the affected workload); latency SLOs computed per-token are inflated because the denominator is wrong; and — most damaging for agent systems — evaluation harnesses that score "tokens per successful task" will rank a broken instrumented agent above a correctly instrumented one, because the broken one appears cheaper. The remediation is not to fix the parent aggregation in isolation; it is to make the generation span's lifecycle independent of the streaming context reset (explicit `span.end()` in a `finally` block bound to the response iterator, not to the context manager), and to add the per-trace invariant check below as a continuous assertion exported as its own metric so the 2.3% silent-drop rate is observable before it becomes a 68% accounting error.

```python
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Iterable


class GenAIAccountingDriftError(RuntimeError):
    """Raised when a trace's GenAI span structure violates the usage-accounting invariant."""


@dataclass
class Span:
    span_id: str
    parent_id: str | None
    operation: str          # e.g. "invoke_agent", "chat", "execute_tool"
    input_tokens: int = 0
    output_tokens: int = 0
    tool_call_id: str | None = None
    children: list["Span"] = field(default_factory=list)


@dataclass
class Trace:
    trace_id: str
    spans: list[Span]


def _flatten(spans: Iterable[Span]) -> list[Span]:
    out: list[Span] = []
    for s in spans:
        out.append(s)
        out.extend(_flatten(s.children))
    return out


def check_trace(trace: Trace, tolerance: float = 0.05) -> dict:
    """
    Enforce the GenAI usage-accounting invariant for one trace.

    Invariants:
      I1. Every tool-call span must coexist with at least one generation span
          (a model must have run to emit a tool call).
      I2. The agent span's declared usage must be >= the sum of its generation
          children's usage, within `tolerance`.
      I3. No single generation child may exceed the agent span's declared usage.
    """
    all_spans = _flatten(trace.spans)
    gens = [s for s in all_spans if s.operation == "chat"]
    tools = [s for s in all_spans if s.operation == "execute_tool"]
    agents = [s for s in all_spans if s.operation == "invoke_agent"]

    # I1: silent-drop detection.
    if tools and not gens:
        raise GenAIAccountingDriftError(
            f"trace {trace.trace_id}: {len(tools)} tool-call span(s) but zero "
            f"generation spans; generation span was dropped, not absent"
        )

    if not agents:
        return {"trace_id": trace.trace_id, "status": "no_agent_span"}

    agent = agents[0]
    declared_in = agent.input_tokens
    declared_out = agent.output_tokens
    child_in = sum(g.input_tokens for g in gens)
    child_out = sum(g.output_tokens for g in gens)

    # I3: child cannot exceed parent.
    for g in gens:
        if g.input_tokens > declared_in or g.output_tokens > declared_out:
            raise GenAIAccountingDriftError(
                f"trace {trace.trace_id}: generation span {g.span_id} usage "
                f"({g.input_tokens}/{g.output_tokens}) exceeds agent span "
                f"declared usage ({declared_in}/{declared_out}); parent "
                f"attribute written from stale counter"
            )

    # I2: aggregate drift.
    if child_in > 0:
        drift_in = (child_in - declared_in) / child_in
        drift_out = (child_out - declared_out) / child_out if child_out else 0.0
        if drift_in > tolerance or drift_out > tolerance:
            raise GenAIAccountingDriftError(
                f"trace {trace.trace_id}: agent span undercounts children by "
                f"in={drift_in:.1%} out={drift_out:.1%} (tolerance {tolerance:.0%})"
            )

    return {
        "trace_id": trace.trace_id,
        "status": "ok",
        "generations": len(gens),
        "tool_calls": len(tools),
        "drift_in": round((child_in - declared_in) / child_in, 4) if child_in else 0.0,
    }
```