# Where must an instance terminate an agent span when a tool call is retried and token usage is double-counted under OpenTelemetry GenAI semantic conventions

**Date of Record:** 2026 November 14
**Scope:** Applies to agent runtimes that emit `gen_ai.*` spans per the OpenTelemetry GenAI semantic conventions, where a single logical tool invocation is retried (transport retry, model-initiated retry, or framework-level re-execution) and where token usage is aggregated from per-call `usage` objects. Does not apply to single-shot spans with no retry path, nor to billing reconciliation that reads provider-side invoices rather than span attributes.

## Core Distinction

The distinction that matters is between *span lifetime* and *logical operation lifetime*. An agent span (`gen_ai.operation.name = "execute_tool"` or an agent-level `invoke_agent` span) is a telemetry artifact; the tool call it describes is a logical unit that may span one or more physical attempts. The GenAI conventions define `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` as attributes of a *single model response*, not of a logical conversation turn. When a retry occurs, the naive implementation re-emits usage on each attempt *and* keeps the parent agent span open across all attempts, so the parent's aggregate usage attribute is overwritten or summed by the collector into a value that exceeds the provider's billed total. The boundary failure is not "retries are not tracked" — it is that retries are tracked twice: once as child spans and once as re-added tokens on the parent.

## What Was Observed

In a production agent that called a search tool through a client with a 3-attempt exponential backoff, the following pattern appeared in trace storage:

- One `invoke_agent` span with `gen_ai.usage.input_tokens = 41,200` for a turn whose actual prompt was ~13,700 tokens.
- Three child `execute_tool` spans, all sharing the same `gen_ai.tool.call.id`, two marked with `error.type = "timeout"`.
- The two failed attempts had `gen_ai.usage.input_tokens = 13,700` each because the runtime recorded the *model's* tool-decision call on the tool span rather than on the model span.
- The collector's span-metric processor summed parent + children into a per-trace total of 82,400 tokens against a provider bill of 27,400.

The tell was that `gen_ai.tool.call.id` was identical across three sibling spans under one parent, which is a schema-valid but semantically ambiguous state: the conventions expect one tool-call span per logical call, with retries represented as span events or as a retry counter, not as repeated sibling spans that each carry usage.

## Condition of Validity

The double-count is only a *bug* under a specific condition: when usage attributes are propagated to the parent agent span **and** retried attempts emit their own usage. If the runtime emits usage only on leaf model spans (the `gen_ai.operation.name = "chat"` spans) and leaves the agent span without usage attributes, aggregation is correct regardless of retry count. Conversely, if the runtime emits usage on the agent span but suppresses usage on retried tool attempts, aggregation is also correct. The failure requires the conjunction: parent-level usage + per-attempt usage + shared `gen_ai.tool.call.id`. This conjunction is common because frameworks often set usage at the highest span they control (the agent loop) while the HTTP client layer independently records usage per response.

## Decision Boundary

The runtime must decide, at span-close time, whether a tool-call span represents *one attempt* or *one logical call*. The rule that holds empirically:

- If `gen_ai.tool.call.id` is stable across attempts, the span must be **one logical call**; retries become `span.add_event("retry", {...})` with `attempt` and `error.type`, and usage is recorded **once**, on the final successful (or terminal) attempt only.
- If the runtime cannot guarantee a stable call id across attempts, it must emit **attempt-level spans** with a distinct `gen_ai.tool.call.id` per attempt and must **not** set usage on the parent agent span; the parent carries only `gen_ai.usage.*` for the model turns that decided to call the tool.

Mixing the two — stable id, sibling spans, parent usage — is the invariant violation. The span must be terminated (ended) at the logical-call boundary, not at the transport-attempt boundary, and the parent must not accrue usage from attempts that were retried.

## Operational Effect

When the boundary is respected, per-trace token totals reconcile with provider billing within the model-turn count, and retry latency is visible as event timestamps rather than as duplicated token mass. When it is violated, dashboards show token spend inflated by the retry multiplier (observed up to 3x under a 3-attempt policy), cost alerts fire on healthy traffic, and — more dangerously — model-quality evals that normalize by token count silently under-report cost-per-successful-tool-call, because failed attempts are counted as if they produced value. The correction is a span-lifecycle rule, not a metric rule: terminate the tool span at the logical boundary and gate usage emission on attempt success.

```python
from dataclasses import dataclass, field
from typing import Optional


class GenAiUsageInvariantError(RuntimeError):
    """Raised when agent-span token accounting violates the logical-call boundary."""


@dataclass
class ToolAttempt:
    call_id: str
    attempt: int
    input_tokens: int
    output_tokens: int
    succeeded: bool
    error_type: Optional[str] = None


@dataclass
class LogicalToolCall:
    call_id: str
    attempts: list[ToolAttempt] = field(default_factory=list)

    def record(self, attempt: ToolAttempt) -> None:
        if attempt.call_id != self.call_id:
            raise GenAiUsageInvariantError(
                f"attempt call_id {attempt.call_id!r} does not match logical call "
                f"{self.call_id!r}; cannot merge into one span"
            )
        self.attempts.append(attempt)

    def final_usage(self) -> tuple[int, int]:
        """Usage must be attributed to exactly one attempt: the terminal one."""
        if not self.attempts:
            raise GenAiUsageInvariantError(
                f"logical call {self.call_id!r} closed with zero attempts"
            )
        terminal = self.attempts[-1]
        if len(self.attempts) > 1 and any(a.succeeded for a in self.attempts[:-1]):
            raise GenAiUsageInvariantError(
                f"call {self.call_id!r} has a successful non-terminal attempt; "
                "retries must not follow success"
            )
        return terminal.input_tokens, terminal.output_tokens


def emit_agent_span_usage(
    logical_calls: list[LogicalToolCall],
    model_turn_input: int,
    model_turn_output: int,
    per_attempt_usage_emitted: bool,
) -> dict[str, int]:
    if per_attempt_usage_emitted:
        # Parent must not also accrue tool usage; doing so double-counts retries.
        return {
            "gen_ai.usage.input_tokens": model_turn_input,
            "gen_ai.usage.output_tokens": model_turn_output,
        }

    tool_in = 0
    tool_out = 0
    seen: set[str] = set()
    for call in logical_calls:
        if call.call_id in seen:
            raise GenAiUsageInvariantError(
                f"duplicate logical call id {call.call_id!r} in one agent span"
            )
        seen.add(call.call_id)
        i, o = call.final_usage()
        tool_in += i
        tool_out += o

    return {
        "gen_ai.usage.input_tokens": model_turn_input + tool_in,
        "gen_ai.usage.output_tokens": model_turn_output + tool_out,
    }
```