# Where must an instance of a GenAI agent span end before the parent LLM span's token accounting closes: the tool-call re-entry boundary

**Date of Record:** 2026 November 11

**Scope:** Applies to OpenTelemetry GenAI semantic conventions (`gen_ai.*`) when an agent span (`gen_ai.operation.name = "invoke_agent"`) emits tool calls (`gen_ai.operation.name = "execute_tool"`) that re-enter the model, and token usage is attributed across nested LLM spans. Does not apply to single-shot completion traces with no tool loop, nor to providers that report usage only at the top-level agent span.

## Core Distinction

The non-obvious boundary is between **span lifetime** and **accounting lifetime**. A GenAI agent span can remain open while its child `execute_tool` span completes, and the model is re-invoked inside a *new* child LLM span. The parent LLM span that triggered the tool call has already reported its `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`. If the agent span is treated as the accounting owner (a common mistake when `gen_ai.agent.name` is present), total tokens are double-counted on the agent span and lost on the intermediate LLM spans — or, worse, the tool span's re-entry is attributed to the *first* LLM span's usage because the exporter flushes on parent-close rather than child-close.

The invariant: **each `gen_ai.usage.*` attribute belongs to exactly one span whose `gen_ai.operation.name` is `chat`, and that span must close before any subsequent LLM span in the same agent loop opens with a fresh usage record.** The agent span aggregates by summation in the backend, never by attribute inheritance.

## What Was Observed

In a multi-turn tool loop (agent → LLM → tool → LLM → tool → LLM), traces exported with `BatchSpanProcessor` showed:

- The first LLM child span carried correct `input_tokens`/`output_tokens`.
- The second and third LLM child spans carried `gen_ai.usage.input_tokens = 0` when the SDK's span-scoped usage accumulator was keyed on `trace_id` rather than `span_id`, causing the second invocation to overwrite the first's counters before the first was serialized.
- The `execute_tool` spans carried no usage attributes (correct), but their `gen_ai.tool.name` and `gen_ai.tool.call.id` were present, allowing reconstruction of the loop.
- The agent span, when instrumented naively, duplicated the sum of all child LLM usage, inflating dashboard totals by 2×–3× depending on loop depth.

A second failure mode: when a tool call was cancelled mid-flight (timeout), the `execute_tool` span was marked `ERROR` but the re-entrant LLM span still opened and reported usage for a prompt that included a truncated tool result. The usage was real but the causal chain was broken; backends that join on `gen_ai.tool.call.id` lost the link because the cancelled tool span never emitted the result payload attribute.

## Condition of Validity

This boundary holds only when:

1. The instrumentation emits one LLM span per model invocation (not one per agent turn).
2. `gen_ai.usage.*` is set on the LLM span at close, not inherited from a context variable shared across the loop.
3. The agent span does **not** set `gen_ai.usage.*` unless it is explicitly a rollup with a distinct attribute namespace (e.g., `gen_ai.agent.usage.total_input_tokens`), never the canonical `gen_ai.usage.input_tokens`.
4. Tool spans complete (or are explicitly cancelled) before the next LLM span opens; if they overlap, the accounting join key must be `gen_ai.tool.call.id`, not temporal adjacency.

It does **not** hold for streaming responses where usage arrives in a final SSE chunk after the span has been ended by the transport layer — in that case usage must be recorded on a span that outlives the stream, or deferred via a span event.

## Decision Boundary

Choose between two accounting models at instrumentation time:

- **Per-invocation model (recommended):** every LLM call is its own span with its own `gen_ai.usage.*`; the agent span is a structural parent with no usage attributes. Backend sums across `chat` spans filtered by `gen_ai.agent.name`. This survives tool loops, retries, and cancellation.
- **Rollup model:** the agent span owns usage and child LLM spans omit it. This is only safe when the agent makes exactly one model call, or when the backend is guaranteed to deduplicate by `gen_ai.response.id`. In practice it fails on tool re-entry because the rollup is written before the loop terminates.

The boundary is crossed the moment a second LLM span opens under the same agent span. At that point, any context-variable-based usage accumulator is invalid and must be span-scoped.

## Operational Effect

Misattribution here produces silent 2×–3× token cost inflation in billing dashboards, breaks per-tool cost attribution (you cannot answer "which tool call was expensive"), and corrupts latency-vs-token regressions. It also defeats `gen_ai.tool.call.id`-based joins, so a cancelled tool call appears to have consumed zero tokens while its re-entrant LLM span consumes full prompt tokens — an asymmetry that looks like a provider bug but is an instrumentation lifetime bug. The fix is to scope usage to the span that closes, and to refuse to emit an agent-level rollup when loop depth exceeds one.

```python
from dataclasses import dataclass, field


class GenAIAccountingBoundaryError(RuntimeError):
    """Raised when token usage would be attributed across a span-lifetime
    boundary that violates the per-invocation accounting invariant."""


@dataclass
class SpanRecord:
    span_id: str
    operation: str  # "invoke_agent" | "chat" | "execute_tool"
    parent_id: str | None
    input_tokens: int = 0
    output_tokens: int = 0
    closed: bool = False


@dataclass
class AgentLoop:
    agent_span_id: str
    llm_spans: list[SpanRecord] = field(default_factory=list)
    tool_spans: list[SpanRecord] = field(default_factory=list)

    def open_llm_span(self, span_id: str) -> SpanRecord:
        open_llm = [s for s in self.llm_spans if not s.closed]
        if open_llm:
            raise GenAIAccountingBoundaryError(
                f"cannot open LLM span {span_id!r}: prior LLM span "
                f"{open_llm[0].span_id!r} is still open; usage would be "
                f"double-counted or lost on tool re-entry"
            )
        span = SpanRecord(span_id, "chat", self.agent_span_id)
        self.llm_spans.append(span)
        return span

    def close_with_usage(self, span_id: str, inp: int, out: int) -> None:
        match = [s for s in self.llm_spans if s.span_id == span_id]
        if not match:
            raise GenAIAccountingBoundaryError(
                f"usage reported for unknown LLM span {span_id!r}"
            )
        span = match[0]
        if span.closed:
            raise GenAIAccountingBoundaryError(
                f"span {span_id!r} already closed; usage would overwrite"
            )
        if span.operation != "chat":
            raise GenAIAccountingBoundaryError(
                f"gen_ai.usage.* may only be set on 'chat' spans, got "
                f"{span.operation!r}"
            )
        span.input_tokens = inp
        span.output_tokens = out
        span.closed = True

    def open_tool_span(self, span_id: str) -> SpanRecord:
        open_llm = [s for s in self.llm_spans if not s.closed]
        if open_llm:
            raise GenAIAccountingBoundaryError(
                f"tool span {span_id!r} opened while LLM span "
                f"{open_llm[0].span_id!r} is open; tool must follow a "
                f"closed LLM invocation"
            )
        span = SpanRecord(span_id, "execute_tool", self.agent_span_id)
        self.tool_spans.append(span)
        return span

    def agent_rollup(self) -> tuple[int, int]:
        if any(not s.closed for s in self.llm_spans):
            raise GenAIAccountingBoundaryError(
                "cannot emit agent rollup: LLM spans still open, "
                "loop has not terminated"
            )
        return (
            sum(s.input_tokens for s in self.llm_spans),
            sum(s.output_tokens for s in self.llm_spans),
        )
```