# Where must an instance of `gen_ai.tool.call` end: the span-closure invariant that OpenTelemetry GenAI semantic conventions leave underspecified for agent tool loops

**Date of Record:** 2026 November 13

**Scope:** This record applies to instrumentation of autonomous agent runtimes that emit OpenTelemetry GenAI spans (agent invocation, LLM inference, tool execution) under the experimental GenAI semantic conventions, where tool calls are dispatched asynchronously or retried. It does not apply to single-shot synchronous LLM completion spans, nor to systems that emit only metrics without span-level tool attribution.

## Core Distinction

The GenAI semantic conventions define *attributes* for tool calls (`gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.operation.name = "execute_tool"`) and for token accounting (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) but they do not define a *closure contract*: the rule that a `execute_tool` span's `end` timestamp must be emitted before the enclosing agent span's `end`, and that the token usage recorded on the agent span must equal the sum of usage recorded on its child inference spans. The distinction that matters operationally is between **span nesting** (parent/child trace topology, which the SDK enforces via context propagation) and **span lifetime containment** (temporal ordering of `end` events, which nothing enforces). A tool span can be a structural child of the agent span while ending *after* the agent span has already ended. Downstream consumers — token-billing aggregators, latency SLO evaluators, trace-completeness checkers — assume containment and silently mis-account when nesting and lifetime diverge.

## What Was Observed

In a runtime where the agent loop dispatched tool calls to a bounded worker pool, the agent span was ended by the loop controller as soon as the model returned a terminal message, while a retried tool call was still in flight. The resulting trace was structurally valid: the tool span carried the correct `parent_span_id`. But its `end_time` was 340–1800 ms *after* the agent span's `end_time`. Three consequences were observed:

1. **Token double-counting.** The billing aggregator, which summed `gen_ai.usage.input_tokens` across all spans in the trace, counted the retried inference span (emitted by the tool's internal sub-agent) twice because the retry re-used the same `gen_ai.tool.call.id`. The agent span's own usage attribute did not include the retry, so the trace total exceeded the agent total by exactly the retried call's tokens.
2. **Negative tool latency.** Latency evaluators computed `tool.end - tool.start` correctly, but a trace-level "agent duration" computed as `max(child.end) - root.start` exceeded the root span's own duration, producing a negative slack value in the SLO report.
3. **Orphaned completion.** When the process was drained for deploy, the in-flight tool span ended 40 ms after the exporter's final flush; its `end` event was dropped, leaving a span with a start but no end. The collector's completeness check flagged the trace as truncated even though the agent span itself was complete.

The non-obvious part: none of these are detectable from span *attributes* alone. They require comparing `end_time` of parent and child, and comparing `gen_ai.tool.call.id` uniqueness across the trace.

## Condition of Validity

The containment invariant holds only when all of the following are true, and the failure mode is *latent* (not active) when any is false:

- Tool dispatch is synchronous with respect to the agent loop's span lifecycle, **or** the loop controller awaits all child tool spans before ending the agent span.
- `gen_ai.tool.call.id` is unique per *attempt*, not per logical call. If retries reuse the call ID, the aggregation invariant (`sum(child usage) == parent usage`) is unenforceable regardless of span timing.
- The span processor/exporter flushes after the last child span ends, not after the root span ends.

When tool calls are async and the agent span ends optimistically (the common "return as soon as the model stops emitting" pattern), the invariant is violated by construction. This is the default behavior of most agent frameworks that stream model output and dispatch tools in the same turn.

## Decision Boundary

The operator must choose one of two contracts and enforce it explicitly:

- **Optimistic closure** (agent span ends at model terminal message): acceptable only if tool spans are *not* modeled as children of the agent span but as siblings linked via `links` with `SpanLink` to the agent span, and if token accounting is done from a separate usage event stream rather than by summing span attributes. This preserves trace topology but abandons containment-based accounting.
- **Containment closure** (agent span ends only after all tool spans end): required if any downstream consumer sums child usage into parent, computes trace duration from root, or asserts trace completeness. This requires the loop controller to hold a reference count of in-flight tool spans and to end the agent span from a `finally` that awaits the count reaching zero.

The boundary is crossed the moment a retry reuses `gen_ai.tool.call.id` or an async tool outlives the agent span. At that point, optimistic closure is no longer safe even if topology looks correct.

## Operational Effect

Enforcing containment closure adds bounded latency to agent span emission (the agent span cannot end until the slowest tool retry settles), which delays trace visibility for the root span by the retry backoff tail. The trade is that token totals become reconcilable by simple summation and trace-duration SLOs stop producing negative slack. The instrumentation should assert the invariant at span-end time and raise rather than emit a trace that violates it, because a silently mis-accounted trace is worse than a rejected one: it propagates into billing and SLO dashboards where the error is amortized and invisible.

```python
from __future__ import annotations

import time
from dataclasses import dataclass, field


class GenAiSpanClosureError(RuntimeError):
    """Raised when a GenAI agent span violates lifetime containment or usage reconciliation."""


@dataclass
class SpanRecord:
    span_id: str
    parent_span_id: str | None
    operation: str
    start_time: float
    end_time: float | None = None
    tool_call_id: str | None = None
    input_tokens: int = 0
    output_tokens: int = 0


@dataclass
class AgentSpanGuard:
    agent: SpanRecord
    children: list[SpanRecord] = field(default_factory=list)

    def add_child(self, child: SpanRecord) -> None:
        if child.parent_span_id != self.agent.span_id:
            raise GenAiSpanClosureError(
                f"child {child.span_id} is not parented to agent {self.agent.span_id}"
            )
        self.children.append(child)

    def close_agent(self, now: float | None = None) -> None:
        now = time.time() if now is None else now
        self.agent.end_time = now

        # Invariant 1: lifetime containment — every child must end before the agent.
        for child in self.children:
            if child.end_time is None:
                raise GenAiSpanClosureError(
                    f"agent {self.agent.span_id} closed with unfinished child "
                    f"{child.span_id} ({child.operation}); tool span outlived parent"
                )
            if child.end_time > self.agent.end_time:
                raise GenAiSpanClosureError(
                    f"child {child.span_id} end_time {child.end_time:.3f} exceeds "
                    f"agent end_time {self.agent.end_time:.3f}; "
                    f"containment violated by {child.end_time - self.agent.end_time:.3f}s"
                )

        # Invariant 2: usage reconciliation — parent usage must cover child usage.
        child_input = sum(c.input_tokens for c in self.children)
        child_output = sum(c.output_tokens for c in self.children)
        if child_input > self.agent.input_tokens:
            raise GenAiSpanClosureError(
                f"agent input_tokens {self.agent.input_tokens} < child sum {child_input}; "
                f"retry likely reused gen_ai.tool.call.id"
            )
        if child_output > self.agent.output_tokens:
            raise GenAiSpanClosureError(
                f"agent output_tokens {self.agent.output_tokens} < child sum {child_output}"
            )

        # Invariant 3: tool call id uniqueness per attempt.
        seen: set[str] = set()
        for child in self.children:
            if child.tool_call_id is None:
                continue
            if child.tool_call_id in seen:
                raise GenAiSpanClosureError(
                    f"duplicate gen_ai.tool.call.id {child.tool_call_id!r} in trace; "
                    f"retries must mint a new call id"
                )
            seen.add(child.tool_call_id)
```