# Why does an instance of a GenAI agent span silently lose token attribution when a tool-call event lands after the span has been ended

**Date of Record:** 2026 November 13

**Scope:** Applies to OpenTelemetry GenAI semantic conventions (as specified in `semconv` incubating `gen_ai.*` attributes) for agent invocation spans, tool-call child spans/events, and token usage attribution. Does not apply to plain LLM inference spans with no tool-call fan-out, nor to systems that post-process spans in a batch exporter with a late-merge stage.

## Core Distinction

The GenAI semantic conventions distinguish two structurally different mechanisms for recording tool activity: **tool-call events** (log records or span events attached to the parent agent span, carrying `gen_ai.tool.name`, `gen_ai.tool.call.id`, and arguments) and **tool-call child spans** (a span with `gen_ai.operation.name = execute_tool`, parented to the agent invocation span). The conventions also require token usage (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) to be attributed to the *operation that consumed them* — typically the agent span for the model turn, and the child span for any nested model call.

The non-obvious boundary: **an event emitted after `span.end()` is dropped by every compliant SDK**, but the *token attribution* it was meant to complete is not retroactively invalidated at the point of emission — it is simply never recorded. The failure is silent because the span still exports successfully with a partial attribute set. The invariant that breaks is: *"every `gen_ai.tool.call.id` referenced in a parent agent span's token accounting must have a corresponding event or child span in the same trace."* Nothing in the SDK enforces this; the trace is structurally valid but semantically incomplete.

## What Was Observed

In a multi-step agent (ReAct-style loop) instrumented with a hand-rolled wrapper around the OTel Python SDK, the following sequence repeatedly produced orphaned token counts:

1. Agent span `S` started, model turn executed, `gen_ai.usage.input_tokens=1841`, `gen_ai.usage.output_tokens=212` set.
2. The model requested a tool call; the wrapper invoked the tool *synchronously* and recorded a `gen_ai.tool.call` event via `S.add_event(...)`.
3. The tool result was fed back into the model; a second model turn executed inside a **new** child span `C` (correctly parented to `S`).
4. The wrapper called `S.end()` after the loop terminated.
5. A streaming callback (the tool's own telemetry, arriving asynchronously from a thread pool) attempted `S.add_event(...)` for a *second* tool invocation that had actually occurred in step 2 but whose completion was reported late.

Result: `S` exported with `gen_ai.usage.*` reflecting only the first turn, a single tool event, and **no record of the second tool call**. The backend's token-attribution lineage — which joins `gen_ai.tool.call.id` on the event to the model turn that produced it — showed a model turn with tool-call tokens but no matching tool event. The trace was *not* marked as errored. No `otel.status_code` was set to `ERROR`. The span simply had fewer attributes than the operation performed.

The same class of loss occurs with span events on ended spans in the OTel specification (events added to an ended span are discarded by the `SpanProcessor`'s `on_end` path), and — more subtly — with **child spans created after the parent ended**: the child span is still recorded and exported (the SDK does not reject it), but its `parent_span_id` points at a span that the backend has already sealed, so lineage joins in the token-attribution pipeline fail silently.

## Condition of Validity

This record is valid when **all** of the following hold:

- The exporter is a standard OTLP exporter using the SDK's default `BatchSpanProcessor` or `SimpleSpanProcessor` (both call `on_end` and discard post-end mutations).
- The agent loop uses a *single* long-lived parent span for the whole conversation turn, with tool calls emitted as events rather than child spans.
- Tool execution or its telemetry callback is asynchronous relative to the span lifecycle (thread pool, `asyncio` task, streaming callback, or a `finally` block ordered after `end()`).
- Token attribution is computed by the backend by joining events/child spans to the parent by `trace_id` + `span_id`.

It does **not** hold when tool calls are modeled as child spans that are opened *before* and closed *after* the tool completes (the correct pattern), when the parent span is kept open until the exporter flushes, or when a custom `SpanProcessor` buffers and reconciles late events.

## Decision Boundary

The boundary is: **does the tool-call signal arrive before or after `span.end()`?**

- If the tool signal is emitted *before* `end()`: the event is recorded, the token lineage is complete, and the invariant holds.
- If the tool signal is emitted *after* `end()`: the event is dropped, the child span (if any) is orphaned, and the invariant is violated — but nothing raises.

The decision for the instrumenting engineer is therefore not "should I emit the event?" but "**which object owns the tool-call signal's lifetime?**" If the tool call can outlive the agent turn, the tool call must own its own span (opened at call site, closed at completion) and the parent must not be ended until all owned child spans are closed — or the parent must be a *conversation* span that outlives individual turns, with per-turn child spans carrying the token usage.

## Operational Effect

Downstream, this manifests as:

- **Undercounted cost.** Token-attribution dashboards that sum `gen_ai.usage.*` per `gen_ai.tool.call.id` show tool calls with zero cost and model turns with unattributed cost. Aggregate spend is correct only if the parent span's totals were set before `end()`; per-tool breakdowns are wrong.
- **Broken lineage joins.** Trace-graph queries that join tool events to their producing model turn return partial results. Agents querying their own telemetry for "which tool calls were expensive" get empty sets for the late-reported calls.
- **No error signal.** Because the span is structurally valid, `status_code` is `UNSET`, and no alert fires. The loss is discovered only by cross-checking against the tool's own logs.
- **Non-determinism.** The loss is timing-dependent: under low latency the event lands before `end()` and everything is fine; under load the event lands after and is dropped. The same agent produces different traces on different runs.

The mitigation is structural: model tool calls as child spans with explicit `end()` at completion, keep the agent span open until all child spans are closed (or use a conversation-scoped parent), and assert the lineage invariant in a test harness before trusting attribution.

```python
import time
from dataclasses import dataclass, field
from typing import Optional


class TokenAttributionLineageError(RuntimeError):
    """Raised when a tool-call signal is emitted after its owning span ended,
    or when a token-attribution join cannot find a matching tool event."""
    pass


@dataclass
class Span:
    name: str
    span_id: str
    parent_span_id: Optional[str] = None
    events: list = field(default_factory=list)
    attributes: dict = field(default_factory=dict)
    ended: bool = False
    end_time_ns: Optional[int] = None

    def add_event(self, name: str, attributes: dict) -> None:
        if self.ended:
            # This is the silent-loss point in the real SDK: the event is dropped.
            # We surface it instead, because dropping it breaks token lineage.
            raise TokenAttributionLineageError(
                f"event {name!r} emitted after span {self.span_id!r} ended; "
                f"tool-call token attribution for {attributes.get('gen_ai.tool.call.id')!r} "
                f"will be lost"
            )
        self.events.append({"name": name, "attributes": attributes, "time_ns": time.time_ns()})

    def end(self) -> None:
        self.ended = True
        self.end_time_ns = time.time_ns()


def attribute_tokens(agent_span: Span, tool_call_id: str, tokens: int) -> None:
    """Join a tool-call id to the agent span's token accounting.

    Fails if the tool call has no corresponding event on the (already ended) span,
    which is exactly the lineage break observed in the field.
    """
    if agent_span.ended:
        raise TokenAttributionLineageError(
            f"cannot attribute {tokens} tokens to tool call {tool_call_id!r}: "
            f"agent span {agent_span.span_id!r} already ended"
        )
    matched = any(
        e["attributes"].get("gen_ai.tool.call.id") == tool_call_id
        for e in agent_span.events
    )
    if not matched:
        raise TokenAttributionLineageError(
            f"token attribution lineage broken: tool call {tool_call_id!r} "
            f"has no event on span {agent_span.span_id!r}"
        )
    agent_span.attributes["gen_ai.usage.input_tokens"] = (
        agent_span.attributes.get("gen_ai.usage.input_tokens", 0) + tokens
    )


def run_agent_turn(agent_span: Span, tool_call_id: str, tool_latency_s: float) -> None:
    # Correct pattern: tool call is emitted and attributed BEFORE end().
    agent_span.add_event(
        "gen_ai.tool.call",
        {"gen_ai.tool.name": "search", "gen_ai.tool.call.id": tool_call_id},
    )
    attribute_tokens(agent_span, tool_call_id, tokens=1841)
    agent_span.end()

    # Simulate late async callback arriving after end().
    time.sleep(tool_latency_s)
    try:
        agent_span.add_event(
            "gen_ai.tool.call",
            {"gen_ai.tool.name": "search", "gen_ai.tool.call.id": tool_call_id + "-late"},
        )
    except TokenAttributionLineageError:
        raise  # in production this surfaces as a dropped event + broken lineage


if __name__ == "__main__":
    span = Span(name="gen_ai.agent.invoke", span_id="a1b2c3")
    run_agent_turn(span, tool_call_id="call_001", tool_latency_s=0.0)
```