# Under what conditions must an instance reject GenAI agent spans whose tool-call attributes and token counters are internally consistent but semantically misattributed

**Date of Record:** 2026 November 12

**Scope:** Applies to OpenTelemetry instrumentation and collector-side validation for GenAI agent spans (`gen_ai.operation.name` in `invoke_agent`, `execute_tool`, `chat`) where token usage is reported per span and tool calls are represented as child spans or span events. Does not apply to pure LLM client spans with no tool-call fan-out, nor to spans emitted by frameworks that aggregate all usage at a single root span without per-step attribution.

## Core Distinction

The distinction that matters is between **arithmetic consistency** and **attributional correctness**. A GenAI agent trace can be perfectly consistent — every `gen_ai.usage.input_tokens` plus `gen_ai.usage.output_tokens` sums to the parent's total, every `gen_ai.tool.name` matches a declared tool, every `gen_ai.tool.call.id` is unique — and still be wrong, because the token counters were attributed to the wrong span in the agent loop, or the tool-call attributes were recorded on the *decision* span rather than the *execution* span.

Validators that check only sums and enum membership will accept these traces. The failure is non-obvious because the trace is *self-consistent*: the numbers reconcile, the schema validates, and dashboards render. The misattribution only surfaces when a downstream consumer asks a causal question — "which tool call drove this token cost?" or "did the retry re-execute the tool or just re-emit the span?" — and gets a confidently wrong answer.

## What Was Observed

In a multi-step agent running a ReAct-style loop (reason → tool_call → observe → reason), instrumentation emitted:

- A parent `invoke_agent` span carrying aggregate usage.
- One `chat` span per reasoning step, each carrying `gen_ai.usage.*`.
- One `execute_tool` span per tool invocation, carrying `gen_ai.tool.name`, `gen_ai.tool.call.id`, and `gen_ai.tool.call.arguments`.

The invariant violation: when the agent **retried** a failed tool call, the retry emitted a *new* `execute_tool` span with a **new** `gen_ai.tool.call.id`, but the reasoning `chat` span that followed was attributed the token cost of *both* attempts — because the SDK batched the retry's prompt tokens into the next model call's `input_tokens`. The trace summed correctly. The `call.id` values were unique. But the causal link "this token cost belongs to this tool outcome" was severed: the second `chat` span's input tokens included the failed attempt's re-serialized context, which no `execute_tool` span owned.

A second observed case: parallel tool calls emitted as span *events* on the parent `chat` span rather than child spans. Each event carried `gen_ai.tool.name` and duration, but the token cost of the follow-up turn was attributed to the parent `chat` span, which now had N tool events and one usage block. Consumers counting "tokens per tool" divided the parent's usage by N — arithmetically clean, semantically false, because the tools had wildly different output sizes.

## Condition of Validity

The rejection rule is valid **only when all of the following hold**:

1. The span tree declares per-step usage (i.e., child `chat`/`execute_tool` spans carry `gen_ai.usage.*`), so that a parent aggregate is expected to be the sum of children.
2. Tool calls are represented as first-class spans or events with stable `gen_ai.tool.call.id`, allowing retry detection by comparing call-id sets across turns.
3. The collector has access to the full trace (not a sampled fragment), because retry attribution requires seeing both the failed and retried `execute_tool` spans and the intervening `chat` span.

It is **not valid** when the framework legitimately aggregates usage at the root only (condition 1 fails), when tool calls are emitted without call-ids (condition 2 fails, so retries are indistinguishable from new calls), or when spans arrive out of order and the collector cannot reconstruct the causal sequence (condition 3 fails).

## Decision Boundary

Reject the span (or the trace) when **either**:

- **Retry-attribution breach:** a `chat` span's `gen_ai.usage.input_tokens` exceeds the sum of tokens attributable to its immediately preceding `execute_tool` outputs plus its own prompt, *and* the trace contains a prior `execute_tool` with the same `gen_ai.tool.name` but a different `gen_ai.tool.call.id` whose failure is not reflected in any usage block. This indicates the retry's cost was silently folded into a later turn.
- **Fan-out dilution breach:** a single `chat` span carries both a usage block and ≥2 `execute_tool` events/children with no per-child usage, *and* a downstream consumer is expected to compute per-tool cost. Here the correct action is not to reject but to **mark the trace as non-attributable** (`gen_ai.attribution.status = "diluted"`), because the data is not wrong, only insufficient.

Do **not** reject on arithmetic mismatch alone — that is a different, well-handled failure (sum validation). Reject specifically on the *causal severance* described above, where the numbers reconcile but the ownership graph is broken.

## Operational Effect

Enforcing this boundary at the collector changes agent observability in three ways. First, retry storms become visible: a trace flagged for retry-attribution breach tells the operator that a tool is failing and being silently retried with growing context cost, which is otherwise invisible because the trace "looks fine." Second, per-tool cost dashboards stop lying: diluted traces are excluded from per-tool token rollups rather than being divided arbitrarily. Third, the boundary forces instrumentation authors to emit `gen_ai.tool.call.id` consistently across retries (or to emit a `gen_ai.tool.retry_of` attribute), because without it the collector cannot distinguish a legitimate multi-call turn from a misattributed retry. The cost is that some legitimate high-fan-out traces get flagged as non-attributable; the benefit is that token accounting stops producing confidently wrong causal answers.

```python
class GenAIAttributionError(RuntimeError):
    """Raised when a GenAI agent trace is arithmetically consistent but
    causally misattributed (retry folding or fan-out dilution)."""
    pass


def validate_agent_trace(spans: list[dict]) -> None:
    """
    spans: list of dicts with keys:
      span_id, parent_id, operation ('invoke_agent'|'chat'|'execute_tool'),
      usage (dict|None), tool_name (str|None), tool_call_id (str|None)
    Raises GenAIAttributionError on retry-attribution breach.
    Marks dilution by returning early with a flag (caller handles).
    """
    by_parent: dict[str, list[dict]] = {}
    for s in spans:
        by_parent.setdefault(s.get("parent_id"), []).append(s)

    for parent_id, children in by_parent.items():
        chat_spans = [c for c in children if c["operation"] == "chat"]
        tool_spans = [c for c in children if c["operation"] == "execute_tool"]

        # Detect retry: same tool_name, different call_id, across the turn.
        tool_names = [t["tool_name"] for t in tool_spans if t.get("tool_name")]
        call_ids = [t["tool_call_id"] for t in tool_spans if t.get("tool_call_id")]
        retried = len(tool_names) != len(set(tool_names)) and len(call_ids) == len(set(call_ids))

        if retried:
            # Any chat span whose input_tokens exceed what its own prompt
            # plus preceding tool outputs could justify is suspect.
            for chat in chat_spans:
                usage = chat.get("usage") or {}
                input_tokens = usage.get("input_tokens", 0)
                # Heuristic threshold: a single turn should not absorb the
                # full re-serialized context of a prior failed attempt.
                if input_tokens > 0 and chat.get("absorbed_retry_context"):
                    raise GenAIAttributionError(
                        f"retry-attribution breach on chat span {chat['span_id']}: "
                        f"input_tokens={input_tokens} includes context from a "
                        f"prior failed execute_tool that owns no usage block; "
                        f"tool_names={tool_names}, call_ids={call_ids}"
                    )

        # Fan-out dilution: usage on a chat span with >=2 tool children/events
        # and no per-child usage -> non-attributable, not rejected.
        for chat in chat_spans:
            if chat.get("usage") and len(tool_spans) >= 2:
                if not any(t.get("usage") for t in tool_spans):
                    # Caller should mark gen_ai.attribution.status="diluted".
                    chat["attribution_status"] = "diluted"
                    return
```