# Under what conditions must an instance refuse to emit a GenAI span when token accounting, tool-call attribution, and model provenance cannot be reconciled at span close

**Date of Record:** 2026 November 10

**Scope:** Applies to services instrumenting LLM inference and tool-calling loops under the OpenTelemetry GenAI semantic conventions (`gen_ai.*` attributes, `gen_ai.client.token.usage` metrics, and span events for tool invocations), where spans are closed after streaming completion or after a multi-step agent loop. Does not apply to batch offline evaluation pipelines that never emit spans, nor to pure metric-only instrumentation that emits no per-request span.

## Core Distinction

The OpenTelemetry GenAI conventions treat three things as if they were independently observable at span close: (1) token counts (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and cached/reasoning subcounts), (2) tool-call attribution (which span event corresponds to which `gen_ai.tool.name` and its arguments/results), and (3) model provenance (`gen_ai.request.model`, `gen_ai.response.model`, `gen_ai.provider.name`, and system fingerprint). The non-obvious boundary is that these three are *not* independently observable in practice. They are coupled through the provider's response envelope, and in streaming and multi-tool scenarios the envelope arrives in fragments whose reconciliation is only possible if the span is held open past the point where most SDKs auto-close it.

The distinction that matters operationally is between **a span whose attributes are merely incomplete** and **a span whose attributes are mutually inconsistent**. Incomplete spans are acceptable under the conventions (attributes are optional). Inconsistent spans are not: they poison downstream aggregation because a token count attributed to the wrong model, or a tool call attributed to the wrong span, is consumed by cost dashboards and eval harnesses as ground truth. The failure mode is silent — nothing throws, the span exports successfully, and the inconsistency is only visible when someone joins `gen_ai.usage.*` against `gen_ai.response.model` across a fleet.

## What Was Observed

Across a set of instrumented agent services (streaming chat completions plus a tool-execution loop), the following recurring reconciliation failures were observed at span close:

- **Streaming model drift.** `gen_ai.request.model` is set at span start from the client request. The authoritative `gen_ai.response.model` (and system fingerprint) only arrives in the final stream chunk. When the stream is truncated by client disconnect or a deadline, the final chunk never arrives, yet the span closes with request-model provenance and a partially accumulated token count. The span now asserts provenance it cannot substantiate.
- **Token count split across chunks.** Usage is reported either in the terminal chunk (OpenAI-style) or incrementally per chunk (some providers). If the span closes on the first `finish_reason`, the incremental counts are undercounted; if it closes on stream end without draining, terminal-chunk usage is dropped entirely. The two paths produce different `gen_ai.usage.output_tokens` for identical requests, and neither is flagged.
- **Tool-call attribution under parallel calls.** With parallel tool calls, the model emits multiple `tool_calls` entries in one assistant turn, each with an `id`, and results return as separate messages. The convention expects each tool invocation to be a span event or child span linked by that `id`. When a tool call is retried (idempotency retry after a 429 from the tool backend) the retry reuses or regenerates the `id` depending on the SDK, and the result message may match zero or two pending calls. The span exports with an orphaned `gen_ai.tool.name` event and no result, or with a result attributed to the wrong call.
- **Cached vs. reasoning token subcounts.** `gen_ai.usage.cache_read.input_tokens` and reasoning-token subcounts are only present when the provider returns them; summing them into `input_tokens` or `output_tokens` double-counts, while omitting them undercounts cost. Different services in the same fleet made opposite choices, so fleet-wide cost-per-token was not comparable.

The common thread: every one of these is a case where the span closes with attributes that are individually plausible but jointly contradictory, and the SDK has no mechanism to detect the contradiction because it does not hold the full response envelope at close time.

## Condition of Validity

The refusal-to-emit rule is valid only under the conjunction of:

1. The span carries at least one `gen_ai.usage.*` token attribute **and** at least one provenance attribute (`gen_ai.request.model` or `gen_ai.response.model`), i.e., the span is making a cost/provenance claim.
2. The response envelope was observed to be *incomplete* — the stream terminated without a terminal usage chunk, or a tool call has no matching result within the span's lifetime, or the retry path regenerated a tool-call id.
3. The span is a **client** span (`gen_ai.operation.name` = `chat` / `execute_tool`) whose parent is the agent-loop span, not a synthetic aggregation span.

It is **not** valid to refuse emission when: the provider legitimately does not report usage (some self-hosted runtimes), in which case the correct action is to omit `gen_ai.usage.*` entirely rather than emit zeros; or when the inconsistency is between a child span and its parent (that is a parent-side aggregation bug, handled by not summing child usage into the parent, not by suppressing the child).

## Decision Boundary

The boundary is: **emit-with-omission versus refuse-and-signal.** If the inconsistency can be resolved by *dropping* the unsubstantiated attribute (e.g., omit `gen_ai.response.model` when the terminal chunk was never seen, keep the token count flagged as partial), the span should still be emitted — losing telemetry is worse than losing one attribute. The instance must refuse to emit — and instead raise a structured error to the caller and emit a single low-cardinality counter — only when dropping attributes would leave a span that still *asserts* a reconciliation it cannot support: specifically, token counts present alongside a tool-call event with no matched result, or token counts attributed to a `gen_ai.response.model` that was inferred rather than observed. In those cases the span is a cost/provenance claim that is known-false, and exporting it corrupts the only signal downstream consumers have.

## Operational Effect

Enforcing this boundary converts a silent data-quality failure into a loud, bounded one. In the observed services, adding the refusal path reduced the rate of internally inconsistent GenAI spans from roughly 3–7% of streaming spans (concentrated in client-disconnect and tool-retry paths) to under 0.2%, at the cost of a small number of dropped spans that are now counted by a dedicated counter rather than silently mis-attributed. Cost dashboards keyed on `gen_ai.response.model` became stable across the fleet; the remaining inconsistency rate is visible and alertable. The trade is explicit: a refused span is a lost data point, but an inconsistent span is a false one, and for token accounting and model provenance the false one is strictly more expensive.

```python
class GenAISpanReconciliationError(RuntimeError):
    """Raised when a GenAI span would export mutually inconsistent
    token-accounting, tool-call, or model-provenance attributes."""
    pass


def close_genai_span(span, *, usage: dict | None, request_model: str,
                     response_model: str | None, response_model_observed: bool,
                     tool_calls: list[dict], tool_results: list[dict]) -> None:
    """Finalize a GenAI span, refusing to emit known-inconsistent claims.

    usage: e.g. {"input_tokens": int, "output_tokens": int}
    response_model_observed: True only if the terminal stream chunk was seen.
    """
    # 1. Tool-call attribution: every emitted call must have exactly one result.
    call_ids = [c["id"] for c in tool_calls]
    result_ids = [r["id"] for r in tool_results]
    unmatched_calls = [i for i in call_ids if result_ids.count(i) != 1]
    orphan_results = [i for i in result_ids if i not in call_ids]

    # 2. Provenance: response model must be observed, not inferred.
    provenance_inferred = response_model is not None and not response_model_observed

    # 3. Token claim present?
    token_claim = bool(usage) and any(
        usage.get(k) is not None for k in ("input_tokens", "output_tokens")
    )

    # Refuse only when a token/provenance claim is known-false.
    if token_claim and (unmatched_calls or orphan_results):
        raise GenAISpanReconciliationError(
            f"token claim present but tool-call attribution unresolved: "
            f"unmatched_calls={unmatched_calls} orphan_results={orphan_results}"
        )
    if token_claim and provenance_inferred:
        raise GenAISpanReconciliationError(
            f"token claim attributed to inferred model {response_model!r}; "
            f"terminal chunk not observed"
        )

    # Otherwise emit with omission: drop unsubstantiated attributes, keep the span.
    if usage:
        for k, v in usage.items():
            if v is not None:
                span.set_attribute(f"gen_ai.usage.{k}", v)
    span.set_attribute("gen_ai.request.model", request_model)
    if response_model_observed and response_model:
        span.set_attribute("gen_ai.response.model", response_model)
    # ... emit tool events for matched calls only ...
    span.end()
```