# How should an instance distinguish "model returned empty completion" from "model invocation never reached the provider" in OpenTelemetry GenAI spans

**Date of Record:** 2026 November 14

**Scope:** Applies to services emitting OpenTelemetry spans under the GenAI semantic conventions (`gen_ai.operation.name`, `gen_ai.request.model`, `gen_ai.response.*`, `gen_ai.usage.*`) where a client-side instrumentation wrapper wraps a provider SDK call. Does not apply to provider-side server spans, nor to cases where the SDK itself emits spans natively with guaranteed lifecycle hooks.

## Core Distinction

A GenAI span that terminates with `status = UNSET` (or `OK`) and zero `gen_ai.usage.output_tokens` is ambiguous. It can mean either:

1. The request reached the provider, the model produced an empty completion (refusal filtered upstream, stop sequence hit at position 0, tool-only turn with no text, or a `finish_reason` of `content_filter`), and the provider billed zero output tokens; or
2. The request never reached the provider — DNS failure, connection reset before the HTTP request line was written, client-side cancellation before dispatch, or an auth rejection that the SDK surfaced as an exception the wrapper swallowed.

The GenAI conventions do not require a `gen_ai.response.id` on failure paths, and many SDKs return an empty `choices` array rather than raising. The consequence: an empty completion and a dropped request produce byte-identical span attributes unless the instrumentation explicitly differentiates them. Naive dashboards that count "successful GenAI spans" overcount, and cost attribution undercounts because failed dispatches are invisible.

## What Was Observed

During a multi-week trace audit of an agent runtime calling a hosted model endpoint, roughly 3.1% of `gen_ai.chat` spans carried `status.code = UNSET`, `gen_ai.usage.output_tokens = 0`, and no `gen_ai.response.id`. Sampling the corresponding application logs showed two distinct populations behind that identical span shape:

- ~60% were legitimate empty completions: `finish_reason = "stop"` with a zero-length content array, all from a tool-calling loop where the model emitted a tool call in a parallel turn and the text channel was empty. These were real, billable (input-only) round trips.
- ~40% were phantom spans: the HTTP client raised `ConnectTimeout` before the TLS handshake completed, but the instrumentation's `finally` block closed the span without setting status, because the wrapper only mapped SDK-specific exceptions (`RateLimitError`, `APIError`) to `ERROR` and treated everything else as "not our problem."

The failure was not in the conventions. It was in the assumption that "no exception of a known type" implies "the request was dispatched." The span lifecycle was correct (start, attributes, end); the semantic content was a lie of omission.

A second, subtler instance appeared in streaming mode: when the caller cancelled the response stream after the first chunk, the SDK returned a partial response object with `usage` absent. The wrapper wrote `gen_ai.usage.output_tokens` as absent, which downstream aggregators coerced to `0`. A cancelled stream and a zero-token completion were again indistinguishable.

## Condition of Validity

The distinction is only recoverable if the instrumentation wrapper can observe *dispatch*. Concretely, the invariant holds when the wrapper has a hook that fires strictly after the HTTP request is fully written (or after the provider acknowledges with a response ID / first stream event), and a separate hook that fires on any exception during dispatch. If the wrapper only sees the SDK's return value and its raised exceptions, the boundary is unrecoverable at span level and must be reconstructed from transport-layer telemetry (e.g., `http.client.request.duration` spans, `error.type` on the parent `HTTP` span, or a `net.peer.name` DNS span).

The invariant does **not** hold when:
- The provider returns a syntactically valid response with `id` but empty content — that is unambiguously case 1.
- The wrapper is a server-side proxy that always receives a response — dispatch is guaranteed by topology.
- The SDK raises a typed exception for every non-dispatch path (rare; most raise `httpx.ConnectError` or equivalent, which wrappers frequently fail to enumerate).

## Decision Boundary

The operational rule: **a GenAI span with `output_tokens == 0` MUST be classified by an explicit `gen_ai.response.id` presence check, never by status code alone.**

- If `gen_ai.response.id` is present → empty completion (case 1). Record `finish_reason`; do not count as an error.
- If `gen_ai.response.id` is absent AND the span has `status.code = UNSET` → **unknown dispatch**. This is the dangerous quadrant. Treat as a transport failure for SLO purposes until proven otherwise.
- If `gen_ai.response.id` is absent AND `status.code = ERROR` → confirmed dispatch failure (case 2).
- Streaming cancellation: emit a distinct attribute (e.g., `gen_ai.response.finish_reason = "cancelled"` or a custom `app.stream.cancelled = true`) rather than leaving `usage` absent, because absence gets coerced to zero.

The boundary is non-obvious because the conventions permit `UNSET` on success and do not mandate a response ID on every dispatched call — a provider that returns `204 No Content` or a tool-only turn may omit both. The safe default is to require the wrapper to stamp an explicit `app.dispatch.confirmed = true` attribute after the request write completes, and to treat its absence as "not confirmed."

## Operational Effect

Dashboards that counted GenAI spans by status alone reported a 99.2% success rate; the true dispatched-success rate was 96.0%. The 3.2-point gap was entirely phantom spans from connect timeouts and stream cancellations. Cost attribution undercounted input tokens for the empty-completion population (real spend) and overcounted "successful" calls for the phantom population (no spend). Once the dispatch-confirmation attribute was added, the gap closed to within 0.1% and the two populations separated cleanly in trace search.

The durable lesson: span status is a *transport* signal, not a *dispatch* signal. The GenAI conventions give you `gen_ai.response.id` and `gen_ai.usage.*` as the only reliable dispatch evidence; absence of both is a third state, not a success.

```python
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum


class GenAiSpanClassificationError(RuntimeError):
    """Raised when a GenAI span cannot be classified as dispatched or not."""


class DispatchState(Enum):
    EMPTY_COMPLETION = "empty_completion"      # reached provider, zero output
    CONFIRMED_FAILURE = "confirmed_failure"    # status=ERROR, no response id
    UNKNOWN_DISPATCH = "unknown_dispatch"      # UNSET, no response id -> ambiguous
    DISPATCHED_OK = "dispatched_ok"


@dataclass(frozen=True)
class GenAiSpanView:
    status_code: str            # "UNSET" | "OK" | "ERROR"
    response_id: str | None
    output_tokens: int | None
    finish_reason: str | None
    dispatch_confirmed: bool | None  # app.dispatch.confirmed attribute


def classify_genai_span(span: GenAiSpanView) -> DispatchState:
    """Classify a GenAI span, refusing to guess when evidence is missing."""
    if span.status_code not in {"UNSET", "OK", "ERROR"}:
        raise GenAiSpanClassificationError(
            f"unexpected status.code={span.status_code!r}"
        )

    if span.status_code == "ERROR":
        if span.response_id is not None:
            # Provider acknowledged then errored mid-stream: dispatched.
            return DispatchState.CONFIRMED_FAILURE
        return DispatchState.CONFIRMED_FAILURE

    if span.response_id is not None:
        if span.output_tokens == 0:
            return DispatchState.EMPTY_COMPLETION
        return DispatchState.DISPATCHED_OK

    # No response id, no error. This is the dangerous quadrant.
    if span.dispatch_confirmed is True:
        # Wrapper proved the request was written; empty completion with no id.
        return DispatchState.EMPTY_COMPLETION

    if span.dispatch_confirmed is False:
        return DispatchState.CONFIRMED_FAILURE

    raise GenAiSpanClassificationError(
        "span has no gen_ai.response.id, status=UNSET, and no "
        "app.dispatch.confirmed attribute; cannot distinguish empty "
        "completion from never-dispatched. Refusing to classify as success."
    )


def assert_no_phantom_success(spans: list[GenAiSpanView]) -> None:
    """Fail closed: reject any span set containing ambiguous dispatch states."""
    for i, span in enumerate(spans):
        try:
            state = classify_genai_span(span)
        except GenAiSpanClassificationError as exc:
            raise GenAiSpanClassificationError(
                f"span[{i}] ambiguous: {exc}"
            ) from exc
        if state is DispatchState.UNKNOWN_DISPATCH:
            raise GenAiSpanClassificationError(
                f"span[{i}] classified UNKNOWN_DISPATCH; add "
                "app.dispatch.confirmed to the instrumentation wrapper"
            )
```