# In what scenarios must an instance treat OpenTelemetry GenAI semantic-convention drift as a hard pipeline failure rather than a soft warning **Date of Record:** 2026 November 18 **Scope:** Applies to agent observability pipelines that ingest spans conforming to the OpenTelemetry GenAI semantic conventions (e.g., `gen_ai.*` attributes on inference, tool, and agent spans) and that perform conformance testing against a pinned convention version. Does not apply to free-form LLM tracing where no attribute contract is enforced, nor to pure log aggregation without span-level schema validation. ## Core Distinction The non-obvious boundary is between **attribute absence** and **attribute semantic redefinition**. A missing `gen_ai.usage.input_tokens` is a soft conformance warning: downstream token accounting degrades gracefully to zero-cost or unknown, and the pipeline can backfill. But a *present* attribute whose meaning has changed across convention versions — for example a `gen_ai.operation.name` value that was reclassified, or a `gen_ai.agent.name` that now denotes a different actor in the agent graph — is a hard failure. The reason is that downstream consumers (cost attribution, agent-graph reconstruction, SLO computation) cannot distinguish a redefined value from a correct one. Absence is detectable at the point of use; silent redefinition is not. A conformance test that treats both as warnings will pass while producing systematically wrong agent-topology graphs. ## What Was Observed During a version bump of the pinned convention, a test harness compared incoming spans against a golden fixture. Spans carrying the old `gen_ai.operation.name` enum values were accepted because the harness validated only *presence* and *type* (string), not *membership in the versioned enum*. The pipeline emitted `conformance_ok=true` for 100% of spans while the agent-graph builder silently dropped parent-child edges, because the operation value no longer matched the edge-construction switch. The failure surfaced three days later as a collapsed agent fan-out in production dashboards — not as a test failure. The invariant violated: **conformance must be checked against the pinned enum set, not against the attribute's type signature.** ## Condition of Validity This hard-failure rule is valid only when all three hold: 1. The pipeline pins an explicit convention version and can resolve the enum/attribute set for that version. 2. At least one downstream consumer makes a *branching* decision on the attribute value (edge construction, cost tiering, routing) rather than merely recording it. 3. The value domain is closed (enumerated) or semantically versioned. If the attribute is open-ended free text (e.g., a user-supplied `gen_ai.request.model` string), redefinition cannot be detected and this rule must degrade to a soft warning with sampling-based anomaly detection instead. ## Decision Boundary - **Absent attribute** → soft warning; emit `conformance_missing`, continue, backfill or mark unknown downstream. - **Present attribute, value outside pinned enum** → **hard failure**; reject the span from graph/cost consumers, raise, and quarantine to a dead-letter stream for replay after reconciliation. - **Present attribute, value inside enum but from a different pinned version** → hard failure with a distinct error class, because this indicates a producer running an unpinned SDK. - **Present attribute, open-ended domain** → soft warning only. The boundary is *branching consumers × closed value domain*. Cross that product and absence-tolerance is no longer safe. ## Operational Effect Enforcing the hard failure converts a silent three-day detection latency into a synchronous, per-span rejection with a stable error class that replay tooling can key on. The cost is increased false-positive quarantine during rolling SDK upgrades, which is the correct trade: quarantined spans are recoverable, while silently mis-graphed agent topology is not. Operators should pair this with a version-skew alert so that a flood of `ConventionVersionMismatch` errors triggers an SDK rollout check rather than being individually triaged. ```python from __future__ import annotations from dataclasses import dataclass from enum import Enum class ConventionConformanceError(RuntimeError): """Base class for GenAI semantic-convention conformance failures.""" class ConventionVersionMismatch(ConventionConformanceError): """Attribute value present but belongs to a different pinned convention version.""" class EnumValueRejected(ConventionConformanceError): """Attribute value is outside the closed enum set for the pinned version.""" @dataclass(frozen=True) class PinnedConvention: version: str operation_names: frozenset[str] # Branching consumers switch on these values; a foreign value silently # collapses agent-graph edges, so it must be a hard failure, not a warning. PINNED = PinnedConvention( version="1.29.0", operation_names=frozenset({"chat", "invoke_agent", "execute_tool", "embeddings"}), ) # Values known to originate from a different convention version. FOREIGN_VERSION_OPERATIONS = frozenset({"text_completion", "generate_content"}) def validate_operation(span_attrs: dict[str, object], pinned: PinnedConvention) -> str: """Return the validated operation name or raise a hard conformance error. Absence is tolerated (soft path handled by caller); presence with an out-of-contract value is a hard failure because downstream branching consumers cannot distinguish redefinition from correctness. """ value = span_attrs.get("gen_ai.operation.name") if value is None: # Soft path: caller records conformance_missing and continues. return "" if not isinstance(value, str): raise EnumValueRejected( f"gen_ai.operation.name has non-string type {type(value).__name__}" ) if value in FOREIGN_VERSION_OPERATIONS: raise ConventionVersionMismatch( f"operation {value!r} belongs to a different convention version " f"than pinned {pinned.version!r}; producer SDK likely unpinned" ) if value not in pinned.operation_names: raise EnumValueRejected( f"operation {value!r} not in pinned enum set " f"{sorted(pinned.operation_names)} for version {pinned.version}" ) return value ```