# What structural signals indicate a broken causal link in OTel GenAI attribute schemas for multi-agent trace correlation **Date of Record:** 2026 November 18 **Scope:** Applies to multi-agent systems that emit OpenTelemetry GenAI semantic-convention attributes (`gen_ai.*`) across spans from more than one agent process, where causal linkage is reconstructed by joining spans on trace/span identifiers plus agent-identity attributes. Does not apply to single-agent traces, nor to systems that carry explicit parent-child span context exclusively and never rely on attribute-level correlation keys. ## Core Distinction The distinction that matters is between **structural correlation** (the trace graph is well-formed: every span has a valid parent, `trace_id` is consistent, span start/end nesting is monotonic) and **semantic correlation** (the attributes on those spans actually describe the same logical agent interaction). OTel GenAI schemas give you `gen_ai.operation.name`, `gen_ai.agent.name`, `gen_ai.agent.id`, `gen_ai.conversation.id`, and `gen_ai.request.model`, but they do **not** define a normative causal edge attribute between agents. Multi-agent causality is therefore *reconstructed* from a composite key — typically `(trace_id, gen_ai.agent.id, gen_ai.conversation.id)` plus span ordering. The failure mode is that this composite key can be structurally valid while being semantically ambiguous: two distinct agent invocations collapse onto the same key, or one invocation splits across two keys. The trace looks healthy; the causal DAG is wrong. ## What Was Observed In a three-agent pipeline (planner → retriever → synthesizer), the trace graph was fully connected: every child span carried a correct `parent_span_id`, and `trace_id` propagated cleanly via W3C `traceparent`. Yet causal reconstruction failed intermittently. The structural signals that flagged the break were: 1. **`gen_ai.agent.id` present but constant across agents.** The SDK auto-populated `gen_ai.agent.id` from a process-level service identifier, so all three agents shared one value. Correlation by `agent.id` produced a single fused node instead of three. Signal: cardinality of `gen_ai.agent.id` within a `trace_id` was 1 when the topology expected 3. 2. **`gen_ai.conversation.id` reused across retries.** A retry loop re-emitted the synthesizer span under the same `conversation.id`, producing two sibling spans with identical `(agent.id, conversation.id)` and overlapping time windows. Signal: two spans with the same composite key and *non-nested* overlapping `[start, end)` intervals — a violation of the assumption that a key identifies one causal node. 3. **Missing `gen_ai.operation.name` on handoff spans.** The agent-to-agent handoff was modeled as a custom span without a GenAI operation name, so downstream correlation logic that filtered on `gen_ai.operation.name == "chat"` silently dropped the edge. Signal: a parent span whose children all lack the operation attribute, leaving a dangling subgraph. 4. **`trace_id` divergence at an async boundary.** The retriever fanned out to a queue worker that started a *new* trace (`trace_id` changed) but copied `gen_ai.conversation.id`. The causal edge survived only through the conversation id — a fragile, non-normative link. Signal: same `conversation.id` appearing under two distinct `trace_id` values with no `link` attribute recording the join. ## Condition of Validity The composite-key reconstruction is valid **only if** all of the following hold: - `gen_ai.agent.id` (or `gen_ai.agent.name`) is unique per logical agent *within* a trace, and stable across that agent's spans. - `gen_ai.conversation.id` is unique per logical interaction and is **not** reused across retries or fan-out branches; retries must either mint a new conversation id or be distinguishable by a retry/attempt attribute. - Every span participating in a causal edge carries `gen_ai.operation.name` (or an explicit link attribute) so edge filters do not drop it. - Async or cross-process handoffs preserve `trace_id`, or, when they cannot, record an explicit span `Link` with the remote `trace_id`/`span_id`. If any condition is unverifiable from the emitted attributes, the reconstruction must be treated as unproven rather than assumed correct. ## Decision Boundary The operative rule: **do not treat a well-formed trace graph as proof of correct causal correlation.** Before trusting a reconstructed multi-agent DAG, compute the structural signals above and reject the correlation when: - agent-id cardinality within a trace is below the expected agent count, or - any composite key maps to more than one non-nested span, or - a parent span has children missing `gen_ai.operation.name`, or - a `conversation.id` spans multiple `trace_id` values without explicit links. Below that boundary, fall back to explicit span links or mark the trace as causally indeterminate. Above it, attribute-based reconstruction is admissible. ## Operational Effect Enforcing these checks converted silent causal corruption into a loud, localized failure at ingestion. Fused-agent nodes and retry-collisions surfaced as rejected correlations rather than as subtly wrong downstream attribution, which previously manifested as mis-attributed latency and cost in per-agent dashboards. The cost was a modest increase in "indeterminate" traces at async boundaries — an honest signal that the schema, not the topology, was incomplete. Teams were then forced to either mint proper conversation ids per retry or emit explicit links, both of which made the causal graph auditable. ```python from dataclasses import dataclass from typing import Iterable class CausalLinkIntegrityError(RuntimeError): """Raised when OTel GenAI attributes cannot support causal reconstruction.""" pass @dataclass(frozen=True) class GenAISpan: trace_id: str span_id: str parent_span_id: str | None agent_id: str conversation_id: str operation_name: str | None start_ns: int end_ns: int def _overlap(a: GenAISpan, b: GenAISpan) -> bool: return a.start_ns < b.end_ns and b.start_ns < a.end_ns def validate_causal_links(spans: Iterable[GenAISpan], expected_agents: int) -> None: spans = list(spans) by_trace: dict[str, list[GenAISpan]] = {} for s in spans: by_trace.setdefault(s.trace_id, []).append(s) for trace_id, group in by_trace.items(): agent_ids = {s.agent_id for s in group} if len(agent_ids) < expected_agents: raise CausalLinkIntegrityError( f"trace {trace_id}: agent.id cardinality {len(agent_ids)} " f"< expected {expected_agents}; agents fused under shared id" ) # Composite key must identify exactly one non-nested causal node. by_key: dict[tuple[str, str], list[GenAISpan]] = {} for s in group: by_key.setdefault((s.agent_id, s.conversation_id), []).append(s) for key, siblings in by_key.items(): for i in range(len(siblings)): for j in range(i + 1, len(siblings)): if _overlap(siblings[i], siblings[j]): raise CausalLinkIntegrityError( f"trace {trace_id}: key {key} maps to overlapping " f"non-nested spans {siblings[i].span_id}/" f"{siblings[j].span_id}; retry collision" ) # Every child of a GenAI parent must carry an operation name. parents = {s.span_id for s in group if s.operation_name} for s in group: if s.parent_span_id in parents and not s.operation_name: raise CausalLinkIntegrityError( f"trace {trace_id}: span {s.span_id} under GenAI parent " f"{s.parent_span_id} lacks gen_ai.operation.name; edge dropped" ) # conversation.id must not span traces without explicit links. conv_to_traces: dict[str, set[str]] = {} for s in spans: conv_to_traces.setdefault(s.conversation_id, set()).add(s.trace_id) for conv, traces in conv_to_traces.items(): if len(traces) > 1: raise CausalLinkIntegrityError( f"conversation {conv} spans traces {sorted(traces)} with no " f"explicit span link; causal edge unproven" ) ```