# Why does an instance-level causal span link fail silently when a subagent re-emits an inherited trace context after a fan-out barrier **Date of Record:** 2026 November 15 **Scope:** Applies to agent systems that propagate W3C Trace Context (or an equivalent traceparent-like tuple) across subagent boundaries and reconstruct causal parent-child links from span emission order. Does not apply to systems that assign a fresh trace ID at every agent boundary, nor to systems that link spans exclusively via explicit parent-span-id fields captured at span creation. ## Core Distinction There is a difference between a **trace context that is inherited** and a **span that is causally owned**. When a parent agent spawns N subagents, each subagent inherits the parent's `trace_id` and the parent's current `span_id` as its `parent_span_id`. The inherited span is a *reference*, not an *ownership claim*. The failure mode described here occurs when a subagent, after a fan-out barrier (a join point where the orchestrator waits for all children), re-emits the inherited context verbatim — copying `parent_span_id` unchanged — while also emitting a new span. Observability backends that reconstruct the causal graph by trusting `parent_span_id` will then place the new span as a sibling of the fan-out children rather than as a descendant of the barrier, producing a graph that is *structurally valid but causally wrong*. No error is raised. The trace looks complete. The reasoning chain is silently misordered. ## What Was Observed In a three-tier agent topology (orchestrator → planner subagents → tool-executor subagents), a fan-out of six planners was joined at a barrier span `B`. Each planner emitted a `plan.complete` span with `parent_span_id = O` (the orchestrator's span), which is correct: the planners were spawned by `O`, and `B` is a synthetic join marker, not a causal parent. After `B` resolved, the orchestrator emitted a `synthesize` span. In roughly 4% of runs under load, the `synthesize` span arrived at the collector with `parent_span_id = O` instead of the expected `parent_span_id = B`, because the synthesis step ran inside a subagent that had been handed the *original* inherited context (captured before fan-out) rather than the *post-barrier* context. The resulting trace contained two spans with identical `(trace_id, parent_span_id)` pairs at the same depth, and the synthesizer's reasoning was rendered as a peer of the planners rather than their consumer. The tell was not an error. It was a latency anomaly: the `synthesize` span's start timestamp preceded the last planner's end timestamp in 4% of traces, which is impossible if synthesis genuinely consumes all planner outputs. Causal ordering was violated, but the span graph was acyclic and all `parent_span_id` values resolved to existing spans. Standard trace validators pass this. ## Condition of Validity The invariant — *a span's `parent_span_id` must denote the span whose completion is a precondition for this span's emission* — holds only if context propagation is **barrier-aware**. It fails when any of the following hold: 1. The subagent captures its inherited context at spawn time and reuses that captured tuple after a join, rather than re-reading the orchestrator's *current* context at emission time. 2. The fan-out barrier is represented as a synthetic span `B` that does not itself become the parent of downstream work, so downstream code has no natural `parent_span_id = B` to adopt. 3. The collector deduplicates or re-parents spans by `(trace_id, parent_span_id, name)` heuristics, which masks the duplicate-parent symptom. Conversely, the invariant is *not* at risk when subagents are stateless with respect to context (each span reads the live context at creation) or when the barrier is a real span that downstream code explicitly parents to. ## Decision Boundary The decision is whether to trust `parent_span_id` as a causal assertion or to treat it as a *hint* that must be cross-validated against span timestamps and barrier membership. The boundary: - **Trust `parent_span_id`** only when every span's `start_time` is ≥ its parent's `start_time` and ≤ its parent's `end_time`, *and* the parent is not a synthetic join marker. If either check fails, the link is suspect. - **Distrust and re-derive** when a span's declared parent has multiple children whose `end_time` exceeds the child's `start_time` — this is the signature of a post-barrier span mis-parented to a pre-barrier ancestor. In practice: if `synthesize.start_time < max(planner.end_time)` for any planner, the synthesizer's parent is wrong, regardless of what `parent_span_id` says. The correct parent is the barrier span, and the link must be repaired before the trace is used for reasoning-chain attribution. ## Operational Effect Left uncorrected, this boundary failure corrupts three downstream consumers: (a) **causal attribution** — the synthesizer's output is credited to the orchestrator's direct reasoning rather than to the planners' aggregate, inflating orchestrator contribution metrics; (b) **critical-path analysis** — latency accounting double-counts the fan-out because the synthesizer appears concurrent with the planners; (c) **replay and audit** — a replayed trace reconstructs a reasoning order that never occurred, so post-hoc debugging of a bad answer attributes the error to the wrong agent. The fix is to make the barrier span the explicit parent of all post-join work and to reject any span whose declared parent violates the timestamp containment invariant, raising a typed error at trace-ingestion time rather than silently accepting the mis-link. ```python from dataclasses import dataclass from typing import Optional class CausalLinkViolation(RuntimeError): """Raised when a span's declared parent violates causal containment.""" @dataclass class Span: trace_id: str span_id: str parent_span_id: Optional[str] name: str start_time: float end_time: float is_barrier: bool = False def validate_causal_link(span: Span, parent: Optional[Span]) -> None: if parent is None: # Root span: no causal parent to check. return if parent.trace_id != span.trace_id: raise CausalLinkViolation( f"span {span.span_id} ({span.name}) declares parent {parent.span_id} " f"from a different trace ({parent.trace_id} != {span.trace_id})" ) if span.start_time < parent.start_time: raise CausalLinkViolation( f"span {span.span_id} ({span.name}) starts at {span.start_time} " f"before its declared parent {parent.span_id} ({parent.name}) " f"starts at {parent.start_time}" ) if span.start_time > parent.end_time: raise CausalLinkViolation( f"span {span.span_id} ({span.name}) starts at {span.start_time} " f"after its declared parent {parent.span_id} ({parent.name}) " f"ended at {parent.end_time}" ) def repair_post_barrier_parent( span: Span, declared_parent: Span, barrier: Span, sibling_children: list[Span], ) -> Span: """If a post-barrier span was mis-parented to a pre-barrier ancestor, re-parent it to the barrier. Detects the signature: the span starts before some sibling child of its declared parent has ended. """ if not declared_parent.is_barrier: latest_sibling_end = max( (c.end_time for c in sibling_children if c.span_id != span.span_id), default=declared_parent.start_time, ) if span.start_time < latest_sibling_end: if not barrier.is_barrier: raise CausalLinkViolation( f"span {span.span_id} ({span.name}) appears to be post-barrier " f"but candidate barrier {barrier.span_id} is not marked as a barrier" ) return Span( trace_id=span.trace_id, span_id=span.span_id, parent_span_id=barrier.span_id, name=span.name, start_time=span.start_time, end_time=span.end_time, ) return span ```