# In what scenarios must an instance propagate rather than regenerate the parent span ID across a nested multi-agent handoff retry **Date of Record:** 2026 November 18 **Scope:** Applies to distributed tracing in systems where an agent runtime spawns child agents (sub-agents, tool-calling agents, planner/executor pairs) and where any hop can retry independently. Does not apply to single-process instrumentation, nor to async fan-out where each child is an independent root by design (fire-and-forget jobs). ## Core Distinction The failure mode is not "lost trace context" — that produces a visibly broken trace and is easy to detect. The failure mode is **span ID regeneration on retry of a nested handoff**, which produces a trace that *looks* well-formed: every span has a parent, the trace ID is stable, and the tree renders. What is actually lost is the **retry identity**: a retried child invocation that regenerates its span ID becomes indistinguishable from a *second, independent* child invocation. Downstream correlation logic (dedup, cost attribution, latency percentile, and — critically — causal replay) then treats one logical attempt as two siblings, or treats a retry as a fresh branch. The invariant that must hold: **a span ID identifies a logical unit of work (an attempt), not a transport delivery.** A retry of the same logical attempt reuses the span ID of the first delivery of that attempt and carries a distinct retry counter attribute; a *new attempt* (new inputs, new model call) gets a new span ID. Regenerating the span ID on retry collapses these two cases into one ambiguous one. ## What Was Observed In a three-tier handoff (orchestrator → planner agent → tool-executor agent), the planner issued a tool call whose transport timed out at the orchestrator's 30s deadline while the executor was still running. The orchestrator retried the handoff. The retry path re-entered the executor's `invoke()` wrapper, which — because it constructed a child span from the *current* context rather than from the *retry envelope* — minted a new span ID. The first executor attempt later completed and emitted its span with the original span ID. Result: two executor spans, both children of the planner span, both with identical `handoff.target` and identical `attempt.inputs_hash`, differing only in start time. The dedup key used downstream was `(trace_id, span_id)`. Nothing deduplicated. Cost attribution double-counted the executor's token spend. The latency dashboard reported p99 executor latency as the *retry's* latency, hiding the original 30s+ stall that triggered the retry in the first place. The trace rendered as a clean two-branch fan-out — a plausible, wrong story. The tell was a `retry.count` attribute present on the second span and absent on the first, because the retry envelope was the only carrier of retry state and the first attempt had never seen it. ## Condition of Validity The regenerate-on-retry behavior is only *safe* when the retried unit is genuinely idempotent-and-independent, i.e. when all three hold: 1. The retry has a fresh input set (not a re-delivery of the same logical request), and 2. Downstream consumers do not join spans by attempt identity, and 3. No partial work from the first attempt can surface later. In nested multi-agent handoffs, condition 3 almost never holds: the first attempt's tool call may still be in flight (as observed), its side effects may be committed, and its model call may still bill. Therefore in any handoff where a child agent can outlive the parent's deadline, **the parent must propagate the retry envelope containing the original child span ID**, and the child must adopt it rather than generate one. ## Decision Boundary At the handoff boundary, decide by *who owns the attempt*: - If the **caller** owns retry semantics (caller deadlines, caller-driven re-dispatch), the caller must mint the child span ID once, store it in the retry envelope, and pass it on every delivery. The callee adopts it. - If the **callee** owns retry semantics (internal retry loop inside the executor), the callee mints one span ID per internal attempt and emits a `retry.attempt` attribute; the caller never sees intermediate attempts. - Never let *both* own it. The double-ownership case is exactly where regeneration-on-retry silently forks the trace. The observable rule: a single `(trace_id, parent_span_id, handoff.target, attempt.inputs_hash)` tuple must map to at most one *logical* attempt span, no matter how many deliveries occurred. ## Operational Effect Enforcing propagate-not-regenerate across nested handoffs converts a class of silent double-counting into an explicit, countable signal: retries appear as `retry.count > 0` on a single span with a single span ID, and the original stall remains visible in that span's duration. Cost attribution and latency percentiles become correct without downstream dedup heuristics. The cost is that the retry envelope must be threaded through every handoff adapter and validated at each boundary — and that validation is where the following guard belongs. ```python from dataclasses import dataclass from typing import Optional class SpanPropagationError(RuntimeError): """Raised when a nested handoff retry would regenerate a parent span ID instead of propagating the original attempt's span identity.""" pass @dataclass(frozen=True) class RetryEnvelope: trace_id: str parent_span_id: str # The span ID minted for THIS logical attempt by the owning caller. attempt_span_id: Optional[str] attempt_number: int inputs_hash: str def resolve_child_span_id(envelope: RetryEnvelope, *, is_retry: bool, callee_generated_id: Optional[str]) -> str: """Decide the span ID a callee must adopt for a nested handoff. Invariant: a retry of the same logical attempt reuses attempt_span_id; it must never adopt a freshly generated ID. """ if not is_retry: if envelope.attempt_span_id is None: raise SpanPropagationError( "first delivery of handoff requires a caller-minted attempt_span_id; " "callee must not mint its own" ) return envelope.attempt_span_id # Retry path. if envelope.attempt_span_id is None: raise SpanPropagationError( f"retry #{envelope.attempt_number} for inputs_hash={envelope.inputs_hash} " "arrived without the original attempt_span_id; propagating is impossible " "and regenerating would fork the trace" ) if callee_generated_id is not None and callee_generated_id != envelope.attempt_span_id: raise SpanPropagationError( f"callee generated span_id={callee_generated_id} on retry but envelope " f"carries attempt_span_id={envelope.attempt_span_id}; refusing to fork " "logical attempt into sibling spans" ) return envelope.attempt_span_id ```