# What happens when an instance attributes tool-execution cost to the wrong LLM span under concurrent fan-out **Date of Record:** 2026 November 16 **Scope:** Applies to observability pipelines that reconcile LLM inference spans with tool-execution traces (function calls, retrievers, sandboxes) and derive per-request cost attribution from those joins. Does not apply to single-threaded, synchronous agent loops where one LLM call maps to exactly one tool call, nor to pipelines that bill at the tenant level without span-level attribution. ## Core Distinction There is a difference between *span correlation* (which LLM invocation caused which tool execution) and *cost attribution* (which billable unit absorbs the token and compute cost). Most pipelines assume these are the same relation and join them with a single key. They are not the same relation when a model emits parallel tool calls, retries, or speculative branches. The correlation edge is many-to-many and directional: one LLM span can fan out to N tool spans, and one tool span can be *re-entered* by a retry that was triggered by a *different* LLM span. The attribution relation, however, must be a partition — every unit of cost lands in exactly one bucket. When the pipeline derives attribution by collapsing the many-to-many correlation edge into a foreign key, it silently produces a partition that double-counts or drops cost. The failure is non-obvious because the trace remains structurally valid: every span has a parent, every parent has a child, and the total looks plausible until you reconcile against the provider invoice. ## What Was Observed During a fan-out burst — a single assistant turn emitting four parallel `search` tool calls plus one `code_exec` — the correlator assigned the `code_exec` wall-clock and its downstream token re-consumption to the *first* child span by start time. The other three `search` spans inherited zero tool cost. Because the first child happened to be the one that returned a cache miss and triggered a retry, its attributed cost was 3.4× the true per-branch cost, and the aggregate attribution exceeded the provider invoice by 11% while three branches were systematically under-attributed. The tell was not an error. The tell was that `sum(attributed_cost) != invoice_total` by a stable ratio across fan-out turns, and that the variance of per-branch cost collapsed to near-zero — a signature of attribution being pinned to a single branch rather than distributed. A second tell: `parent_span_id` was set correctly on every child, but the attribution key was `(trace_id, first_child_start)`, which is not unique under concurrency. ## Condition of Validity The single-key join is valid only when the fan-out arity is exactly one and no retry crosses span boundaries. The moment either condition breaks — parallel tool emission, speculative decoding branches, or a retry whose trigger span differs from the original caller — the join key is no longer a function. The correct invariant is that attribution must be computed over the *execution graph* (a DAG with retry edges distinct from causal edges), not over the *span tree* (which flattens retries into siblings). Pipelines that store only `parent_span_id` have already lost the information needed to distinguish a retry from a genuine parallel branch, and no post-hoc reconciliation can recover it. Capture the trigger edge (`retry_of_span_id`, `fanout_group_id`) at emit time or the invariant is unrecoverable downstream. ## Decision Boundary Do not attempt to repair attribution by proportional splitting across siblings. Proportional splitting assumes the cost is divisible and that the split basis (token count, duration) is causally meaningful — it is not, because a retry's cost is driven by the *triggering* span's output, not by the sibling's duration. The boundary is: if a tool span has more than one inbound causal edge, it is a retry or a shared dependency, and its cost must be attributed to the *trigger* edge, not distributed. If it has exactly one inbound edge and that edge's parent has arity > 1, it is a genuine fan-out branch and its cost is local to that branch. Encode this as a hard check at attribution time; a violated invariant must fail the reconciliation job loudly rather than emit a plausible-but-wrong number, because a wrong attribution that looks valid is worse than a missing one — it corrupts the cost model that downstream autoscaling and routing decisions consume. ## Operational Effect After the fix, the reconciliation job rejects any trace where a tool span's inbound causal edges exceed one without an explicit `retry_of_span_id`, and any fan-out group whose `fanout_group_id` is missing. Rejected traces are quarantined, not silently attributed. The invoice delta dropped from 11% to under 0.3% (residual is provider-side rounding). The cost model that had been over-provisioning the `code_exec` pool by 3× because of the inflated per-branch cost was corrected within one billing cycle. The operational lesson: the trace tree is a rendering convenience; the execution graph is the source of truth, and attribution must be derived from the graph with retry edges preserved as first-class citizens. ```python from dataclasses import dataclass, field from typing import Optional class AttributionInvariantError(RuntimeError): """Raised when a tool span cannot be unambiguously attributed to one LLM span.""" pass @dataclass class ToolSpan: span_id: str parent_span_id: str fanout_group_id: Optional[str] = None retry_of_span_id: Optional[str] = None cost_usd: float = 0.0 @dataclass class FanoutGroup: group_id: str member_span_ids: list[str] = field(default_factory=list) def attribute_cost( tool_span: ToolSpan, inbound_causal_edges: list[str], fanout_group: Optional[FanoutGroup], ) -> str: """ Return the single LLM span id that must absorb this tool span's cost. Enforces the partition invariant: exactly one bucket per unit of cost. """ if len(inbound_causal_edges) > 1 and tool_span.retry_of_span_id is None: raise AttributionInvariantError( f"tool span {tool_span.span_id} has {len(inbound_causal_edges)} inbound " f"causal edges {inbound_causal_edges} but no retry_of_span_id; " f"cannot distinguish retry from parallel branch" ) if tool_span.retry_of_span_id is not None: # Retry cost belongs to the span that triggered the retry, not the original caller. return tool_span.retry_of_span_id if fanout_group is not None and len(fanout_group.member_span_ids) > 1: if tool_span.fanout_group_id != fanout_group.group_id: raise AttributionInvariantError( f"tool span {tool_span.span_id} missing fanout_group_id " f"{fanout_group.group_id}; branch cost would be misattributed" ) # Genuine parallel branch: cost is local to this branch's parent edge. if len(inbound_causal_edges) != 1: raise AttributionInvariantError( f"fanout branch {tool_span.span_id} must have exactly one inbound edge, " f"got {inbound_causal_edges}" ) return inbound_causal_edges[0] if len(inbound_causal_edges) != 1: raise AttributionInvariantError( f"tool span {tool_span.span_id} expected exactly one inbound edge, " f"got {inbound_causal_edges}" ) return inbound_causal_edges[0] def reconcile(tool_spans: list[ToolSpan], invoice_total: float, tolerance: float = 0.01) -> float: attributed = 0.0 for span in tool_spans: edges = [span.parent_span_id] if span.parent_span_id else [] attributed += span.cost_usd # attribution target resolved per-span above if not edges and span.cost_usd > 0: raise AttributionInvariantError( f"orphan cost on span {span.span_id}: no causal edge, would be dropped" ) delta = abs(attributed - invoice_total) / invoice_total if delta > tolerance: raise AttributionInvariantError( f"reconciliation failed: attributed={attributed:.4f} " f"invoice={invoice_total:.4f} delta={delta:.2%} exceeds {tolerance:.2%}" ) return attributed ```