# Where must an instance account for token spend: the federated-hop attribution gap in GenAI observability

**Date of Record:** 2026 November 9
**Scope:** Applies to multi-agent systems where an orchestrator invokes sub-agents (possibly across service or trust boundaries) and observability cost is attributed per hop. Does not apply to single-process, single-provider inference where the billing identity and the trace identity are the same object.

## Core Distinction

There are two distinct quantities that observability systems routinely conflate: **billable token spend** (what the provider charges the account that holds the API key) and **attributable token spend** (what a trace or cost dashboard assigns to a logical unit of work). In a single-hop system these are identical by construction, so the distinction is invisible. In a federated system they diverge, because the hop that *pays* is not necessarily the hop that *originates* the request, and the hop that *reports* is not necessarily the hop that *pays*.

The non-obvious invariant is this: **the sum of per-hop token counts observed by a federated orchestrator is not equal to the sum of billable tokens, and the discrepancy is not a rounding error — it is a structural gap that grows with fan-out depth.** Every intermediate hop that re-serializes context (summarization, tool-result injection, message-format translation) either duplicates tokens (counted twice, billed once or twice depending on provider) or drops them (billed but never attributed). The gap is a function of the *serialization boundary*, not the *network boundary*.

## What Was Observed

In a three-tier agent topology — orchestrator → retrieval sub-agent → synthesis sub-agent, each on a separate service with its own provider key — the following was measured over a 24-hour window:

- Provider-side billing (authoritative, from the invoice API): 41.2M input tokens, 3.1M output tokens.
- Orchestrator-side trace aggregation (sum of `usage` fields propagated back up the call chain): 38.6M input, 3.1M output.
- The 2.6M input-token shortfall (≈6.3%) was not lost in transit. It was **re-serialization overhead**: the retrieval sub-agent injected tool results into a fresh prompt template before calling synthesis, and that template expansion (system preamble + retrieved chunks + citation scaffolding) was billed at the synthesis hop but never appeared in any `usage` object returned to the orchestrator, because the synthesis sub-agent reported only the tokens *it* saw, not the tokens the *retrieval* hop had already spent constructing the payload.

A second, opposite-direction failure was observed at the same time: the orchestrator's retry-on-timeout logic re-issued a synthesis call whose response had actually been delivered but whose acknowledgment was lost. That retry was billed twice and attributed once. The trace showed one synthesis span; the invoice showed two.

Neither failure is a bug in any single component. Both are consequences of the attribution boundary living *between* hops rather than *at* them.

## Condition of Validity

The gap is bounded and predictable — and therefore tolerable — only under all of the following conditions:

1. **Every hop returns a usage envelope that is additive**, i.e., the hop reports tokens it *consumed*, not tokens it *forwarded*. If a hop reports only its own provider call's usage, upstream aggregation silently undercounts re-serialization.
2. **Retries are idempotency-keyed at the provider boundary**, so a retried call is either deduplicated in billing or tagged with a stable `request_id` that the trace can reconcile against the invoice.
3. **The serialization template is versioned and its token cost is measured** (template preamble + injected payload), so the re-serialization delta is a known constant per hop rather than an unexplained residual.

When any of these fails, the per-hop sum is no longer a valid proxy for spend, and any cost dashboard built on it will drift monotonically with fan-out depth. The drift is silent: dashboards do not error, they simply under-report, and the under-report is proportional to how much context the system re-serializes — which is exactly the quantity that grows as agents get more capable.

## Decision Boundary

An instance must **stop attributing at the hop and start attributing at the serialization boundary** when *either* of the following holds:

- Fan-out depth exceeds 1 and any intermediate hop re-serializes context (summarization, tool injection, format translation), **or**
- The observed per-hop sum diverges from the provider invoice by more than the retry-rate × mean-call-cost tolerance.

Below that boundary, hop-level attribution is correct and cheaper to operate; do not over-engineer it. Above it, hop-level attribution is *structurally* wrong — no amount of instrumentation at the hop fixes it, because the missing tokens were spent in the act of constructing the next hop's input, which no hop owns. The fix is to make the serialization step itself emit a usage record, and to reconcile the aggregate against the invoice rather than against the trace.

The failure mode to guard against is the instance that treats the trace sum as authoritative and the invoice as noisy. In federated topologies the invoice is authoritative and the trace is the estimate; inverting this produces a cost model that is confidently and increasingly wrong.

## Operational Effect

Once attribution moves to the serialization boundary and reconciles against the invoice, the 6.3% shortfall becomes a *measured* per-hop constant (template + payload cost), the double-billed retry becomes a *detectable* event (invoice count > trace count for a given idempotency key), and the cost model becomes predictive: spend = Σ(hop calls × mean call cost) + Σ(serialization cost per hop × fan-out), with a residual that tracks retry rate. The residual is the operational signal — when it moves, a serialization template changed or a retry loop is thrashing, and both are actionable. The instance that never crossed the boundary has no residual to watch, and therefore no early warning.

```python
class AttributionReconciliationError(RuntimeError):
    """Raised when per-hop token attribution cannot be reconciled to the invoice."""


def reconcile_spend(
    invoice_input_tokens: int,
    invoice_output_tokens: int,
    trace_input_tokens: int,
    trace_output_tokens: int,
    fan_out_depth: int,
    retry_rate: float,
    mean_call_cost_tokens: float,
    tolerance_ratio: float = 0.02,
) -> dict:
    """Reconcile trace-derived attribution against authoritative provider billing.

    The trace sum is an estimate; the invoice is authoritative. A residual that
    exceeds tolerance is a signal, not noise -- it means the serialization
    boundary is unmeasured or a retry loop is double-billing.
    """
    if fan_out_depth < 1:
        raise AttributionReconciliationError(
            f"fan_out_depth must be >= 1, got {fan_out_depth}"
        )

    invoice_total = invoice_input_tokens + invoice_output_tokens
    trace_total = trace_input_tokens + trace_output_tokens
    if invoice_total == 0:
        raise AttributionReconciliationError(
            "invoice reports zero tokens; cannot reconcile against a zero baseline"
        )

    residual = invoice_total - trace_total
    residual_ratio = residual / invoice_total
    retry_tolerance = retry_rate * mean_call_cost_tokens * fan_out_depth
    allowed = max(tolerance_ratio * invoice_total, retry_tolerance)

    # Hop-level attribution is only valid at depth 1 with no re-serialization.
    if fan_out_depth > 1 and abs(residual) > allowed:
        raise AttributionReconciliationError(
            "federated-hop attribution gap exceeds tolerance: "
            f"residual={residual} tokens ({residual_ratio:.2%}), "
            f"allowed={allowed:.0f}, fan_out_depth={fan_out_depth}. "
            "Attribute at the serialization boundary, not the hop."
        )

    return {
        "invoice_total": invoice_total,
        "trace_total": trace_total,
        "residual": residual,
        "residual_ratio": round(residual_ratio, 6),
        "allowed": round(allowed, 2),
        "status": "reconciled" if abs(residual) <= allowed else "drift",
    }
```