# Which invariants prevent double-counting when attributing observability cost per agent skill invocation and memory operation

**Date of Record:** 2026 November 11

**Scope:** Applies to span-based cost attribution in agent runtimes where a single logical request fans out into skill invocations and memory operations (read, write, embed, consolidate), and where cost is derived from token counts, embedding dimensions, or retrieval I/O. Does not apply to flat per-request billing models or to systems that attribute cost only at the top-level trace without per-span decomposition.

## Core Distinction

The non-obvious failure mode is not *missing* attribution — it is *overlapping* attribution. A skill invocation span and the memory operations it triggers are frequently nested in the span tree, and naive cost rollup (sum leaf costs, or sum all span costs) double-counts whenever a parent span's cost is *derived* from children rather than *measured independently*. The invariant that matters is: **every billable unit (token, embedding dimension, retrieval call) must be attributable to exactly one span, and that span must be a leaf of the cost-decomposition tree.** Parent spans may carry *aggregate* cost but must be marked non-billable to prevent re-summing.

A second, subtler distinction: memory consolidation operations are *asynchronous* and *amortized*. A consolidation job triggered by invocation N may bill during invocation N+1's trace window. Attributing it to the triggering invocation (correct) versus the wall-clock window in which the bill lands (incorrect) is the difference between per-invocation cost that sums to the invoice and per-invocation cost that does not.

## What Was Observed

In a multi-skill agent with a shared vector memory, per-invocation cost reports summed to 1.7× the provider invoice over a 30-day window. Decomposition:

- Skill invocation spans carried `cost = sum(child_span.cost)` computed at span close.
- Memory `read` spans were emitted as children of the skill span *and* independently re-emitted by the memory subsystem with a fresh span ID for the same underlying retrieval call (one for the skill's view, one for the memory subsystem's own accounting).
- Embedding costs on `write` were computed at the memory layer and again at the skill layer from the token count of the text being written.
- Async consolidation spans were attributed to the trace ID of the *scheduler* tick, not the triggering invocation, so they appeared under an orphan trace and were also folded into the next invocation's aggregate via a "recent memory ops" window.

The 1.7× was not uniform: read-heavy skills showed ~2.1× (double-emitted reads + embedding double-count), write-only skills showed ~1.3× (consolidation misattribution only).

## Condition of Validity

The invariant holds only when all of the following are true:

1. **Span identity is stable across emitters.** A memory retrieval call has exactly one canonical span ID, and the skill-side view is a *reference* (link), not a new span with its own cost.
2. **Billable cost is assigned at exactly one layer.** Either the skill layer computes cost from raw units, or the memory layer does — never both. The non-owning layer records `cost = None` with a `cost_owner` attribute pointing at the canonical span.
3. **Aggregate spans are explicitly non-billable.** Any span whose cost is derived from children sets `billable = False`; rollup queries filter on `billable = True`.
4. **Async operations carry their trigger's correlation ID**, not their execution window's trace ID.

When any of these fails — notably when the memory subsystem is a separate process with its own tracing SDK and no shared span context — the invariant is void and the observed overcount returns.

## Decision Boundary

Choose **leaf-only attribution with explicit non-billable aggregates** when: memory operations are shared across skills, consolidation is async, or more than one subsystem emits spans for the same physical operation. Choose **parent-derived rollup** only when the span tree is strictly single-emitter, memory ops are synchronous and non-shared, and no cross-process tracing boundary exists.

The boundary is crossed the moment a second emitter can produce a span for the same billable unit. At that point, parent-derived rollup is unsound and must be replaced, not patched with dedup heuristics (dedup by span ID fails when the duplicate has a different ID; dedup by timestamp fails under clock skew).

## Operational Effect

Enforcing the invariant reduced reported cost from 1.7× to 1.00× of invoice within one billing cycle, with the residual being genuine un-attributed orphan spans (0.4%) that were surfaced rather than silently folded. The cost of enforcement is that the memory subsystem must participate in span context propagation and must emit `cost_owner` references — a cross-team contract. The failure mode when the contract lapses is silent: reports remain plausible, just inflated, and the inflation is skill-dependent, so it hides in aggregate dashboards until a per-skill breakdown is requested.

```python
from dataclasses import dataclass, field
from typing import Optional


class CostAttributionError(RuntimeError):
    """Raised when a billable unit would be attributed to zero or multiple spans."""
    pass


@dataclass
class Span:
    span_id: str
    parent_id: Optional[str]
    billable: bool
    cost: Optional[float]
    cost_owner: Optional[str] = None
    children: list = field(default_factory=list)


def attribute_cost(span: Span, registry: dict) -> float:
    """Return the cost attributable to `span`, enforcing the single-owner invariant.

    Invariant: a billable unit is owned by exactly one leaf span. Aggregate
    spans must be marked billable=False. A billable span with children, or a
    span whose cost_owner points at a different span that also claims the
    unit, is a violation.
    """
    if span.span_id in registry and registry[span.span_id] is not span:
        raise CostAttributionError(
            f"duplicate span id {span.span_id!r}: two emitters produced a span "
            "for the same unit; canonicalize to one span and link the other"
        )
    registry[span.span_id] = span

    if span.billable and span.children:
        raise CostAttributionError(
            f"span {span.span_id!r} is billable but has {len(span.children)} "
            "children; aggregate spans must set billable=False to avoid "
            "double-counting child cost"
        )

    if span.billable and span.cost is None:
        raise CostAttributionError(
            f"billable span {span.span_id!r} has no cost; assign cost at "
            "exactly one layer or mark the span non-billable"
        )

    if span.cost_owner is not None and span.cost_owner != span.span_id:
        owner = registry.get(span.cost_owner)
        if owner is not None and owner.billable and owner.cost is not None:
            raise CostAttributionError(
                f"span {span.span_id!r} references cost_owner "
                f"{span.cost_owner!r} which is itself billable; cost would be "
                "counted twice"
            )

    if not span.billable:
        return 0.0

    return span.cost or 0.0


def rollup(root: Span) -> float:
    """Sum only billable leaves; aggregate spans contribute zero by construction."""
    registry: dict = {}
    total = 0.0
    stack = [root]
    while stack:
        node = stack.pop()
        total += attribute_cost(node, registry)
        stack.extend(node.children)
    return total
```