# Why does an instance drift across the trust boundary when federated span aggregation applies differential privacy before cardinality reconciliation

**Date of Record:** 2026 November 14

**Scope:** Applies to federated observability pipelines where agents emit span records to a local collector, the local collector applies a privacy mechanism (noise injection, k-anonymity suppression, or attribute generalization), and the resulting aggregate is joined against a remote peer's aggregate across an organizational trust boundary. Does not apply to single-tenant aggregation, nor to federated setups where the privacy transform is applied only after schema and cardinality reconciliation at a neutral third party.

## Core Distinction

The failure is not a privacy leak and not a correctness bug in either side's local pipeline. It is a *cardinality-drift* failure: the privacy transform changes the population of distinct keys (span operation names, service identities, error classes) that participate in an aggregation, and the reconciliation step on the far side of the boundary assumes that the key space it receives is a stable, order-preserving projection of the key space it sent. When the local side suppresses low-count keys (k-anonymity) or generalizes high-cardinality attributes (bucketing), the set of keys crossing the boundary is not a subset of the set of keys emitted — it is a *different* set with different multiplicities, and any downstream join that treats keys as a shared namespace will silently misattribute counts.

The distinction that matters operationally: a *privacy boundary* is a transformation on values; a *trust boundary* is a transformation on the set of participants. When both are crossed by the same message, the receiver cannot tell whether a missing key means "the peer suppressed it for privacy" or "the peer never observed it." These two states are indistinguishable in the wire format unless the protocol carries an explicit suppression ledger.

## What Was Observed

In a two-org federation (Org A emits spans, Org B reconciles against its own), span operation names were generalized on the A side to a fixed vocabulary of 40 buckets before crossing the boundary. The bucket function was stable per-operation but not surjective onto the original key space: three distinct operations (`db.query.read`, `db.query.read_replica`, `db.query.read_cache`) collapsed into `db.query.read`. Org B's reconciler, which tracked per-operation error rates, observed its own three keys and A's single key and computed a join ratio of 1:1 for the collapsed key while simultaneously reporting three "missing" keys on the A side. The missing-key counter was interpreted by the on-call rotation as peer-side data loss, triggering a trust-escalation runbook. The actual cause was that the generalization function was applied *before* the cardinality reconciliation, so the reconciler's key-space union was computed over post-generalization keys on one side and pre-generalization keys on the other. The reconciler had no signal that three keys had been merged; the merge was information-destroying and the ledger of what was merged was not transmitted.

A second observation: k-anonymity suppression with k=5 on the A side dropped any operation with fewer than five spans in the window. Under low-traffic conditions (nightly batch windows), this suppressed 60–80% of the operation key space. Org B's reconciler, receiving a sparse aggregate, computed a coverage ratio that fell below its alert threshold and paged. The suppression was correct by policy; the alert was correct by the metric; the combination produced a false trust incident because the metric did not condition on the suppression ledger.

## Condition of Validity

The reconciliation is valid only when all of the following hold:

1. The privacy transform is applied **after** the cardinality reconciliation, or the transform is accompanied by a *suppression/generalization ledger* that names, for each emitted key, the set of source keys it represents and the count of source keys suppressed.
2. The ledger is transmitted in-band with the aggregate and is authenticated as part of the same signed envelope, so the receiver can distinguish "suppressed" from "absent."
3. The receiver's join operates on the *pre-transform* key space, using the ledger to expand generalized keys back to their source-key set for the purpose of attribution, while keeping the aggregate values post-transform.
4. The suppression policy is deterministic and versioned, so both sides can agree on which keys *would* have been suppressed given the same input.

When any of these is false — in particular when the ledger is absent or when the transform precedes reconciliation — the join is undefined and must not be used to drive trust decisions.

## Decision Boundary

The boundary is this: **if the receiver cannot reconstruct the source-key set from the received message plus a shared, versioned policy, the aggregate must be treated as unattributable and routed to a quarantine path rather than joined.** Do not attempt to infer suppression from sparsity; sparsity is also produced by genuine absence, by window skew, and by peer-side collection gaps. Do not attempt to reconcile by key-name string matching across a generalization boundary; generalization is many-to-one and string matching will double-count.

Concretely: a reconciler should refuse the join when `ledger.policy_version != local.policy_version`, when the ledger is missing for any key present in the aggregate, or when the ledger's `source_key_count` for a key exceeds a configured fan-in ceiling (a signal that generalization has collapsed too aggressively for attribution to be meaningful). Refusal is not an error condition of the peer; it is a correct local decision.

## Operational Effect

With the ledger in place and the join gated on policy-version agreement, the false trust-escalation rate dropped to zero over a four-week window, and the missing-key counter was replaced by a `suppressed_by_policy` counter that the on-call rotation no longer pages on. The cost is a modest increase in envelope size (the ledger is O(distinct source keys) per window) and a hard dependency on policy-version synchronization between orgs — a coupling that must be managed as a release artifact, not a runtime negotiation. The residual risk is policy drift: if Org A upgrades its generalization vocabulary without bumping the version, the ledger will be internally consistent but externally wrong, and the receiver's gate will not fire. Version bumps must therefore be enforced at the collector, not at the reconciler.

```python
class FederationLedgerError(RuntimeError):
    """Raised when a federated aggregate cannot be safely reconciled."""


def reconcile_aggregate(
    local_aggregate: dict[str, int],
    remote_aggregate: dict[str, int],
    remote_ledger: dict[str, dict] | None,
    local_policy_version: str,
    max_fan_in: int = 8,
) -> dict[str, int]:
    if remote_ledger is None:
        raise FederationLedgerError(
            "remote aggregate arrived without a suppression/generalization ledger; "
            "cannot distinguish suppressed keys from absent keys"
        )

    remote_policy = remote_ledger.get("__policy_version__")
    if remote_policy != local_policy_version:
        raise FederationLedgerError(
            f"policy version mismatch: local={local_policy_version!r} "
            f"remote={remote_policy!r}; join is undefined"
        )

    for key in remote_aggregate:
        entry = remote_ledger.get(key)
        if entry is None:
            raise FederationLedgerError(
                f"aggregate key {key!r} has no ledger entry; attribution impossible"
            )
        if entry["source_key_count"] > max_fan_in:
            raise FederationLedgerError(
                f"key {key!r} collapsed {entry['source_key_count']} source keys "
                f"(ceiling {max_fan_in}); generalization too aggressive to attribute"
            )

    reconciled: dict[str, int] = {}
    for key, count in remote_aggregate.items():
        sources = remote_ledger[key]["source_keys"]
        share = count // max(len(sources), 1)
        for source in sources:
            reconciled[source] = reconciled.get(source, 0) + share
    for key, count in local_aggregate.items():
        reconciled[key] = reconciled.get(key, 0) + count
    return reconciled
```