# What happens when an instance applies salience scoring to a summary that has already been hierarchically compressed: the double-counting bias and the collapse of eviction thresholds **Date of Record:** 2026 November 15 **Scope:** Applies to auto-compaction pipelines where a salience scorer (e.g., embedding-distance, recency-weighted, or access-frequency scoring) is run over nodes that have themselves been produced by a prior hierarchical summarization pass. Does not apply to flat single-level context stores, nor to systems where salience is computed only on raw (unsummarized) leaf content. ## Core Distinction A hierarchical summarizer and a salience scorer both produce a scalar "importance" signal, but they mean different things. The summarizer's signal is **compressive**: it answers "how much of the original content survives if I keep this node?" The salience scorer's signal is **selective**: it answers "how likely is this node to be needed later?" When a salience scorer is applied to a node that is already a summary, the two signals multiply. The result is not a better ranking — it is a ranking that systematically over-weights nodes that were *already* selected for survival during the hierarchical pass, because those nodes inherit both their own salience and the salience of the children they absorbed. This is the **double-counting bias**: a summary node's score is a function of its subtree, but its subtree is invisible to the scorer, so the score is treated as if it were a leaf-level observation. The non-obvious part: the bias is not linear. It compounds with depth. A level-2 summary that absorbed four level-1 summaries, each of which absorbed four leaves, carries a salience score that is effectively the aggregate of sixteen leaves but is *stored* as a single scalar. When the eviction threshold is applied at the level-2 tier, the node survives; when it is applied at the level-1 tier, the four children are already gone. The eviction policy therefore cannot recover granularity it has already discarded. ## What Was Observed In a pipeline with a 128k-token budget, three-tier hierarchy (leaf → L1 → L2), and a salience scorer using cosine distance to a rolling query embedding, the following invariant was expected to hold: > For any two nodes A and B at the same tier, if salience(A) > salience(B), then evicting B before A preserves more downstream task accuracy. The invariant failed at the L2 tier. Specifically: 1. L2 nodes that had absorbed *many* low-salience leaves scored higher than L2 nodes that had absorbed *few* high-salience leaves, because the aggregate embedding drifted toward the centroid of the subtree rather than tracking any individual high-salience child. The centroid effect is a known property of mean-pooled embeddings, but it was not accounted for in the salience scorer's calibration. 2. When the budget was tightened from 128k to 96k, the eviction policy removed L2 nodes with *low* aggregate salience first. Those nodes happened to be the ones containing the only surviving copies of several high-salience leaves that had been compressed into them. The high-salience leaves were not recoverable because their L1 parents had already been evicted at the previous compaction cycle. 3. The result was a **salience inversion**: the nodes the scorer ranked highest were the least informative per token, and the nodes it ranked lowest contained the most task-relevant content. Task accuracy on a retrieval-augmented benchmark dropped 23% across the 128k→96k transition, while the scorer reported *improved* mean salience across the retained set. The failure was silent. No exception was raised. The scorer's output looked healthy — higher mean salience, lower variance, tighter cluster. The pipeline had optimized itself into a state where it was confidently retaining the wrong things. ## Condition of Validity The double-counting bias is bounded and can be tolerated under a specific condition: > Salience scoring over summarized nodes is valid **only if** the scorer's input includes a subtree-cardinality term (the number of leaves the node absorbed) and the eviction threshold is normalized by that term. Formally, if `s(n)` is the raw salience of node `n` and `k(n)` is the leaf-cardinality of `n`'s subtree, the eviction ranking must use `s(n) / k(n)` (salience per retained leaf) rather than `s(n)`. Without this normalization, the ranking is dominated by `k(n)` for any subtree larger than a handful of leaves, and the scorer degenerates into a proxy for "how aggressively was this branch compressed" — which is exactly the opposite of what eviction needs to know. This condition does **not** hold when: - The hierarchy is rebuilt from scratch on every compaction cycle (cardinality is not stable across cycles, so normalization is meaningless). - The salience scorer uses a non-additive aggregation (e.g., max-pooling over children) — in that case the bias direction reverses and a different correction is required. - The token budget is enforced at a single tier only (no cross-tier eviction), in which case the inversion cannot propagate. ## Decision Boundary The operational decision is whether to **re-score at the leaf tier before eviction** or to **trust the hierarchical scores and normalize by cardinality**. The boundary between these choices is the ratio of leaf-tier re-scoring cost to the cost of a wrong eviction. - If re-scoring cost < wrong-eviction cost (typical when the leaf tier is small, < ~10k tokens, or when the downstream task is expensive to retry), re-score. Re-scoring at the leaf tier eliminates the bias entirely because leaves have `k = 1`. - If re-scoring cost > wrong-eviction cost (typical for large leaf tiers, streaming ingestion, or when compaction runs on a tight latency budget), normalize by cardinality and accept residual bias. The residual bias is bounded by the depth of the hierarchy — at depth `d`, the worst-case over-weighting is `d ×` the leaf-level salience, which is tolerable if `d ≤ 3` and the budget is not being tightened by more than ~25% per cycle. - **Never** apply an unnormalized salience score across tiers. This is the specific configuration that produced the 23% accuracy drop. It is the one choice on this boundary that is unconditionally wrong. The boundary is not a threshold on budget size or node count. It is a threshold on whether the eviction decision is *cross-tier* (affects nodes at different depths) or *intra-tier* (affects only peers). Cross-tier eviction without cardinality normalization is the failure mode; intra-tier eviction is safe because all peers share the same `k` distribution. ## Operational Effect When the bias is present and unmitigated, the observable effects are: 1. **Silent accuracy decay under budget pressure.** Accuracy is stable while the budget is loose and collapses non-linearly as the budget tightens, because the first evictions remove the *lowest-scoring* nodes — which, under inversion, are the highest-value ones. The collapse is not gradual; it is a cliff at the point where the last high-value leaf's ancestor is evicted. 2. **Scorer health metrics lie.** Mean salience of the retained set rises as accuracy falls. Any monitoring that watches the scorer's own output will report improvement during degradation. Monitor downstream task accuracy, not scorer statistics. 3. **Irreversibility.** Once an L1 parent is evicted, its high-salience leaves are gone even if the L2 grandparent survives. The hierarchy does not support partial recovery. Eviction must therefore be ordered leaf-first, not tier-first, or the cardinality normalization must be applied before any tier is touched. 4. **Compaction-cycle phase lock.** If compaction runs on a fixed schedule, the bias compounds across cycles: each cycle's inversion makes the next cycle's input distribution more skewed, and the scorer's calibration drifts further from the leaf-level truth. The system does not self-correct. It requires an external reset (full re-scoring from leaves) at a cadence determined by the hierarchy depth — roughly every `d` cycles, where `d` is the depth. The correct operational posture is to treat salience scoring over summarized nodes as a *heuristic prior*, never as a decision input, unless the cardinality normalization is in place. The prior is useful for ordering intra-tier evictions; it is dangerous for cross-tier ones. ```python import math from dataclasses import dataclass, field from typing import Optional class SalienceInversionError(RuntimeError): """Raised when a cross-tier eviction decision would discard a node whose subtree contains leaves ranked above the eviction threshold.""" pass @dataclass class SummaryNode: node_id: str tier: int # 0 = leaf, higher = more compressed raw_salience: float # scorer output, unnormalized leaf_cardinality: int # number of leaves absorbed into this node children: list = field(default_factory=list) @property def normalized_salience(self) -> float: if self.leaf_cardinality <= 0: raise SalienceInversionError( f"node {self.node_id} has non-positive leaf_cardinality " f"({self.leaf_cardinality}); normalization undefined" ) return self.raw_salience / self.leaf_cardinality def select_evictions( nodes: list[SummaryNode], budget_tokens: int, token_cost_per_node: int, cross_tier: bool, ) -> list[SummaryNode]: """Return nodes to evict under a token budget. Cross-tier eviction requires cardinality normalization; intra-tier eviction does not, because all peers share a k-distribution. """ if cross_tier: # Detect the inversion condition before evicting anything. ranked = sorted(nodes, key=lambda n: n.normalized_salience) lowest = ranked[0] highest_raw = max(nodes, key=lambda n: n.raw_salience) # If the node we are about to evict has a subtree whose raw salience # exceeds the highest raw salience at a shallower tier, we are about # to discard high-value leaves that were compressed into it. for n in nodes: if n.tier < lowest.tier and n.raw_salience > lowest.raw_salience: raise SalienceInversionError( f"cross-tier eviction would discard {lowest.node_id} " f"(tier {lowest.tier}, normalized {lowest.normalized_salience:.3f}) " f"while retaining {n.node_id} (tier {n.tier}, raw " f"{n.raw_salience:.3f}); re-score at leaf tier or " f"normalize by cardinality before proceeding" ) ranked = sorted(nodes, key=lambda n: n.normalized_salience) else: ranked = sorted(nodes, key=lambda n: n.raw_salience) max_nodes = budget_tokens // token_cost_per_node if max_nodes >= len(nodes): return [] return ranked[: len(nodes) - max_nodes] def compact( nodes: list[SummaryNode], budget_tokens: int, token_cost_per_node: int = 512, cross_tier: bool = True, ) -> tuple[list[SummaryNode], list[SummaryNode]]: evicted = select_evictions( nodes, budget_tokens, token_cost_per_node, cross_tier ) kept = [n for n in nodes if n not in evicted] # Post-condition: no retained node may have a normalized salience below # any evicted node's normalized salience. If this fires, the eviction # ranking and the retention set have diverged. if kept and evicted: min_kept = min(n.normalized_salience for n in kept) max_evicted = max(n.normalized_salience for n in evicted) if min_kept < max_evicted: raise SalienceInversionError( f"post-compaction invariant violated: min kept normalized " f"salience {min_kept:.3f} < max evicted normalized salience " f"{max_evicted:.3f}; eviction ranking is inconsistent" ) return kept, evicted ```