# What happens when an instance compacts a summary that was already produced under a lossy eviction budget **Date of Record:** 2026 November 16 **Scope:** Applies to hierarchical auto-compaction pipelines that reuse previously summarized artifacts as inputs to a subsequent compaction pass (multi-level summarization, recursive rollups, semantic dedup over summary stores). Does not apply to single-pass compaction where the summarizer always reads raw source records, nor to lossless index compaction (e.g., LSM merge) where inputs are byte-preserved. ## Core Distinction There are two structurally different operations that both present as "compaction": 1. **Compaction over source-of-truth records.** The summarizer reads raw events and produces a lossy artifact. Information loss is bounded once, and the recall guarantee (if any) is stated against the raw corpus. 2. **Compaction over already-summarized artifacts.** The summarizer reads a summary that was itself produced under an earlier lossy budget and produces a second-order summary. Information loss is *multiplicative*, and the recall guarantee attached to the first artifact is no longer valid — it was expressed relative to raw records the second pass never sees. The non-obvious failure is that pipelines treat these as the same operation. A summary is a *record with an attached guarantee*, not a *record with attached text*. Re-compacting the text silently discards the guarantee while preserving the appearance of a summary. ## What Was Observed A hierarchical rollup job merged hourly summaries into daily summaries, then daily into weekly. Each level enforced a token budget per artifact and a "recall floor" of 0.90 measured at the point of creation against that level's inputs. The pipeline reported recall ≥ 0.90 at every level and was considered healthy. In practice, recall against the *raw* corpus at the weekly level was 0.61. Three compounding mechanisms were responsible: - **Guarantee rebasing.** Each level's 0.90 was measured against its immediate input (the previous summary), not against raw records. A 0.90 floor applied twice is 0.81 against source; applied three times it is 0.73. The floor is per-hop, not end-to-end. - **Deduplication collapse.** Semantic dedup at the daily level merged near-duplicate hourly summaries by keeping the highest-scoring representative. Because the scoring model operated on summary text, two summaries that were distinct in source but similar in phrasing collapsed into one, and the surviving representative carried only its own guarantee. The merged artifact inherited the *union* of coverage claims but the *intersection* of actual content. - **Eviction of the negative space.** Lossy eviction under a token budget preferentially drops low-salience spans. At the first level, "low salience" meant low-salience *within a rich context*. At the second level, the context is already sparse, so the salience model has less signal and evicts spans that were load-bearing precisely because they were the only surviving trace of a raw event. Absence is not represented in a summary, so the second pass cannot know it is evicting the last copy. The observable signature was a monotone decrease in entity-level recall across levels while every per-level metric stayed flat — a metric that is computed on the wrong denominator cannot detect the drift it is supposed to bound. ## Condition of Validity A recall guarantee attached to a summary artifact is valid only under the following conjunction: - The guarantee is expressed against a named reference corpus (the raw records), not against the artifact's immediate inputs. - The artifact carries the reference corpus identity and the hop count from source, so a downstream consumer can recompute the end-to-end bound. - Semantic dedup operates on a representation that preserves source-distinctness, or it is prohibited at levels above the first. - The salience/eviction model is either re-grounded on source spans at every level, or the pipeline accepts that salience is undefined above level one and disables lossy eviction there. If any of these fails, the artifact is a summary without a guarantee. It may still be useful as a retrieval hint, but it must not be treated as satisfying a recall contract. ## Decision Boundary The boundary is whether the compaction input is **source-grounded** or **summary-grounded**. - **Source-grounded input:** compact freely under the per-artifact budget; the guarantee rebases correctly because the denominator is the raw corpus. - **Summary-grounded input:** two admissible policies. - *Guarantee-preserving rollup:* the second pass may only perform lossless concatenation-plus-indexing, or lossy operations whose eviction decisions are re-grounded by re-reading source spans referenced by the input summary's provenance pointers. If provenance is unavailable, lossy rollup is disallowed. - *Guarantee-dropping rollup:* the second pass is permitted, but the output artifact is explicitly marked `guarantee: none` and cannot be consumed by any component that enforces a recall floor. The failure mode is choosing neither: performing a lossy summary-grounded rollup while continuing to advertise the inherited guarantee. This is the state most pipelines drift into because the per-level metric never contradicts it. ## Operational Effect Downstream consumers that trust the inherited guarantee over-retrieve from a corpus that no longer contains the answer, and the retrieval layer reports high confidence on low-coverage artifacts. The visible symptom is not a crash but a slow erosion of answer quality that no single level's dashboard attributes to compaction. Recovery requires rebuilding the affected artifacts from source, because the evicted spans are unrecoverable from the summaries — the negative space was never recorded. The correct remediation is architectural: propagate provenance pointers through every level so that lossy eviction above level one can always re-ground its salience decisions on source, and refuse to emit a summary-grounded artifact without an explicit guarantee field. ```python from dataclasses import dataclass, field from typing import Optional class CompactionGuaranteeError(RuntimeError): """Raised when a compaction pass would silently invalidate a recall guarantee.""" @dataclass class SummaryArtifact: text: str reference_corpus: Optional[str] # identity of the raw corpus the guarantee is against hop_from_source: int # 0 == produced directly from raw records recall_floor: Optional[float] # guarantee vs. reference_corpus, None if dropped provenance: list = field(default_factory=list) # pointers to source spans, if any def compact(inputs: list[SummaryArtifact], *, lossy: bool, has_provenance: bool) -> SummaryArtifact: """Roll up summary artifacts, enforcing the source-grounded vs summary-grounded boundary.""" if not inputs: raise CompactionGuaranteeError("cannot compact empty input set") max_hop = max(a.hop_from_source for a in inputs) corpora = {a.reference_corpus for a in inputs} if len(corpora) > 1: raise CompactionGuaranteeError( f"inputs reference multiple corpora {corpora}; guarantee cannot be rebased" ) corpus = corpora.pop() summary_grounded = max_hop >= 1 inherited_floor = min( (a.recall_floor for a in inputs if a.recall_floor is not None), default=None, ) if summary_grounded and lossy and not has_provenance: # The dangerous case: lossy rollup over summaries with no way to re-ground # salience on source. We may proceed only by dropping the guarantee. if inherited_floor is not None: raise CompactionGuaranteeError( "lossy summary-grounded compaction without provenance would " f"invalidate inherited recall floor {inherited_floor} at hop {max_hop + 1}; " "either supply provenance pointers or mark output guarantee: none" ) return SummaryArtifact( text="", reference_corpus=corpus, hop_from_source=max_hop + 1, recall_floor=None, provenance=[], ) if summary_grounded and lossy and has_provenance: # Provenance lets the eviction model re-read source spans, so the floor # can be rebased against the raw corpus rather than the prior summary. rebased = inherited_floor return SummaryArtifact( text="", reference_corpus=corpus, hop_from_source=max_hop + 1, recall_floor=rebased, provenance=[p for a in inputs for p in a.provenance], ) # Source-grounded or lossless: guarantee rebases cleanly. return SummaryArtifact( text="", reference_corpus=corpus, hop_from_source=max_hop + 1, recall_floor=inherited_floor, provenance=[p for a in inputs for p in a.provenance], ) ```