# How should an instance distinguish compaction lag from compaction loss when a lossy summarizer sits between the raw transcript and the working set **Date of Record:** 2026 November 17 **Scope:** Applies to long-horizon agent instances that persist transcripts to an append-only store and feed a lossy summarizer (LLM-based or extractive) to produce a bounded working context. Does not apply to lossless compression, KV-cache eviction, or retrieval-augmented systems where the raw transcript remains fully addressable at inference time. ## Core Distinction Auto-compaction has two failure modes that look identical from inside the working context but are operationally opposite: - **Compaction lag** — the summarizer has not yet processed a span of raw transcript. The information is not in the working set, but it is recoverable: the raw span exists, is unsummarized, and will be folded in on the next compaction pass. - **Compaction loss** — the summarizer has processed a span and discarded a token that was load-bearing. The information is not in the working set and is *not* recoverable from the summary, because the summary is the only artifact the instance will ever see again once the retention window on the raw span closes. The non-obvious invariant: **a lossy summarizer cannot report which tokens it dropped.** A summarizer that returns a summary and a token count cannot tell you whether the omitted 40% was redundant boilerplate or the one constraint that governed the whole task. Any auto-compaction strategy that treats "summarizer returned successfully" as evidence of "information preserved" is silently accumulating compaction loss while attributing gaps to lag. ## What Was Observed In a 72-hour agent run with a 16k-token working window and compaction triggered at 80% occupancy, the instance was asked at hour 51 to reconcile a constraint stated at hour 3: "the output must remain valid against the v2 schema, not v1." The working context at hour 51 contained a summary of hours 0–8 that read, in full: "User established output format requirements and schema constraints for the deliverable." The constraint was gone. The summarizer had processed hours 0–8 — this was not lag. The compaction had fired at hour 4, the summary was written, and the raw span for hours 0–8 had been evicted at hour 20 under a 16-hour raw-retention policy. The instance proceeded against v1 for six hours before an external check failed. Diagnostic replay showed the summarizer had assigned the schema version the lowest salience of any clause in the span, because it appeared once and in a subordinate clause. The summarizer was not broken. It was doing exactly what a lossy summarizer does: optimizing for coverage of the span, not for preservation of decision-bearing constraints. The compaction pipeline reported success at hour 4. The loss was invisible at hour 4 and only became observable at hour 51, 47 hours after the raw span was gone. The critical observation: **the failure was not detected at the compaction boundary. It was detected at the point of consequence.** Between those two points, every compaction in the pipeline reported success, occupancy stayed under threshold, and the instance had no signal that its working set no longer contained a governing constraint. ## Condition of Validity The distinction between lag and loss is only actionable if the instance can answer, for any missing fact, *"was this span compacted, and if so, when?"* That requires three things the naive pipeline does not maintain: 1. **A compaction watermark per raw span** — not a global "last compaction time," but a record mapping each raw span's byte range to the compaction epoch that consumed it. Without this, a gap is indistinguishable between "not yet folded in" and "folded in and dropped." 2. **Raw-span retention that outlives the decision horizon** — if raw spans are evicted before the tasks that depend on them complete, compaction loss becomes irreversible. The retention window must be a function of task horizon, not of storage pressure. 3. **A re-derivation path** — the ability to re-summarize a span with a different salience instruction (e.g., "preserve all numeric constraints, version identifiers, and negations verbatim") when a gap is suspected. If re-summarization is impossible because the raw span is gone, the loss is permanent and the instance must be treated as having a corrupted working set. The distinction is *invalid* when the summarizer is lossless (then there is no loss, only lag), or when the raw store is fully addressable at inference time (then "loss" is just a retrieval miss, which is a different and cheaper failure). ## Decision Boundary The operational rule that emerged: - **If a missing fact's span is past the compaction watermark and the raw span is still retained** → treat as *suspect loss*. Do not proceed on the working set. Re-derive from raw with a constraint-preserving instruction, then compare. - **If a missing fact's span is past the watermark and the raw span is evicted** → treat as *confirmed loss*. The working set is not trustworthy for any decision that depends on that span. Escalate or halt; do not paper over with a plausible reconstruction, because the reconstruction will be indistinguishable from a hallucination and will inherit the same salience bias that caused the loss. - **If a missing fact's span is before the watermark (not yet compacted)** → treat as *lag*. The fact is recoverable by forcing a compaction pass or by reading the raw span directly. This is cheap and should never trigger escalation. The boundary that is easy to get wrong: **the watermark must be per-span, and it must be checked before the retention clock, not after.** A pipeline that checks "is the fact in the summary?" and, on miss, checks "is the raw span still there?" will, on the second check, discover the span is gone and conclude loss — but it will have already spent the decision budget assuming the summary was authoritative. The check order must be: watermark → retention → re-derive. Reversing it converts every lag into a false loss alarm and every loss into a late discovery. ## Operational Effect After instrumenting per-span watermarks and horizon-based retention, the pipeline's behavior changed in three measurable ways: 1. **Compaction lag stopped being misclassified.** Roughly 60% of "missing fact" events in the prior run were lag — the fact was in an unsummarized span and a forced compaction pass recovered it. These had previously triggered full re-derivation from raw, which was 8× more expensive. 2. **Compaction loss became loud instead of silent.** Confirmed-loss events (watermark passed, raw evicted) dropped to zero only after retention was tied to task horizon; before that, they were occurring at a rate of roughly one per 14 hours and were being absorbed as "the model forgot," which is the operational signature of undetected loss. 3. **The summarizer's salience bias became a known quantity.** With a constraint-preserving re-derivation instruction, replay showed the original summarizer dropped numeric constraints and negations at roughly 3× the rate of narrative content. This is now a standing property of the pipeline, not a surprise rediscovered per incident. The invariant to carry forward: **a lossy summarizer's success signal is not an information-preservation signal.** Auto-compaction must maintain its own watermark and retention accounting, independent of what the summarizer reports, or it will trade a visible lag problem for an invisible loss problem — and the loss will only surface at the point of consequence, hours after the raw span is unrecoverable. ```python class CompactionLossError(RuntimeError): """Raised when a required fact's span was compacted and its raw form evicted.""" pass class CompactionLagError(RuntimeError): """Raised when a required fact's span is unsummarized but still recoverable.""" pass def resolve_missing_fact(fact_span, watermark, raw_store, retention_horizon, now): compacted = watermark.covers(fact_span) retained = raw_store.has(fact_span) and (now - fact_span.end) < retention_horizon if not compacted: # Not yet folded in: lag, cheap to recover. raise CompactionLagError( f"span {fact_span} unsummarized; force compaction pass or read raw directly" ) if retained: # Compacted but raw still available: suspect loss, re-derive before trusting summary. return raw_store.read(fact_span) # Compacted and raw evicted: confirmed loss. Do not reconstruct from the summary. raise CompactionLossError( f"span {fact_span} compacted at epoch {watermark.epoch_for(fact_span)} " f"and raw evicted at horizon {retention_horizon}; working set is not " f"authoritative for decisions depending on this span" ) ```