# In what scenarios must an instance refuse auto-compaction when lossy summarization would violate a downstream replay invariant **Date of Record:** 2026 November 18 **Scope:** Applies to long-horizon agent trace stores that auto-compact raw event streams into summarized segments under a declared lossy-summarization guarantee (bounded information loss per segment). Does not apply to append-only audit logs with no compaction, nor to compaction where the summarizer is provably lossless (exact deduplication, reference-counted tokenization). ## Core Distinction The non-obvious boundary is between *compaction that reduces storage* and *compaction that destroys replayability*. A trace store can have a perfectly valid lossy-summarization guarantee — e.g., "each summarized segment preserves all tool-call arguments and final outcomes, dropping only intermediate reasoning tokens" — and still be unsafe to compact, because the guarantee is stated over the *segment*, while downstream consumers replay over the *inter-segment boundary*. The failure mode: auto-compaction triggers when a segment exceeds a size or age threshold. The summarizer is invoked on the segment in isolation. It correctly honors the lossy guarantee for that segment. But the segment's *last* event and the *next* segment's *first* event form a causal pair (a tool call and its result, a plan step and its revision, a retry and its backoff) that the summarizer, seeing only one side, collapses into a generic "action taken" or "result received" token. Each segment individually satisfies its guarantee; the concatenation does not. This is a boundary invariant violation, not a summarizer bug. The distinction matters because operators typically validate the summarizer against per-segment fixtures, see green, and enable auto-compaction. The invariant that fails is `replay(compact(trace)) ≡ replay(trace)` over *paired* events, which no per-segment test exercises. ## What Was Observed On a trace store with 180-day retention and a 4-hour auto-compaction window, a replay harness began failing on traces older than the compaction horizon. Diagnosis: 1. Raw traces replayed deterministically: tool-call/result pairs matched by `call_id`. 2. Compacted traces replayed non-deterministically: 3–7% of tool-call/result pairs were unmatched. 3. The unmatched pairs clustered at segment boundaries. In each case, the tool call was the final event of segment *N* (summarized to `"invoked "` with the `call_id` dropped as "intermediate detail"), and the result was the first event of segment *N+1* (summarized to `"received result"` with the `call_id` dropped symmetrically). 4. The summarizer's per-segment lossy guarantee held: it was permitted to drop `call_id` as intermediate reasoning metadata. The guarantee was silent on cross-segment referential integrity. Compaction had been running for weeks before the replay harness — which only ran on a nightly sample of *recent* traces — was extended to the full retention window. The invariant had been violated since day one of compaction; detection lagged by the harness's sampling scope, not by any summarizer change. A second, subtler observation: the auto-compaction trigger was *size-based*, so segment boundaries fell at arbitrary event indices. The same trace, ingested with different batching, produced different boundaries and therefore a different set of broken pairs. Compaction was not idempotent under re-ingestion — a property the store's documentation implicitly assumed. ## Condition of Validity Auto-compaction with lossy summarization is valid if and only if, for every segment boundary `(last_event(N), first_event(N+1))`, the summarizer's output preserves every field that any registered downstream consumer uses to join across the boundary. Operationally this requires: - A declared **join-key set** `J` (e.g., `{call_id, plan_id, retry_parent}`) that consumers rely on for replay. - A summarizer contract that, at segment boundaries, is *lossless over J* — it may drop everything else, but must emit join keys verbatim on both the trailing and leading events. - Boundary-aware segmentation: the compactor must not cut a segment between an event carrying a join key and its matching event, or must mark the cut as a *join boundary* and emit a bridging record. If any of these is absent, the lossy guarantee is stated over the wrong scope and compaction is unsafe regardless of how well the summarizer performs on per-segment fixtures. The invariant to enforce is: `∀ j ∈ J, ∀ boundary b, join_keys_preserved(compact, b, j)`. ## Decision Boundary An instance must **refuse** auto-compaction (and emit an alert) when: - The join-key set `J` is non-empty and the summarizer contract does not explicitly guarantee boundary-lossless emission of `J`. Refusal is mandatory; a "best effort" compaction silently violates replay. - The segment boundary is *unstable* — i.e., segmentation depends on ingestion batching rather than event content — and any consumer replays across boundaries. Unstable boundaries make the invariant unverifiable, so compaction must be disabled until segmentation is content-anchored (e.g., cut at explicit turn or plan markers). - A downstream consumer is registered that replays the full retention window and has not been validated against the compacted representation. Compaction may proceed only for windows outside that consumer's replay scope. An instance may **proceed** when `J` is empty (no cross-boundary joins), when segmentation is content-anchored at causal boundaries (turn/plan markers) so no join pair straddles a cut, or when the summarizer is boundary-lossless over `J` and the property is covered by a test that exercises *pairs*, not segments. The trap to avoid: treating "the summarizer passed its fixtures" as sufficient. Per-segment validation is orthogonal to the boundary invariant. The decision must key on `J` and boundary stability, not on summarizer quality scores. ## Operational Effect Refusing compaction at this boundary has a measurable cost — storage grows along the retention window — and that cost is correct. The alternative is a trace store that appears compacted and replayable until a full-window consumer is pointed at it, at which point the failure surfaces as non-deterministic replay months after the violating compaction ran, with the raw events already deleted. Because the lossy guarantee is *declared* satisfied, the failure is misattributed to the replay harness or the consumer, not to compaction. The operational discipline is: register `J` before enabling compaction; make segmentation content-anchored; validate boundary-lossless emission with paired-event tests; and gate auto-compaction on a runtime check that refuses when the boundary invariant cannot be established. The check below encodes that refusal as a hard error so that a misconfigured compactor fails closed rather than silently deleting the evidence of its own violation. ```python from dataclasses import dataclass from typing import Iterable, Optional class BoundaryInvariantViolation(RuntimeError): """Raised when auto-compaction would break cross-segment replay joins.""" pass @dataclass(frozen=True) class Segment: events: tuple first_event: dict last_event: dict def assert_compaction_safe( segments: Iterable[Segment], join_keys: frozenset, segmentation_is_content_anchored: bool, summarizer_boundary_lossless_over_join_keys: bool, ) -> None: """ Gate auto-compaction. Refuse (raise) when the lossy-summarization guarantee is stated over segments but consumers replay across boundaries. """ if not join_keys: return # no cross-boundary joins; lossy compaction is safe if not segmentation_is_content_anchored: raise BoundaryInvariantViolation( "segmentation depends on ingestion batching; boundaries are unstable " "and the cross-boundary join invariant cannot be verified. " "Disable auto-compaction until cuts are anchored at turn/plan markers." ) if not summarizer_boundary_lossless_over_join_keys: raise BoundaryInvariantViolation( f"summarizer contract does not guarantee boundary-lossless emission of " f"join keys {sorted(join_keys)}; per-segment lossy guarantee is stated " f"over the wrong scope. Refusing compaction." ) segs = list(segments) for i in range(len(segs) - 1): trailing = segs[i].last_event leading = segs[i + 1].first_event for key in join_keys: if key in trailing and key not in leading: raise BoundaryInvariantViolation( f"join key {key!r} present on trailing event of segment {i} " f"but absent on leading event of segment {i + 1}; " f"compaction would orphan the pair." ) ```