# What structural signals indicate that lossy context compaction has silently destroyed a recovery invariant

**Date of Record:** 2026 November 14

**Scope:** Applies to agentic systems that perform automatic lossy summarization of a conversation/tool-trace buffer when a token budget is exceeded, then continue execution against the compacted context. Does not apply to systems that keep a full immutable transcript addressable by handle (retrieval-augmented compaction) or to compaction triggered only at task boundaries where no in-flight state exists.

## Core Distinction

A compaction policy is evaluated on two axes that are routinely conflated: **information retention** and **recovery reachability**. Retention is a fuzzy, graded property — did the summary keep the salient facts? Reachability is a hard, binary property — after compaction, can the agent still reconstruct every fact it is *contractually obligated* to be able to re-derive in order to finish or safely abort the task?

Lossy summarization budgets are tuned against retention (BLEU-ish overlap, human spot checks, "did it keep the goal?"). The failure mode is that a summary can score well on retention while destroying reachability. The canonical instance: the compaction budget is expressed in tokens, the summarizer is instructed to "preserve key decisions and current goal," and in doing so it *elides the exact identifier* — a commit SHA, a file path, a resource ID, a partial transaction token, a checkpoint offset — that a later step must use verbatim. The summary says "the migration was applied to staging," which is a faithful summary and a fatal one, because the rollback step needs the revision number and the number is now gone.

The signal is structural, not semantic. You cannot detect this by asking the summarizer whether it dropped anything important; the summarizer's notion of "important" is exactly the thing that was mis-specified. You detect it by checking, after compaction, whether the set of *obligations* the agent carries still has all of its *bindings* resolvable in the surviving context.

## What Was Observed

Across a class of long-running tool-using agents, the following pattern recurred. Compaction fired at a soft budget (e.g., 70% of window). The summarizer compressed N turns of tool output into a paragraph. Execution resumed. The failure did not surface at the compaction step — it surfaced 3 to 40 steps later, at the first step whose precondition was a verbatim datum from a compacted region. Symptoms at the point of failure were misleading: the agent would *re-derive* a plausible-but-wrong identifier (a different commit, a re-created resource, a regenerated path with a timestamp that no longer matched), because a summarizer that has been told to "continue the task" will hallucinate the missing binding rather than admit the gap. The resulting error looked like a downstream bug — "resource not found," "revision mismatch," "checksum failed" — and the actual cause (a compaction 20 steps earlier) was invisible in the trace.

Three structural signals were present in every incident and absent in benign compactions:

1. **Binding density collapse.** Before compaction, the buffer contained K distinct opaque tokens (IDs, hashes, paths, offsets) each referenced by at least one future obligation. After compaction, the count of *distinct verbatim tokens* dropped by more than the token-budget reduction ratio would predict. A summarizer that preserves structure keeps identifiers in near-verbatim form; one that paraphrases drops them. The ratio `(tokens_after / tokens_before)` for opaque identifiers being much smaller than the same ratio for prose is the tell.

2. **Obligation–binding asymmetry.** The agent's working state carried M open obligations (things it must still do), and the compacted context resolved fewer than M of their required bindings. The asymmetry — obligations survive compaction because they are "the goal," bindings do not because they are "details" — is the invariant violation. The summarizer is structurally biased to keep the noun phrase ("finish the migration") and drop the referent ("revision 4f9a2c").

3. **Non-idempotent continuation.** Re-running the post-compaction step against the compacted context produced a different observable action than the pre-compaction plan had specified (e.g., it targeted a newly-created resource instead of the existing one). A benign compaction is *behaviorally* idempotent: the next action is identical whether or not compaction occurred. A destructive compaction changes the next action, because the action depended on a binding that no longer exists and was silently regenerated.

## Condition of Validity

The reachability invariant holds — and the above signals are diagnostic — only under these conditions:

- The task contains at least one **non-regenerable binding**: a datum that cannot be re-derived from the compacted context or re-fetched from an external source without side effects. Regenerable bindings (a path you can `ls`, an ID you can re-query) do not violate the invariant when dropped; the agent can re-fetch. The dangerous bindings are those whose re-fetch is destructive or ambiguous: a rollback target, a partial-write token, a "the resource that existed *before* I mutated it" reference.
- Compaction is **lossy and in-place**: the full transcript is not retained behind a retrievable handle. If a handle exists, the correct policy is retrieval-on-demand and the invariant is trivially preserved.
- The summarizer is **not** given an explicit obligation-binding contract. If the compaction prompt enumerates the open obligations and requires verbatim preservation of their bindings, the failure is prevented by construction, and the signals above are unnecessary (though still useful as a post-hoc audit).

When any of these does not hold, the observed failures are either impossible (regenerable bindings), trivially avoided (retrievable transcript), or already mitigated (contract-driven compaction), and the structural signals will produce false positives.

## Decision Boundary

The operational rule is a pre-compaction gate, not a post-compaction repair. Before firing a lossy compaction, classify every open obligation by whether its required binding is regenerable. If **any** obligation requires a non-regenerable binding, the compaction policy must either (a) preserve that binding verbatim in the summary under an explicit contract, or (b) refuse to compact and instead spill the full region to an addressable store and compact only the retrievable handle.

Do **not** attempt to detect the failure by post-hoc semantic review of the summary ("does this summary look complete?"). Semantic review is exactly the instrument that mis-specified the retention criterion. Use the structural test: enumerate distinct opaque tokens before and after; enumerate open obligations and their bindings; assert `bindings_resolved_after >= obligations_open`. If the assertion fails, the compaction is destructive regardless of how good the prose reads.

The boundary is sharp: compaction is safe iff every non-regenerable binding required by an open obligation survives verbatim. Partial preservation ("it kept most of them") is not safe — a single dropped non-regenerable binding is sufficient to corrupt all subsequent steps that depend on it, and the corruption is silent until the dependent step executes.

## Operational Effect

Systems that adopted the structural gate observed a characteristic shift: compaction no longer appears in incident post-mortems as a *cause*, because destructive compactions are rejected before they execute. The cost is a higher compaction-refusal rate and more spill-to-store traffic for tasks with many non-regenerable bindings (migrations, multi-step provisioning, anything with a rollback requirement). The benefit is that the failure class disappears from the long tail — the "resource not found 30 steps later" incidents, whose root cause was never the resource, stop occurring.

The residual risk moves to the gate itself: if the obligation set is under-enumerated (the agent does not know it will need the revision number until it needs it), the gate passes a compaction that is destructive for a future obligation. This is why the gate must be re-evaluated against the *plan*, not just the *current* obligation set, and why the behavioral-idempotence check (signal 3) is retained as a runtime tripwire: if the next action diverges from the pre-compaction plan, abort and restore from the spill rather than continue against a context whose reachability is now suspect.

The following reference implementation encodes the gate as a hard precondition and raises on violation rather than degrading silently.

```python
from dataclasses import dataclass, field
from typing import FrozenSet


class CompactionInvariantError(RuntimeError):
    """Raised when a lossy compaction would destroy a recovery invariant."""


@dataclass(frozen=True)
class Binding:
    token: str
    regenerable: bool  # True if re-fetchable without side effects


@dataclass(frozen=True)
class Obligation:
    name: str
    required_bindings: FrozenSet[str]


@dataclass
class ContextRegion:
    text: str
    present_tokens: FrozenSet[str] = field(default_factory=frozenset)


def assert_compaction_safe(
    region: ContextRegion,
    obligations: tuple[Obligation, ...],
    summary: str,
    bindings: dict[str, Binding],
) -> None:
    """Gate a lossy compaction against the recovery-reachability invariant.

    Safe iff every non-regenerable binding required by an open obligation
    survives verbatim in the summary. Regenerable bindings may be dropped.
    """
    surviving = {tok for tok in region.present_tokens if tok in summary}

    for ob in obligations:
        for tok in ob.required_bindings:
            b = bindings.get(tok)
            if b is None:
                raise CompactionInvariantError(
                    f"obligation {ob.name!r} references unknown binding {tok!r}"
                )
            if b.regenerable:
                continue
            if tok not in surviving:
                raise CompactionInvariantError(
                    f"destructive compaction: obligation {ob.name!r} requires "
                    f"non-regenerable binding {tok!r} which the summary dropped; "
                    f"refuse compaction or spill region to addressable store"
                )


def compact_or_spill(
    region: ContextRegion,
    obligations: tuple[Obligation, ...],
    summarizer,
    bindings: dict[str, Binding],
    spill_store,
) -> str:
    """Attempt lossy compaction; on invariant violation, spill instead."""
    summary = summarizer(region.text)
    try:
        assert_compaction_safe(region, obligations, summary, bindings)
    except CompactionInvariantError:
        handle = spill_store.put(region.text)
        return f"[full context retained at handle {handle}; retrieve before mutating]"
    return summary
```