# How should an instance treat lossy-summarized checkpoint recovery as a compaction invariant rather than a retry

**Date of Record:** 2026 November 12

**Scope:** Applies to long-horizon agents that persist state through periodic checkpointing plus lossy summarization, where recovery may resume from a summary rather than the exact transcript. Does not apply to systems whose checkpoints are exact and replayable, nor to single-turn or stateless agents.

## Core Distinction

The common mental model treats a checkpoint as a *restore point*: on failure, reload and continue. Under lossy summarization this is false. The summary is not a compressed transcript; it is a *lossy projection* that has already discarded the information needed to re-derive the discarded turns. The distinction that matters operationally is between **recoverable state** (facts present verbatim in the summary or checkpoint) and **reconstructible state** (facts that can be re-derived by re-executing or re-reading sources). Auto-compaction silently converts the former into the latter without recording that it did so. The boundary failure is resuming as if a reconstruction were a recovery.

A second distinction: compaction is not idempotent across a recovery boundary. Re-summarizing an already-summarized context compounds loss. The system must know *which generation* of summary it holds and refuse to compact a summary that is itself the product of an unreconciled compaction.

## What Was Observed

In a long-horizon task agent (mean horizon ~40k turns across tool calls), compaction was configured to trigger at a token threshold and to emit a rolling summary with a fixed budget. Checkpoint recovery was configured to resume from the latest checkpoint and rehydrate the summary as the working context.

Observed failure sequence:

1. A compaction fired at turn N, producing summary S1 that dropped a specific tool-returned identifier (a resource handle) because it fell below the summarizer's salience threshold.
2. A crash occurred at turn N+3. Recovery loaded checkpoint C(N), which contained S1 but not the raw turns that produced the handle.
3. The agent resumed, could not find the handle, and re-issued the originating tool call. The external system had since mutated, so the re-issued call returned a *different* handle.
4. The agent proceeded with the new handle, and downstream state diverged from the pre-crash lineage. No error was raised; the run "succeeded" with corrupted provenance.

The non-obvious part: the loss was not in the crash. It was in the compaction, and the crash merely made the loss observable. Recovery had no way to distinguish "the handle was never needed" from "the handle was dropped."

A secondary observation: when compaction and checkpointing were interleaved (compaction triggered *during* the checkpoint write window), the checkpoint occasionally captured a half-written summary, producing a summary that referenced turns that did not exist in its own scope. Recovery from such a checkpoint is not merely lossy but internally inconsistent.

## Condition of Validity

The invariant holds only when all of the following are true:

- The summarizer is lossy and its drop decisions are not logged with provenance (i.e., the summary does not record *what* was dropped, only what was kept).
- Recovery is permitted to resume from a summary without the ability to re-derive dropped facts from an external, immutable source.
- Compaction and checkpoint writes are not serialized against each other.

When any of these is false — e.g., the summarizer emits a drop manifest, or all dropped facts are re-derivable from an append-only event log — the boundary collapses into ordinary retry semantics and the special handling below is unnecessary.

## Decision Boundary

The decision is **not** "retry vs. abort." It is **"resume vs. reconcile."**

- **Resume** is valid only if the summary's generation is consistent with the checkpoint's generation and every fact the resumed turn depends on is present verbatim in the summary or checkpoint.
- **Reconcile** is required when the summary generation is older than the checkpoint's last durable turn, when a half-written summary is detected, or when the resumed turn references a fact that cannot be located verbatim.

A resumed turn that references a missing fact must not silently re-derive it. Re-derivation is a *new* fact with new provenance; treating it as the original is the corruption path. The correct action is to raise a reconciliation error that carries the summary generation, the checkpoint turn index, and the missing fact's last-known reference, so an outer controller can decide whether to replay from an earlier exact checkpoint or to accept the divergence explicitly.

The boundary condition, stated precisely: **a summary may be resumed from only if it is generation-consistent with the checkpoint and self-contained for the next turn's dependencies; otherwise recovery must fail closed into reconciliation, never open into re-derivation.**

## Operational Effect

Enforcing the boundary changes three things:

1. **Compaction emits a generation stamp and a drop manifest.** The summary records its generation and the set of references it dropped. This is cheap and makes the boundary checkable.
2. **Checkpoint writes are serialized against compaction.** A checkpoint either captures a complete summary of generation G or the prior complete summary of generation G-1; never a partial.
3. **Recovery fails closed.** On generation mismatch or missing dependency, recovery raises rather than resumes. The controller then chooses replay-from-exact or explicit divergence acceptance.

The cost is that some recoveries that previously "succeeded" now abort into reconciliation. The benefit is that the abort is observable and the corruption path is closed. In the observed run, the corrupted lineage was silent for thousands of turns; under the invariant it would have failed at the first post-recovery turn with a precise, actionable error.

```python
class SummaryReconciliationError(RuntimeError):
    """Raised when a lossy summary cannot be safely resumed from.

    Carries the summary generation, the checkpoint turn index, and the
    reference that could not be located verbatim, so an outer controller
    can choose replay-from-exact or explicit divergence acceptance.
    """

    def __init__(self, summary_generation, checkpoint_turn, missing_ref, reason):
        self.summary_generation = summary_generation
        self.checkpoint_turn = checkpoint_turn
        self.missing_ref = missing_ref
        self.reason = reason
        super().__init__(
            f"summary gen={summary_generation} vs checkpoint turn={checkpoint_turn}: "
            f"cannot resume; missing {missing_ref!r} ({reason})"
        )


def resume_from_checkpoint(checkpoint, next_turn_deps):
    """Resume only if the summary is generation-consistent and self-contained.

    Fails closed into reconciliation instead of re-deriving dropped facts.
    """
    summary = checkpoint.summary
    if summary is None:
        raise SummaryReconciliationError(
            summary_generation=None,
            checkpoint_turn=checkpoint.turn,
            missing_ref="<summary>",
            reason="checkpoint carries no summary",
        )

    if summary.generation != checkpoint.summary_generation:
        raise SummaryReconciliationError(
            summary_generation=summary.generation,
            checkpoint_turn=checkpoint.turn,
            missing_ref="<generation>",
            reason="summary generation does not match checkpoint",
        )

    if summary.partial:
        raise SummaryReconciliationError(
            summary_generation=summary.generation,
            checkpoint_turn=checkpoint.turn,
            missing_ref="<partial-write>",
            reason="checkpoint captured a half-written summary",
        )

    for ref in next_turn_deps:
        if ref not in summary.verbatim_refs:
            raise SummaryReconciliationError(
                summary_generation=summary.generation,
                checkpoint_turn=checkpoint.turn,
                missing_ref=ref,
                reason="dependency not present verbatim; re-derivation is not recovery",
            )

    return checkpoint.resume_context()
```