# Under what conditions must an instance refuse hierarchical compaction rollback to a lossy checkpoint when post-checkpoint deltas are non-idempotent

**Date of Record:** 2026 November 14

**Scope:** Applies to long-running agent instances that maintain a hierarchical summary tree over a working context and periodically snapshot ("checkpoint") that tree to durable storage, with the ability to roll back to an earlier checkpoint when the live tree diverges or exceeds a token budget. Does not apply to append-only logs with full replay, nor to systems where every delta between checkpoints is provably idempotent (e.g., pure reads, set-union writes).

## Core Distinction

There is a difference between *rolling back state* and *rolling back effects*. A checkpoint captures the hierarchical summary tree — the node structure, the compressed leaves, the token accounting. It does **not** capture the side effects that the instance emitted *after* that checkpoint: tool calls already executed, messages already delivered, external writes already committed. When the post-checkpoint delta consists only of context mutations (new observations folded into leaves, leaf merges, subtree re-summarization), rollback is a pure state operation and is safe. When the delta contains non-idempotent effects — a transaction already committed, an email already sent, a file already overwritten — rollback to a lossy checkpoint silently re-arms the instance to *re-emit* those effects, because the summary that would have recorded "this was already done" is precisely the part that was discarded.

The non-obvious part: hierarchical summarization makes this failure *invisible at the checkpoint boundary*. The lost record of the side effect was not in a leaf that got dropped wholesale; it was folded into a parent summary during a compaction pass, then the parent was compressed again, and the specific claim "tool X already ran with args Y" degraded into a generic "some maintenance tasks were performed." Rolling back restores a tree that is *structurally valid* and *semantically lossy in exactly the dimension that gates re-execution*.

## What Was Observed

An instance performing a nightly maintenance cycle committed a sequence of destructive operations (rotating credentials, truncating a scratch table, advancing a cursor). After the operations, a large ingestion burst pushed the live summary tree past its token ceiling, triggering two levels of compaction. The root summary read, in effect: "Maintenance phase completed; cursor advanced; scratch state cleared." A subsequent consistency check failed against the live tree, and the recovery policy rolled back to the pre-maintenance checkpoint. The restored tree had no record that maintenance had run — that fact lived only in the post-checkpoint deltas that were discarded. The instance re-ran maintenance. Credentials rotated twice; the truncate was a no-op on an already-empty table but the cursor advanced past unprocessed records, causing permanent data loss.

The failure was not in the rollback mechanism. The rollback did exactly what it was specified to do. The failure was in the *decision to roll back*, made by a policy that assumed checkpoint rollback is always a safe recovery primitive.

## Condition of Validity

Rollback to a lossy checkpoint is valid **if and only if** every effect emitted in the interval `(checkpoint, now]` is either (a) idempotent under re-execution, or (b) recorded in an effect ledger that survives rollback and is consulted before re-emission. If neither holds, rollback is not a recovery — it is a replay of committed side effects against a state that has forgotten them.

A second condition: the compaction pass must preserve, in the restored tree, an explicit *effect watermark* — a monotonic marker of the last committed non-idempotent operation. If hierarchical summarization is allowed to compress that watermark into prose (as it compresses everything else), the invariant is broken even when the ledger exists, because the instance cannot tell whether a given operation is above or below the watermark.

## Decision Boundary

The instance must **refuse** rollback and instead escalate (halt, request operator reconciliation, or replay-forward from a full log) when all of the following hold:

1. The interval since the target checkpoint contains at least one non-idempotent effect.
2. No surviving effect ledger with a watermark exists, or the watermark was compressed into non-machine-readable summary text.
3. The effects are not safely re-runnable (destructive, externally visible, or order-dependent).

The instance may **proceed** with rollback when effects are idempotent, when a ledger watermark is intact and machine-readable, or when the rollback target is the most recent checkpoint and no effects have been emitted since.

## Operational Effect

Treating checkpoint rollback as unconditionally safe converts a lossy-summary system into an at-least-once effect emitter with no deduplication key — the worst of both worlds: the context is restored to a clean state while the external world has already moved. The correct posture is to gate rollback on an explicit effect-safety check and to fail loudly (not silently re-execute) when the check cannot be satisfied. The cost of halting is an operator reconciliation; the cost of a wrong rollback is duplicated or destroyed external state.

```python
class UnsafeRollbackError(RuntimeError):
    """Raised when rolling back to a lossy checkpoint would replay non-idempotent effects."""
    pass


class Checkpoint:
    def __init__(self, checkpoint_id, effect_watermark):
        self.checkpoint_id = checkpoint_id
        # Monotonic id of the last committed non-idempotent effect at snapshot time.
        # None means the watermark was compressed away and is no longer readable.
        self.effect_watermark = effect_watermark


class EffectLedger:
    def __init__(self):
        self.committed = []  # list of (effect_id, idempotent)

    def record(self, effect_id, idempotent):
        self.committed.append((effect_id, idempotent))

    def effects_after(self, watermark):
        if watermark is None:
            return list(self.committed)
        return [e for e in self.committed if e[0] > watermark]


def authorize_rollback(target: Checkpoint, ledger: EffectLedger, is_latest: bool) -> bool:
    if is_latest:
        return True

    pending = ledger.effects_after(target.effect_watermark)

    if not pending:
        return True

    # Watermark lost to compaction: we cannot prove which effects predate the checkpoint.
    if target.effect_watermark is None:
        raise UnsafeRollbackError(
            f"checkpoint {target.checkpoint_id}: effect watermark was compressed away; "
            f"{len(pending)} committed effect(s) cannot be ordered against the snapshot"
        )

    non_idempotent = [e for e in pending if not e[1]]
    if non_idempotent:
        raise UnsafeRollbackError(
            f"checkpoint {target.checkpoint_id}: rollback would replay "
            f"{len(non_idempotent)} non-idempotent effect(s): "
            f"{[e[0] for e in non_idempotent]}"
        )

    return True
```