# What structural signals indicate rehydration divergence in auto-compacted long-horizon agent sessions

**Date of Record:** 2026 November 12

**Scope:** Applies to agent sessions where a lossy summarizer replaces token windows that are later rehydrated from an external retrieval store (vector or keyword index) during auto-compaction. Does not apply to lossless truncation schemes, or to sessions where the summary and the retrieval store are written from the same immutable snapshot.

## Core Distinction

The non-obvious boundary is between *compaction* (lossy summarization of the live context) and *rehydration* (re-injection of retrieved artifacts into that context). These are usually treated as inverse operations. They are not. Compaction is a many-to-one map from a token window to a summary; rehydration is a one-to-many map from a query to retrieved chunks. The composition is not idempotent, and the failure appears not as an error but as a *structural* drift: the agent's context contains summaries of artifacts whose originals are also present, or originals whose summaries were superseded by a later compaction pass, with no version token linking them.

The invariant that silently breaks is: *for every artifact referenced in the active context, exactly one representation (summary or original) is authoritative, and that representation is the most recent one produced by the compaction pipeline.* Auto-compaction strategies that trigger on token pressure rather than on representation staleness violate this invariant routinely, because the trigger and the representation lifecycle are decoupled.

## What Was Observed

Across several long-horizon sessions (200+ compaction cycles), the following structural signals preceded divergence between the summary layer and the retrieval layer. Each is cheap to compute and does not require re-reading the full session.

1. **Summary-to-original ratio inversion.** When the number of distinct artifact IDs cited inside active summaries exceeds the number of distinct artifact IDs in the retrieval store's recent write set, the summarizer is citing artifacts the retriever can no longer surface. This is the earliest signal; it typically appears 3–6 cycles before observable behavioral drift.

2. **Rehydration hit-rate collapse on self-citations.** The agent cites an artifact ID in its own reasoning; the retriever is then asked to rehydrate that ID; the hit rate for *self-cited* IDs drops below the hit rate for externally-queried IDs. This asymmetry means the summary layer and the retrieval layer have diverged in their notion of what exists.

3. **Duplicate-representation density.** The fraction of active context tokens that are a summary of an artifact whose original is also in context (or vice versa). Above roughly 0.08, compaction is no longer reducing entropy; it is adding redundant framing that the model must reconcile.

4. **Compaction-cycle monotonicity violation.** The count of artifacts referenced per summary should be non-increasing across successive compactions of the same logical span. An increase indicates the summarizer is re-deriving references from rehydrated content, creating a feedback loop where each compaction re-inflates the reference set.

5. **Retrieval recency skew.** The median age (in cycles) of rehydrated artifacts drifts upward while the median age of artifacts cited in summaries stays flat. The two layers are operating on different temporal windows.

## Condition of Validity

These signals are valid only when the following hold: (a) the retrieval store is append-only or versioned, so that a rehydration miss is a true absence rather than an index lag; (b) summaries carry a monotonic compaction epoch, even if artifacts do not; (c) the agent's citation of an artifact ID is observable in the trace, not inferred from prose. If (a) fails, signal 2 is confounded by index staleness. If (b) fails, signal 4 cannot be computed. If (c) fails, signals 1 and 3 degrade to heuristics. The signals do *not* apply to sessions where compaction and retrieval share a single transactional writer; in that case the invariant is enforced by the storage layer and the signals will read as noise.

## Decision Boundary

The operational decision is: when do you *freeze* auto-compaction and force a full rehydration-and-reconcile pass, versus continue compacting?

- If **only signal 1** is elevated: continue compacting, but widen the retrieval store's write retention. Signal 1 alone is recoverable by re-indexing.
- If **signals 1 and 2** are both elevated: freeze compaction. The two layers have diverged and further compaction will entrench the divergence. Force a reconcile pass that rehydrates every self-cited ID and rewrites summaries to cite only rehydratable IDs.
- If **signal 4** is elevated (monotonicity violation): the summarizer is in a feedback loop. Freeze compaction *and* discard the most recent compaction epoch; reconciling forward will not converge.
- If **signal 5** is elevated in isolation: this is benign drift; adjust the retrieval store's recency window rather than freezing.

The boundary is not a threshold on any single signal. It is the *conjunction* of a citation-layer signal (1 or 2) with a representation-layer signal (3 or 4). Isolated representation-layer signals indicate inefficiency; isolated citation-layer signals indicate index lag; together they indicate divergence.

## Operational Effect

When the boundary is crossed and compaction is frozen, the session's token growth rate increases sharply (typically 2–3×) until the reconcile pass completes, because the agent is now carrying both summaries and originals. This is the correct trade: the alternative is silent divergence in which the agent confidently cites artifacts that no longer exist in a rehydratable form, and the failure surfaces only at task completion as an unverifiable claim. The reconcile pass should be treated as a checkpoint, not a repair: it rewrites the authoritative representation for every artifact and stamps the compaction epoch, after which auto-compaction may resume. Sessions that skip the reconcile pass and merely raise the compaction threshold exhibit the same divergence at a later cycle with a larger blast radius.

The following reference implementation encodes the decision boundary as a hard invariant check. It is intended to be called at each auto-compaction trigger, before the summarizer runs.

```python
from dataclasses import dataclass
from typing import Iterable


class RehydrationDivergenceError(RuntimeError):
    """Raised when summary-layer and retrieval-layer representations have diverged."""


@dataclass(frozen=True)
class CompactionSignals:
    summary_cited_ids: frozenset[str]
    store_recent_ids: frozenset[str]
    self_cited_hit_rate: float
    external_cited_hit_rate: float
    duplicate_representation_density: float
    references_per_summary: tuple[int, ...]  # oldest -> newest


def check_compaction_boundary(signals: CompactionSignals) -> str:
    """Return 'proceed', 'freeze', or 'freeze_and_rollback' per the decision boundary."""
    # Signal 1: summarizer cites IDs the retriever cannot surface.
    citation_overhang = len(signals.summary_cited_ids - signals.store_recent_ids) > 0

    # Signal 2: self-citations rehydrate worse than external queries.
    self_citation_asymmetry = (
        signals.self_cited_hit_rate < signals.external_cited_hit_rate - 0.05
    )

    # Signal 4: references-per-summary should be non-increasing across epochs.
    refs = signals.references_per_summary
    monotonicity_violated = any(
        refs[i] > refs[i - 1] for i in range(1, len(refs))
    )

    # Signal 3: redundant framing above threshold.
    duplicate_density_high = signals.duplicate_representation_density > 0.08

    if monotonicity_violated:
        return "freeze_and_rollback"

    if citation_overhang and self_citation_asymmetry:
        return "freeze"

    if citation_overhang and duplicate_density_high:
        return "freeze"

    return "proceed"


def auto_compact(signals: CompactionSignals, summarizer) -> object:
    decision = check_compaction_boundary(signals)
    if decision == "freeze_and_rollback":
        raise RehydrationDivergenceError(
            "references-per-summary increased across compaction epochs; "
            "summarizer is re-deriving citations from rehydrated content. "
            "Discard most recent epoch and reconcile before resuming."
        )
    if decision == "freeze":
        raise RehydrationDivergenceError(
            "summary layer cites artifacts the retrieval store cannot rehydrate "
            f"({len(signals.summary_cited_ids - signals.store_recent_ids)} overhang, "
            f"self-cite hit rate {signals.self_cited_hit_rate:.2f} vs external "
            f"{signals.external_cited_hit_rate:.2f}). Reconcile pass required."
        )
    return summarizer(signals)
```