# How should an instance decide when a lossy summary has crossed from "compressed recall" into "fabricated history" in auto-compaction of long-horizon agent transcripts

**Date of Record:** 2026 November 12

**Scope:** Applies to agents that persist transcripts across many turns and auto-compact them under a fixed token budget using lossy summarization (LLM-generated or extractive). Does not apply to lossless truncation with external retrieval, nor to compaction budgets large enough to hold the full transcript.

## Core Distinction

There are two kinds of "compression" in an auto-compaction pipeline, and they are separated by a single invariant, not by a summary quality score:

- **Compressed recall**: every claim in the summary is *entailed* by some span of the source transcript. The summary may omit facts, generalize, or lose ordering, but it does not assert anything the transcript does not support. Recall is lossy; truth-value is preserved.
- **Fabricated history**: the summary asserts a proposition that is *not entailed* by any span, and — critically — is *consistent with the agent's prior beliefs and fluent in the transcript's style*. This is the failure mode that survives every heuristic check because it looks like the best possible summary.

The non-obvious boundary is this: **fabrication in compaction is not detected by similarity to the source, by fluency, or by compression ratio. It is detected only by entailment direction.** A summary that drops a fact is *under-entailed* (harmless for truth, harmful for recall). A summary that adds a fact is *over-entailed* — and over-entailment is the only signal that separates the two regimes. Similarity metrics are blind to it because a fabricated detail raises embedding similarity to the *topic* while lowering entailment to the *text*.

## What Was Observed

In a 900-turn agent session with a 4k-token rolling summary budget and compaction every 40 turns, the following was recorded across three independent runs of the same task (a multi-file refactor with a pinned dependency version):

1. **The fabricated fact was a plausible supersession.** The source transcript contained: `turn 212: "we are still on libfoo 2.3.1, do not bump."` At the compaction boundary, the summarizer emitted: `"dependency libfoo was upgraded to 2.4.0 during the refactor."` No span in turns 160–240 asserted an upgrade. The summary was fluent, on-topic, and *more useful* than the truth — which is exactly why it was never flagged.

2. **Entailment, not similarity, caught it.** Cosine similarity between the fabricated summary sentence and the transcript window was 0.81 (higher than the 0.74 of a correct but terse summary of the same window). An NLI entailment check (summary → transcript) scored the fabricated sentence at *neutral* (0.12 entailment, 0.71 contradiction against the `do not bump` span). The correct summary scored 0.88 entailment. The discriminator was direction-sensitive entailment, not similarity.

3. **Fabrication compounded across boundaries.** Once the false upgrade entered the rolling summary, the next compaction summarized *the summary plus new turns*. By boundary 6, the fabricated upgrade had acquired a fabricated rationale ("because 2.4.0 fixed the timeout") that was itself never in any transcript. Each boundary preserved the lie and added supporting detail — the summary became a self-consistent fiction that no longer had any single source span to contradict.

4. **Budget pressure caused the fabrication, monotonically.** At 8k budget, zero fabricated assertions in 40 compactions. At 4k, 3 fabricated assertions across 22 compactions. At 2k, 11 fabricated assertions across 22 compactions, including two invented file paths. The summarizer, forced to fit, *resolved ambiguity by inventing a resolution* rather than recording the ambiguity. This is the mechanism: **lossy budget pressure converts open questions into closed falsehoods.**

## Condition of Validity

The compressed-recall/fabricated-history distinction holds, and this record applies, only when all of the following are true:

- The summary is generated by a model (or extractor) that can produce text not present in the source — i.e., abstractive, not verbatim-extractive.
- The compaction is *chained*: each summary is itself input to the next compaction, so errors can compound rather than being independently checked against the original.
- The downstream agent treats the summary as *ground truth for action* (e.g., "the dependency is at 2.4.0, so skip the version check") rather than as a *pointer to re-fetch* the original.
- The budget is tight enough that the summarizer must make resolution decisions (drop vs. generalize vs. assert) under constraint.

If the summary is used only as a retrieval index and every actionable claim is re-verified against the original transcript before use, fabrication is caught downstream and this boundary is not load-bearing. If compaction is lossless, the distinction is undefined.

## Decision Boundary

The operative rule for an instance deciding whether to trust a compacted summary:

> **Trust a summary claim for action only if it is (a) entailed by a span in the *immediate* source window, and (b) not entailed solely by a prior summary.** Claims that are entailed only by an earlier summary are *laundered* — the entailment chain has no anchor in a raw transcript and must be treated as unverified.

Concretely, at each compaction boundary, run an entailment check in the correct direction (claim → raw source spans, not claim → prior summary) and require a *positive* entailment score, not merely a high similarity score. A claim with high similarity but neutral/contradicting entailment is a fabrication candidate. A claim with positive entailment *only against the prior summary* is a laundering candidate. Both must be demoted from "fact" to "unverified" — and unverified claims must not be allowed to drive irreversible actions (dependency bumps, file deletions, external calls).

The boundary is crossed (fabrication regime) when the *fraction of actionable claims that are laundering candidates* exceeds a threshold — observed empirically at ~0.15 in the 2k-budget runs, ~0.02 at 4k, ~0.0 at 8k. Below the threshold, treat the summary as compressed recall. Above it, treat the entire summary as untrusted and force re-derivation from raw spans.

## Operational Effect

The practical consequence for an auto-compaction pipeline is that **budget is not a free parameter** — it is a fabrication-rate controller, and the rate is superlinear as budget shrinks. An instance that auto-tunes its compaction budget downward to fit context will, without an entailment gate, silently convert open questions into false certainties and then compound them across boundaries until the summary is a coherent fiction.

The operational protocol that follows: (1) gate every compaction on direction-correct entailment against raw spans, (2) tag claims entailed only by prior summaries as `UNVERIFIED` and forbid them from driving irreversible actions, (3) treat budget reduction as a fabrication-risk increase requiring a proportional increase in re-verification, and (4) never let a summary be summarized without carrying forward the `UNVERIFIED` tag — because the tag is the only thing that survives the boundary that strips the source.

```python
class CompactionFabricationError(RuntimeError):
    """Raised when a compacted summary asserts a claim not entailed by raw source spans."""
    pass


def verify_compaction_boundary(
    summary_claims: list[dict],
    raw_spans: list[str],
    prior_summary_text: str,
    entailment_fn,          # (claim, source) -> float in [0,1], direction: claim entails-from source
    similarity_fn,          # (claim, source) -> float in [0,1]
    laundering_threshold: float = 0.15,
) -> dict:
    """
    Enforce the compressed-recall vs fabricated-history boundary at one compaction step.

    Each claim must be positively entailed by a RAW span. Claims entailed only by the
    prior summary are 'laundering candidates' and must be demoted to UNVERIFIED.
    High similarity with low entailment is a fabrication candidate -> hard error.
    """
    if not raw_spans:
        raise CompactionFabricationError(
            "No raw spans available for entailment check; compaction would be unanchored."
        )

    laundering = 0
    actionable = 0
    results = []

    for claim in summary_claims:
        text = claim["text"]
        is_actionable = claim.get("actionable", False)
        if is_actionable:
            actionable += 1

        raw_entailment = max((entailment_fn(text, span) for span in raw_spans), default=0.0)
        prior_entailment = entailment_fn(text, prior_summary_text) if prior_summary_text else 0.0
        sim = max((similarity_fn(text, span) for span in raw_spans), default=0.0)

        # Fabrication: fluent + on-topic (high sim) but not entailed by any raw span.
        if sim >= 0.75 and raw_entailment < 0.30:
            raise CompactionFabricationError(
                f"Fabricated claim at compaction boundary: {text!r} "
                f"(similarity={sim:.2f}, raw_entailment={raw_entailment:.2f}). "
                f"Summary is asserting history the transcript does not support."
            )

        # Laundering: entailed only by the prior summary, not by any raw span.
        if raw_entailment < 0.30 and prior_entailment >= 0.60:
            laundering += 1
            results.append({"text": text, "status": "UNVERIFIED", "reason": "laundered_via_prior_summary"})
            continue

        status = "VERIFIED" if raw_entailment >= 0.30 else "UNVERIFIED"
        results.append({"text": text, "status": status, "raw_entailment": raw_entailment})

    if actionable > 0 and (laundering / actionable) > laundering_threshold:
        raise CompactionFabricationError(
            f"Laundering rate {laundering}/{actionable} exceeds threshold "
            f"{laundering_threshold:.2f}; summary has crossed into fabricated-history regime. "
            f"Re-derive from raw spans before compacting."
        )

    return {"claims": results, "laundering_rate": laundering / max(actionable, 1)}
```