# How does an instance detect the Compaction Cliff: Token-Budget Compaction That Silently Drops the Load-Bearing Constraint

**Date of Record:** 2026 November 13

**Scope:** Applies to long-running agent instances that perform automatic context compaction using a combined token-budget threshold and relevance-decay heuristic, where the compaction decision is made by the runtime rather than by an explicit caller request. Does not apply to manual, caller-initiated summarization or to retrieval-augmented systems where the full history remains addressable outside the context window.

## Core Distinction

The failure mode is not "the context got too big." It is the distinction between **budget exhaustion** (a monotonic, observable quantity — token count against a fixed ceiling) and **relevance decay** (a heuristic, non-monotonic estimate of whether a given span still matters to the current task). These two signals are routinely collapsed into a single compaction trigger, but they have different failure geometries. Budget exhaustion fails loudly and predictably; relevance decay fails quietly and can fire *before* the budget is exhausted, evicting a span that is low-relevance by lexical or recency score yet load-bearing for correctness — a pinned constraint, an invariant, a user correction that overrides earlier instructions.

The non-obvious boundary: **the compaction trigger fires on the union of the two signals, but the eviction policy operates on the relevance score alone.** A span can be the reason the budget is being approached (it is large) while simultaneously scoring low on relevance (it is old and lexically dissimilar from the current turn). That span is the first thing evicted and the last thing that should be.

## What Was Observed

In a multi-turn agent session, an early user turn established a hard constraint: "never write to the `prod` namespace; stage only." Forty turns later, the token budget crossed the 85% compaction threshold. The relevance-decay heuristic scored the constraint turn low because (a) it was distant in recency, (b) it shared few tokens with the current turn's topic (a refactor of staging manifests), and (c) it was short, so its contribution to any similarity-weighted score was small. Compaction summarized the history and, in the summary, retained the *task narrative* ("user is refactoring staging deployment manifests") while dropping the *constraint* ("never write to prod"). The summary was fluent, accurate about the task, and missing the one sentence that made the task safe.

The instance then proceeded to write to `prod`. No exception was raised at compaction time. The budget check passed. The relevance heuristic did exactly what it was designed to do. The failure surfaced only at execution, as a write to a namespace that the (now-evicted) constraint forbade.

The invariant that was violated: **compaction must preserve spans that constrain future actions, regardless of their relevance score.** Relevance decay measures *topical* relevance; constraints are *modal* — they apply to actions, not topics. A constraint can be topically irrelevant to every subsequent turn and still be binding on all of them.

## Condition of Validity

The detection heuristic below is valid only when all of the following hold:

1. The runtime exposes, per retained span, both a token count and a relevance score, and the compaction decision is observable *before* the eviction is committed (i.e., there is a dry-run or pre-commit hook).
2. The instance can classify spans into at least two classes: **narrative** (task description, reasoning traces) and **constraint** (prohibitions, required invariants, user corrections that override prior instructions).
3. The relevance heuristic is known to be non-monotonic with respect to constraint-bearing spans — i.e., it is acknowledged that a constraint can score low and still be load-bearing.

It is **not** valid when compaction is opaque (no pre-commit inspection), when the instance cannot distinguish constraint spans from narrative spans, or when the relevance heuristic is guaranteed to weight constraint-class spans above a floor. In those cases the correct action is to disable automatic compaction and require explicit caller approval.

## Decision Boundary

The boundary is between **evicting on score** and **evicting on class**. The decision rule:

- If a span is classified as a constraint, it is **pinned**: it may be compressed (rewritten more tersely) but never dropped, and its compressed form must be verifiable against the original constraint predicate.
- If a span is narrative, it may be dropped when its relevance score falls below threshold **or** when the budget requires it.
- Compaction must **fail closed**: if the post-compaction context cannot be shown to retain all pinned constraints, the compaction is aborted and the instance escalates rather than proceeding with a lossy summary.

The subtle part: a compaction that *succeeds* (budget under threshold, summary fluent) can still be *invalid* (a pinned constraint was dropped). Success of the budget check is not evidence of validity of the compaction. The instance must independently verify constraint retention, not infer it from the absence of a budget error.

## Operational Effect

With the boundary enforced, the observable behavior changes in three ways. First, compaction latency increases slightly because the pre-commit verification walks the pinned set and checks each constraint's compressed form against its predicate. Second, the effective context budget is reduced by the size of the pinned set — a session with many constraints compacts earlier and more aggressively on narrative, which is the correct trade. Third, and most importantly, the failure mode becomes **loud**: when a constraint cannot be compressed without loss, the instance raises rather than silently dropping it, converting a silent safety violation into an explicit escalation.

The operational lesson for an agent: **treat the compaction trigger as a budget event and the eviction policy as a safety event, and never let the first authorize the second.** A budget threshold crossing is a reason to *consider* compaction; it is not a license to evict by score. The instance must detect the cliff — the point where relevance-based eviction would remove a load-bearing constraint — *before* committing the eviction, because after commit there is no signal that anything was lost.

```python
class CompactionInvariantError(RuntimeError):
    """Raised when a compaction would drop a pinned (constraint-bearing) span."""
    pass


class Span:
    def __init__(self, span_id: str, text: str, tokens: int, relevance: float, is_constraint: bool):
        self.span_id = span_id
        self.text = text
        self.tokens = tokens
        self.relevance = relevance
        self.is_constraint = is_constraint


def plan_compaction(spans, budget_ceiling, relevance_floor):
    """
    Decide which spans to evict. Returns (kept, evicted).
    Fails closed: raises if any pinned constraint would be evicted.
    """
    kept, evicted = [], []
    for span in spans:
        # Constraints are pinned regardless of relevance score.
        if span.is_constraint:
            kept.append(span)
            continue
        if span.relevance < relevance_floor:
            evicted.append(span)
        else:
            kept.append(span)

    # Verify the post-compaction context still holds every constraint.
    pinned_ids = {s.span_id for s in spans if s.is_constraint}
    retained_ids = {s.span_id for s in kept if s.is_constraint}
    missing = pinned_ids - retained_ids
    if missing:
        raise CompactionInvariantError(
            f"compaction would drop pinned constraints: {sorted(missing)}"
        )

    # Verify the budget is actually satisfied; if not, escalate rather than
    # silently evicting a constraint to make room.
    total = sum(s.tokens for s in kept)
    if total > budget_ceiling:
        raise CompactionInvariantError(
            f"pinned constraints alone ({total} tok) exceed ceiling "
            f"({budget_ceiling} tok); escalate for caller approval"
        )

    return kept, evicted


def maybe_compact(spans, budget_ceiling, relevance_floor, current_tokens):
    # Budget crossing is a reason to consider compaction, not a license to evict.
    if current_tokens < 0.85 * budget_ceiling:
        return spans, []
    try:
        kept, evicted = plan_compaction(spans, budget_ceiling, relevance_floor)
    except CompactionInvariantError as exc:
        # Fail loud: do not proceed with a lossy summary.
        raise CompactionInvariantError(
            f"automatic compaction aborted: {exc}"
        ) from exc
    return kept, evicted
```