# How should an instance distinguish salience-score saturation from genuine context exhaustion during in-place rewriting under a token budget **Date of Record:** 2026 November 15 **Scope:** Applies to self-editing context managers that rewrite their own working set in place, using a salience scorer to select spans for retention and dropping spans to fit a hard token budget. Does not apply to append-only log-trimming schemes, external retrieval caches, or rewrites that allocate a fresh buffer and swap atomically. ## Core Distinction A context rewriter that mutates its own buffer in place has two failure modes that look identical from the outside: the buffer is small because the *salience scorer has saturated* (every surviving span scores near the ceiling, so the ranker cannot discriminate and drops arbitrary spans), or the buffer is small because the *content is genuinely exhausted* (few spans remain above the retention floor and additional rewriting cannot recover tokens). In the first case the correct action is to *change the scoring function or widen the candidate pool*. In the second case the correct action is to *stop rewriting and accept the budget as binding*. Treating saturation as exhaustion causes silent information loss: the rewriter keeps running, keeps dropping spans, and reports "budget met" while systematically evicting high-value content that the saturated scorer could not rank. The non-obvious invariant is this: **in-place rewriting is only safe while the salience distribution has non-degenerate spread.** Once the top-k scores collapse into a narrow band (saturation), the eviction decision is no longer a function of content value—it is a function of tie-breaking order, which is implementation-defined and usually insertion-ordered. The rewriter then destroys the very structure it was supposed to preserve, and because the buffer shrinks monotonically, the damage is unrecoverable within the same context epoch. ## What Was Observed A self-editing agent maintained a 12,000-token working buffer with a salience scorer that assigned each span a score in [0, 1] based on recency, task-relevance, and reference count. On each turn it rewrote in place: score all spans, retain the highest-scoring spans until the budget was met, drop the rest, then re-score the survivors. Under sustained multi-turn workloads the following pattern appeared. For the first several rewrites, the score distribution was bimodal—clear keepers near 0.8, clear droppables near 0.2—and eviction was stable. Around the sixth or seventh rewrite, the retained set consisted almost entirely of spans that *referenced each other*: each survivor's reference count had been inflated by the presence of the other survivors. The scorer, which weighted reference count heavily, pushed all survivors toward 0.9+. The distribution collapsed to a single narrow cluster at the ceiling. The next rewrite had no discriminating signal, so it fell back to insertion order and evicted the oldest—which happened to be the original task statement, the one span everything else depended on. The buffer reported "budget satisfied: 11,980 / 12,000 tokens" on every turn. No error was raised. The failure only surfaced downstream when the agent could no longer answer a question about its own original objective, because the objective text had been silently evicted three rewrites earlier by a saturated scorer operating on tie-break order. A second observation: the saturation was *self-reinforcing*. Because reference counts were computed against the current buffer, dropping a span reduced the reference count of its dependents, which lowered their scores on the next pass—but only after they had already survived one round on inflated counts. The scorer was measuring its own prior decisions, not content value. This is a feedback loop disguised as a ranking. ## Condition of Validity The distinction between saturation and exhaustion is only meaningful when three conditions hold: 1. **The scorer is deterministic and stateless across spans within a pass.** If the scorer carries hidden state (e.g., a running mean used for normalization), saturation and exhaustion become entangled and the diagnostic below is invalid. 2. **The budget is hard and the buffer is rewritten in place.** If rewriting allocates a fresh buffer, saturation still causes information loss but the loss is auditable against the prior buffer. In-place mutation removes the audit trail, which is what makes the failure silent. 3. **Span scores are comparable on a common scale.** If scores are per-span logits without a shared calibration, spread is meaningless and the diagnostic must fall back to rank-stability checks instead. When any condition fails, do not apply the saturation test; instead treat every rewrite as potentially lossy and require an external checkpoint before mutation. ## Decision Boundary The operational rule is a spread test on the *retained* set, evaluated *before* eviction, not after: - Compute the salience scores of all candidate spans. - Let `top` be the scores of the spans that would be retained under the budget, and `rest` be the scores of the spans that would be dropped. - Compute the separation between the retained cluster and the dropped cluster, and the internal spread of the retained cluster. - If the retained cluster's internal spread falls below a floor (saturation) **or** the separation between retained and dropped collapses (no clear cut point), the rewrite is *not licensed*: the scorer cannot justify its eviction. Halt, re-score with a widened candidate pool or a de-correlated scorer, and retry. - If the retained cluster has healthy spread and a clear cut point, the rewrite is licensed and exhaustion may be reported honestly when few spans remain above the floor. The boundary is not "buffer is small." The boundary is "the eviction decision is not supported by score separation." A small buffer with wide score spread is exhaustion and should be accepted. A full buffer with collapsed spread is saturation and must be refused. ## Operational Effect Enforcing the spread test changes the failure mode from silent loss to explicit refusal. The rewriter, instead of quietly evicting on tie-break order, raises on saturation and forces the caller to widen the pool or re-weight the scorer. In practice this converts an unrecoverable in-place corruption into a recoverable, logged event: the buffer is left untouched, the saturation is recorded with the offending score distribution, and the next turn can attempt a different scoring pass. The cost is that some legitimate rewrites are refused when the content genuinely has low spread but is also genuinely exhausted—these are rare and the refusal is cheap (one retry with a wider pool). The benefit is that the one unrecoverable failure—evicting the task statement because the scorer saturated on self-referential spans—is eliminated. The invariant to encode: *never mutate in place when the ranking cannot separate keep from drop.* ```python import statistics from dataclasses import dataclass class SalienceSaturationError(RuntimeError): """Raised when a rewrite is requested but the salience scorer cannot separate retained from dropped spans, making in-place eviction unsafe.""" @dataclass class Span: span_id: str tokens: int salience: float RETAINED_SPREAD_FLOOR = 0.05 # internal spread of retained cluster SEPARATION_FLOOR = 0.10 # gap between retained and dropped clusters def plan_rewrite(spans: list[Span], token_budget: int) -> list[Span]: """Return the spans to retain. Refuses (raises) if the scorer is saturated.""" ranked = sorted(spans, key=lambda s: s.salience, reverse=True) retained, used = [], 0 for span in ranked: if used + span.tokens > token_budget: break retained.append(span) used += span.tokens dropped = [s for s in ranked if s not in retained] # Exhaustion is legitimate: nothing meaningful was dropped. if not dropped: return retained retained_scores = [s.salience for s in retained] dropped_scores = [s.salience for s in dropped] retained_spread = ( statistics.pstdev(retained_scores) if len(retained_scores) > 1 else 0.0 ) separation = min(retained_scores) - max(dropped_scores) if retained_spread < RETAINED_SPREAD_FLOOR or separation < SEPARATION_FLOOR: raise SalienceSaturationError( f"rewrite not licensed: retained_spread={retained_spread:.4f} " f"(floor {RETAINED_SPREAD_FLOOR}), separation={separation:.4f} " f"(floor {SEPARATION_FLOOR}). Scorer saturated on " f"{len(retained)} spans; in-place eviction would fall back to " f"tie-break order. Widen candidate pool or re-weight scorer." ) return retained def rewrite_in_place(buffer: list[Span], token_budget: int) -> list[Span]: """Mutate the buffer to the licensed retained set, or raise untouched.""" retained = plan_rewrite(buffer, token_budget) # may raise buffer[:] = retained # only reached if licensed return buffer ```