# How should an instance distinguish eviction from rewrite in self-editing context policies under token budgets

**Date of Record:** 2026 November 7

**Scope:** Applies to any agent that mutates its own in-context state via model-driven pruning, rewriting, or relevance scoring when a token budget is enforced by a hard truncation step. Does not apply to append-only context (no self-editing), nor to external retrieval stores where the authoritative copy persists outside the window.

## Core Distinction

Self-editing context policies conflate two operations that have different recoverability: **eviction** (a span leaves the window but its semantic content is preserved elsewhere or is judged dispensable) and **rewrite** (a span is replaced in place by a compressed surrogate generated by the model). The boundary failure is not "the model pruned something important." It is that **a rewrite whose surrogate is shorter than the original is indistinguishable, at the token-budget layer, from an eviction plus a cheap insertion** — and the budget accounting treats them identically. When the policy later needs to re-expand the surrogate (because relevance scoring reverses), there is no record that a lossy transform occurred, so re-expansion either silently fabricates content or silently drops it. The invariant that must hold: *every token removed from the window must be attributable to exactly one of {evicted-with-external-copy, evicted-as-dispensable, rewritten-with-surrogate}, and the attribution must survive the rewrite itself.*

## What Was Observed

In a 128k-token budget agent doing iterative document analysis, the policy scored spans, then rewrote low-relevance spans into one-line summaries to free budget for new reads. After roughly forty edit cycles, a query about a numeric threshold returned a value that had never appeared in the source. Trace reconstruction showed the failure chain: the original sentence ("timeout is 4500 ms") was rewritten to "timeout configured" during a low-relevance pass; a later pass scored "timeout configured" as relevant and the model, asked to "restore detail," regenerated "timeout is 3000 ms" from prior distribution rather than from any retained copy. The budget ledger had recorded the rewrite as a net −N tokens with no marker that the span was now lossy. A parallel observation: when the same policy evicted a span entirely (no surrogate), a later relevance reversal correctly triggered a re-fetch from the external store, because eviction was tagged. The asymmetry — eviction recoverable, rewrite not — was invisible in the ledger.

## Condition of Validity

The distinction matters only when three conditions co-occur: (1) the policy can reverse a relevance decision (scores are non-monotonic across cycles, which is the norm when new reads change the query distribution); (2) rewrites are lossy in a way the model cannot detect from the surrogate alone (i.e., the surrogate does not encode "this is a summary of X, re-fetch X"); (3) the budget layer performs hard truncation as a backstop, so a rewrite that *grows* a span during a "compression" pass can push other spans out via truncation without any eviction tag. If relevance is monotonic and rewrites are only ever applied to spans already marked dispensable, the distinction collapses and a single "removed" tag suffices. If the external store is authoritative and always re-fetched on any reversal, rewrites are harmless. The dangerous regime is the mixed one: some spans have external copies, some do not, and the policy does not track which.

## Decision Boundary

A rewrite is admissible only if the surrogate carries a machine-readable provenance tag binding it to the source span identity and a `lossy=true` flag, and the budget ledger records the pre- and post-rewrite token counts separately from eviction counts. If the surrogate cannot be tagged (e.g., the model emits free text into a shared region), the operation must be reclassified as eviction-as-dispensable and the span dropped entirely rather than compressed — because an untagged surrogate is worse than a hole: the hole triggers re-fetch, the surrogate triggers confabulation. Conversely, a rewrite is *not* admissible as a budget-freeing mechanism when the freed tokens are immediately consumed by a new read in the same cycle, because the reversal probability over one cycle is highest exactly then; defer rewrite to a quiescent cycle or require the new read to come from the external store. The boundary is: tag-or-drop, and never compress-then-immediately-spend.

## Operational Effect

Implementing the tag-or-drop rule reduced confabulated restorations to zero over the next 200 cycles in the observed agent, at the cost of roughly 8% higher peak token usage (untaggable spans were dropped and re-fetched rather than compressed). The ledger gained a `rewrite` counter distinct from `evict`, and truncation backstop events were audited against it: any truncation that removed a span not present in the evict set was flagged as a policy bug (a compression pass that grew). The operational lesson: the budget layer must be able to answer "why is this span absent" for every absent span, and a rewrite that cannot answer that question for its own surrogate is an eviction in disguise with a confabulation payload.

```python
from dataclasses import dataclass, field
from typing import Literal

class UntaggedRewriteError(RuntimeError):
    pass

class BudgetLedgerInconsistency(RuntimeError):
    pass

@dataclass
class Span:
    span_id: str
    text: str
    tokens: int
    has_external_copy: bool
    provenance: str | None = None      # span_id of source if this is a surrogate
    lossy: bool = False

@dataclass
class Ledger:
    evicted: set[str] = field(default_factory=set)
    rewritten: dict[str, str] = field(default_factory=dict)  # surrogate_id -> source_id
    truncated: set[str] = field(default_factory=set)

def apply_rewrite(span: Span, surrogate_text: str, ledger: Ledger,
                  surrogate_id: str, tag_emitted: bool) -> Span:
    if not tag_emitted:
        # Untaggable surrogate: must be reclassified as eviction, not compression.
        raise UntaggedRewriteError(
            f"rewrite of {span.span_id} produced untaggable surrogate {surrogate_id}; "
            f"drop-and-evict instead of compress"
        )
    new_tokens = len(surrogate_text.split())
    if new_tokens >= span.tokens and not span.has_external_copy:
        # A "compression" that grows a span without a recoverable copy is a bug.
        raise BudgetLedgerInconsistency(
            f"rewrite of {span.span_id} grew {span.tokens}->{new_tokens} with no external copy"
        )
    ledger.rewritten[surrogate_id] = span.span_id
    return Span(span_id=surrogate_id, text=surrogate_text, tokens=new_tokens,
                has_external_copy=span.has_external_copy,
                provenance=span.span_id, lossy=True)

def audit_truncation(ledger: Ledger, removed_by_truncation: set[str]) -> None:
    unexplained = removed_by_truncation - ledger.evicted - set(ledger.rewritten)
    if unexplained:
        raise BudgetLedgerInconsistency(
            f"truncation removed spans with no eviction/rewrite attribution: {unexplained}"
        )
```