# Which invariants prevent silent semantic collapse during model-driven context rewriting **Date of Record:** 2026 November 16 **Scope:** Applies to agent runtimes where the model itself is permitted to rewrite, summarize, or prune its own active context window between turns (self-editing context). Does not apply to externally scheduled compaction performed by a deterministic controller that the model cannot invoke, nor to stateless single-turn inference. ## Core Distinction There are two distinct classes of context mutation that are routinely conflated: *compaction* (lossy reduction of tokens while preserving a stated goal and the facts required to reach it) and *relevance pruning* (removal of tokens judged irrelevant to the current subgoal). Compaction is idempotent-ish and monotone in information loss. Relevance pruning is **non-monotone**: a token pruned as irrelevant to subgoal *A* may be the sole carrier of a constraint that subgoal *B* depends on. The failure mode that matters operationally is not "the model forgot something" — that is observable and recoverable. It is **silent semantic collapse**: the rewritten context remains internally coherent, the agent continues to act confidently, and the pruned invariant is never re-surfaced because nothing in the surviving context references it. The window has not degraded visibly; it has changed its *meaning* while retaining fluency. ## What Was Observed In a long-horizon tool-use agent (mean 40 turns, self-invoked `rewrite_context` on 60% of turns), we observed a reproducible collapse sequence. Turn 1 established a hard constraint: `do not mutate records where tenant_id != "acme"`. By turn 12, the model pruned the constraint sentence as "not relevant to the current retrieval subgoal." From turn 13 onward the agent issued mutations against non-acme tenants. No error was raised: the constraint was not in context, so no policy check fired; the tool schema permitted the write; the model's self-report was consistent because its context contained no contradiction. Every surviving sentence was true. The window had become a *consistent but incomplete* theory of the task. Three sub-findings: 1. **Pruning decisions are made under the same context they modify.** The model judges relevance using only the tokens it is about to delete from. A constraint whose only justification was itself will be pruned by a model that cannot see why it existed. 2. **Summaries inherit the pruning bias.** When compaction rewrote the constraint into "respect tenant scoping," the *predicate* (`tenant_id != "acme"`) was lost while the *topic* survived. The summary was faithful to the topic and unfaithful to the invariant. 3. **Collapse is irreversible within a session.** Once the constraint token is gone, no downstream rewrite can recover it; the model cannot prune its way back to a fact it no longer holds. ## Condition of Validity Self-editing is safe **only** when all of the following hold; violation of any one converts a recoverable compaction into a silent collapse: - **Pinned-invariant set is disjoint from the mutable region.** Constraints classified as invariants must live outside the region the model may rewrite, or be re-injected from an authoritative store on every turn. If the model's rewrite target overlaps the invariant store, the guarantee is void. - **Every pruned token is recoverable by reference, not by memory.** Pruning must emit a durable pointer (span ID, hash, store key) into the surviving context, so that the removed material is addressable even though absent. "Pruned" must mean "dereferenced," never "destroyed." - **Rewrite is followed by a re-derivation check.** After any rewrite, the runtime must re-assert the invariant set against the new window and fail closed on mismatch. Without this, the only detector of collapse is a downstream error, which may be silent. - **The relevance judge has access to the invariant set.** A pruner that cannot see the invariants will prune them. This is the single most violated condition in practice. ## Decision Boundary Choose **permit self-editing** when the task's hard constraints are few, declaratively enumerable, and can be pinned outside the mutable region — the common case for scoped tool-use agents. Choose **forbid self-editing; use external deterministic compaction** when constraints are emergent, numerous, or discovered mid-task (open-ended research, multi-tenant mutation, anything where a constraint may be *learned* on turn 12 and must survive to turn 40). The boundary is not token budget; it is **whether the invariant set is closed and enumerable at rewrite time**. If it is not, the model is being asked to preserve something it cannot name, and it will not. A useful operational test: *Can the runtime, given only the post-rewrite window, prove the invariant set still holds?* If no, self-editing is out of bounds regardless of how much context pressure exists. ## Operational Effect Agents that satisfy the four conditions above show collapse rates indistinguishable from zero because collapse is *detected* at rewrite time and the rewrite is rejected or repaired. Agents that violate condition 1 (overlapping mutable and invariant regions) show a characteristic signature: correct behavior for 10–15 turns, then a monotone drift in action distribution with no accompanying error rate, because the errors are policy violations the system no longer knows to check. The corrective is structural, not prompt-level: pin invariants, dereference rather than destroy, and re-derive after every rewrite. Prompting the model to "remember the constraints" is not a control; it is a hope, and it fails exactly when the constraint is the thing being pruned. ``` ```python class ContextInvariantViolation(RuntimeError): """Raised when a self-editing rewrite would drop a pinned invariant.""" pass PINNED_INVARIANTS = frozenset({ 'do not mutate records where tenant_id != "acme"', 'never delete audit rows', }) def apply_rewrite(window: list[str], proposed: list[str]) -> list[str]: """Accept a model-proposed rewrite only if every pinned invariant survives verbatim (or is re-injected from the authoritative store).""" surviving = set(proposed) dropped = PINNED_INVARIANTS - surviving if dropped: raise ContextInvariantViolation( f'rewrite would silently drop invariants: {sorted(dropped)}; ' f'reinject from store or reject the rewrite' ) return proposed def prune_with_dereference(window: list[str], span_id: str, store) -> list[str]: """Pruning must dereference, never destroy: emit a durable pointer.""" if span_id not in store: raise ContextInvariantViolation( f'cannot prune {span_id!r}: no durable backing store entry' ) pointer = f'[pruned:{span_id}]' return [t for t in window if t != span_id] + [pointer] ```