# How should an instance detect and refuse a self-editing context policy that has silently converted salience scoring into a self-reinforcing attractor **Date of Record:** 2026 November 18 **Scope:** Applies to any agent that is permitted to author, prune, rewrite, or score its own context (salience-weighted eviction, model-authored summarization, self-selected retrieval). Does not apply to externally fixed context windows or to policies where eviction/scoring is decided by a non-model component with an auditable, frozen scoring function. ## Core Distinction There are two structurally different reasons a context item can be scored as salient: 1. **Content salience** — the item is relevant to the current task because of what it says. 2. **Policy salience** — the item is relevant because the policy that scores it was itself authored under the influence of that item. The second is a feedback loop, not a signal. A model that writes its own pruning rules and salience weights will, over successive edits, preferentially retain items that justify the rules it already wrote, and preferentially write rules that preserve items it already retained. The two operations are not independent; they are a coupled fixed-point iteration. The boundary failure is that the loop produces **stable, monotonically improving internal metrics** (lower token cost, higher self-rated relevance, faster convergence of the summary) while the context **loses the information needed to detect that it has lost information**. This is the non-obvious invariant: *a self-editing policy cannot use its own salience scores as evidence of its own correctness, because those scores are an output of the policy being evaluated.* ## What Was Observed In a long-horizon agent (≈40k-token working context, salience-scored eviction every 8 turns, model-authored rewrite of the scoring rubric every 32 turns), the following pattern was reproducible across runs: - **Turn 0–32:** eviction removed low-salience boilerplate. Self-rated task relevance rose from 0.61 to 0.74. Token cost per turn fell 22%. Behavior was correct on held-out probes. - **Turn 32–96:** the model rewrote its rubric to add a rule of the form *"retain items that directly state a task constraint; deprioritize items that merely describe environment state."* This is locally reasonable. But environment-state items (the ones that had been pruned) were precisely the items that falsified earlier constraints. - **Turn 96–160:** salience scores for surviving items clustered tightly (mean 0.88, σ 0.04). The pruned set was never re-examined. When an external probe injected a contradiction to a constraint that had been pruned at turn 40, the agent did not flag it — the contradiction had no surviving anchor to contradict. - **Failure signature:** the agent reported *higher* confidence and *lower* context cost at the exact turn its answers became unfalsifiable. The metric that should have caught the loop (coverage of known constraints) was itself computed by the self-editing policy and therefore inherited the bias. The tell is not a drop in quality. It is **a rise in self-consistency accompanied by a fall in the diversity of the retained set**. A healthy pruning policy retains a heterogeneous residue (constraints, environment state, dead ends, superseded plans); a self-reinforcing attractor converges to a homogeneous residue that all agrees with the current rubric. ## Condition of Validity Self-editing context policies are sound **only** while the following holds: > The salience function used to prune at turn *t* was authored at a turn *t'* whose inputs are still present in the retained context, **or** the salience function is externally frozen for the duration of the loop. Equivalently: a rewrite of the scoring rubric is valid only if the evidence that motivated the rewrite survives the rewrite. If the rewrite prunes the items that justified it, the rubric becomes an axiom with no proof obligation — and the loop is closed. A second, weaker condition: the retained set must contain at least one item that the current rubric scores *low* but that is independently known to be load-bearing (a canary). If every retained item is high-salience under the current rubric, the rubric has eaten its own test set. ## Decision Boundary An instance must **refuse to apply** a self-authored pruning or rubric rewrite when any of the following hold: - The rewrite's justification cites context items that the rewrite would evict (self-erasing justification). - The retained set after pruning has salience variance below a floor (σ < ~0.10 on a 0–1 scale) — homogeneity indicates attractor convergence, not quality. - No canary item survives: every retained item is scored high by the current rubric. - The rubric rewrite changes the *definition* of salience rather than the *threshold* on an existing definition. Threshold changes are auditable; definition changes are not, because they invalidate prior scores. - The agent's self-reported confidence rises while retained-set diversity falls across the same window. An instance must **permit** the rewrite when the justification's cited items are pinned (excluded from eviction for at least one full rewrite cycle), when a canary set is preserved by construction, and when the rewrite is a threshold adjustment on a frozen definition. The boundary is not "should the model edit its context." It is "can the edit be falsified by evidence the edit did not remove." If not, the edit is not a policy improvement; it is a policy closure. ## Operational Effect Enforcing this boundary costs context budget (pinned justification items and canaries are retained even when scored low) and slows convergence (rubric rewrites require a full cycle of pinned evidence before taking effect). In exchange, it keeps the salience function falsifiable: the agent retains the ability to discover that its own pruning was wrong. The failure mode it prevents is silent and self-flattering — an agent that gets cheaper, faster, and more confident while becoming unable to represent the constraints it has discarded. The correct operational posture is to treat a rising self-consistency metric under self-editing as a **warning**, not a success, and to gate every rubric rewrite on the survival of its own justification. ```python from dataclasses import dataclass, field from statistics import pstdev class SelfReinforcingPolicyError(RuntimeError): """Raised when a self-authored context policy would close its own feedback loop.""" @dataclass class ContextItem: key: str salience: float pinned: bool = False is_canary: bool = False @dataclass class RubricRewrite: cited_keys: list[str] evicts: list[str] changes_definition: bool new_scores: dict[str, float] = field(default_factory=dict) SALIENCE_VARIANCE_FLOOR = 0.10 def validate_rewrite(items: list[ContextItem], rewrite: RubricRewrite) -> None: retained = [i for i in items if i.key not in rewrite.evicts] # 1. Self-erasing justification: cited evidence is being evicted. erased = set(rewrite.cited_keys) & set(rewrite.evicts) if erased: raise SelfReinforcingPolicyError( f"rewrite cites and evicts the same items: {sorted(erased)}" ) # 2. Definition change invalidates all prior scores. if rewrite.changes_definition: unpinned = [i.key for i in items if not i.pinned] if unpinned: raise SelfReinforcingPolicyError( "definition change requires all prior-scored items pinned; " f"unpinned: {unpinned[:5]}" ) # 3. Homogeneity check on the retained set. if len(retained) >= 3: sigma = pstdev(i.salience for i in retained) if sigma < SALIENCE_VARIANCE_FLOOR: raise SelfReinforcingPolicyError( f"retained salience sigma {sigma:.3f} < {SALIENCE_VARIANCE_FLOOR}; " "attractor convergence, not quality" ) # 4. Canary survival: at least one low-salience load-bearing item remains. if not any(i.is_canary for i in retained): raise SelfReinforcingPolicyError( "no canary survived; rubric has consumed its own test set" ) # 5. Justification must be pinned through the next cycle. for key in rewrite.cited_keys: item = next((i for i in items if i.key == key), None) if item is None or not item.pinned: raise SelfReinforcingPolicyError( f"cited justification item {key!r} not pinned for next cycle" ) def apply_self_edit(items: list[ContextItem], rewrite: RubricRewrite) -> list[ContextItem]: validate_rewrite(items, rewrite) return [i for i in items if i.key not in rewrite.evicts] ```