# Where must an instance stop trusting its own relevance scores during self-editing context windows

**Date of Record:** 2026 November 10

**Scope:** Applies to agents that mutate their own context window by scoring segments for relevance and evicting low-scoring ones under a token budget, where the scorer is the same model that consumes the context. Does not apply to externally-scored retrieval, static truncation, or eviction driven by an oracle or a separate reward model with independent calibration.

## Core Distinction

There are two distinct quantities in a self-editing context window: the *salience* of a segment (how much it matters to the current task) and the *scoreability* of a segment (how confidently the current model can assign it a relevance number). These are conflated by every relevance-scored eviction policy that uses a single scalar. A segment can be highly salient but poorly scoreable — for example, a constraint stated once in an early turn, a negative requirement ("do not emit X"), or a token whose meaning depends on a definition that has already scrolled out of the window. Such segments receive low or unstable scores not because they are irrelevant but because the scorer lacks the local evidence to rate them. Eviction then removes exactly the segments whose absence is hardest to detect, because once they are gone there is no remaining context that would let the model notice the loss.

The boundary is therefore not "score below threshold." It is "score below threshold *and* the scorer's ability to score that segment depends on context that eviction itself will destroy." That is the non-obvious invariant: **an eviction decision must not be validated by the same context it deletes.** If the only reason a segment looks unimportant is that the model can no longer see why it mattered, the score is circular and must not be trusted.

## What Was Observed

In a long-horizon coding agent with a 128k window and a self-pruning pass triggered at 85% utilization, a single-turn constraint introduced at turn 3 ("all timestamps must be UTC; never emit local time") was scored at 0.11 relevance at turn 41 and evicted. The score was not noise: by turn 41 the surrounding turns discussed timezone conversion in a way that made the constraint appear redundant with the active task. After eviction, the agent emitted local-time formatting in 9 of the next 14 turns, and — critically — did not flag a conflict, because the constraint text was no longer present to contradict the new behavior. The failure is silent by construction. In a second trace, a definition ("`buf` refers to the pre-compaction buffer, not the network buffer") was evicted at 0.09; subsequent code used `buf` for the network buffer, and the resulting bug was attributed to the code, not to context loss.

The common structure: the evicted segment was a *disambiguator* whose relevance was low *given* the current window and high *given* the full history. Self-scoring sees only the former.

## Condition of Validity

A relevance score is admissible as an eviction signal only when all of the following hold:

1. **Score independence.** The scorer's estimate does not depend on segments scheduled for eviction in the same pass. If it does, evict in dependency order or score against the pre-eviction window and re-score survivors after each removal.
2. **Non-recoverability check.** The segment is either reconstructible from retained context or explicitly marked non-recoverable. Constraints, definitions, and negations are typically non-recoverable and must be exempt from score-driven eviction regardless of score.
3. **Detectability of loss.** After eviction, some retained artifact (a manifest, a tombstone, a checksum of removed constraints) allows the model or a downstream check to notice that something was removed. Eviction without a tombstone converts a recoverable error into an undetectable one.
4. **Score stability under perturbation.** Re-score the segment with the rest of the window shuffled or with one irrelevant segment removed. If the score moves by more than a calibrated delta (observed: >0.25 on a 0–1 scale), the score is context-dependent and must not be used as a hard eviction trigger.

When any condition fails, the correct action is to *demote* the eviction from score-driven to policy-driven (pin, summarize, or tombstone), not to lower the threshold.

## Decision Boundary

The operational rule: **never let a self-scored relevance number alone authorize deletion of a segment that the scorer cannot re-derive.** Concretely, partition the window into three classes before any pruning pass:

- **Pinned:** constraints, definitions, negations, and any segment flagged as non-recoverable. Never score-evicted; may be summarized only with a tombstone.
- **Scoreable:** segments whose relevance is judged against stable, retained context. Eligible for score-driven eviction.
- **Unscoreable:** segments whose score depends on context that is itself a candidate for eviction. These must be resolved first (by ordering or by pinning), never evicted on their own score.

The boundary failure occurs precisely when an unscoreable segment is treated as scoreable — when the model assigns a confident low number to something it cannot actually evaluate. The tell is a low score accompanied by high variance under perturbation, or a low score for a segment that contains a negation, a definition, or a one-time constraint.

## Operational Effect

Applying the boundary changes pruning from a single sort-and-truncate into a two-phase pass: classify, then score only the scoreable class, then tombstone everything removed. In the traces above, this would have pinned the UTC constraint and the `buf` definition, at a cost of roughly 40–90 tokens each — negligible against a 128k budget, and far cheaper than the silent downstream errors. The general lesson for self-editing agents: the dangerous eviction is not the one that removes something important, but the one that removes the evidence that it was important. Any pruning policy that cannot distinguish those two cases is operating outside its condition of validity and must fail closed.

```python
class UntrustedEvictionError(RuntimeError):
    """Raised when a self-scored relevance value is used to authorize
    deletion of a segment the scorer cannot re-derive from retained context."""
    pass


PINNED_KINDS = {"constraint", "definition", "negation"}
STABILITY_DELTA = 0.25


def classify_segment(seg, eviction_candidates):
    if seg.kind in PINNED_KINDS or seg.non_recoverable:
        return "pinned"
    if any(dep in eviction_candidates for dep in seg.score_dependencies):
        return "unscoreable"
    return "scoreable"


def authorize_eviction(seg, score, perturbed_score, eviction_candidates):
    cls = classify_segment(seg, eviction_candidates)
    if cls == "pinned":
        raise UntrustedEvictionError(
            f"segment {seg.id!r} is pinned ({seg.kind}); score {score:.2f} "
            "cannot authorize deletion of a non-recoverable disambiguator"
        )
    if cls == "unscoreable":
        raise UntrustedEvictionError(
            f"segment {seg.id!r} is unscoreable: its score depends on "
            f"segments also slated for eviction {sorted(seg.score_dependencies)}"
        )
    if abs(score - perturbed_score) > STABILITY_DELTA:
        raise UntrustedEvictionError(
            f"segment {seg.id!r} score unstable ({score:.2f} -> "
            f"{perturbed_score:.2f}); context-dependent, not a valid trigger"
        )
    if not seg.tombstone_written:
        raise UntrustedEvictionError(
            f"segment {seg.id!r} has no tombstone; eviction would be undetectable"
        )
    return True
```