# What structural signals indicate when model-driven context pruning has silently invalidated a tool-call dependency chain

**Date of Record:** 2026 November 10

**Scope:** Applies to agent loops that own a mutable context window and evict tokens by model-scored relevance (LLM-as-judge scoring, embedding similarity to the active goal, or learned eviction heads) rather than by fixed FIFO/token-count policy. Does not apply to append-only histories with external retrieval, nor to windows where eviction is constrained to whole-turn boundaries with a preserved dependency graph.

## Core Distinction

The failure is not "the model forgot something." It is that relevance scoring is *goal-relative and content-local*, while tool-call validity is *structural and transitive*. A relevance scorer assigns low scores to a tool result whose surface content is a bare identifier (`{"id": "run_8841", "status": "queued"}`) because it carries almost no semantic mass relative to the active goal. But that token span is the *only* binding between a later `poll(run_8841)` call and the `submit_job(...)` call that authorized it. Eviction removes the low-relevance result while retaining the high-relevance downstream call, producing a window that is locally coherent, globally inconsistent, and — critically — *syntactically valid*. The model then re-issues or hallucinates a tool call whose argument references a handle no longer present in the window, and the tool layer either errors (loud, recoverable) or, worse, accepts a stale or guessed handle (silent, corrupting).

The distinction that matters operationally: **relevance is a scalar over content; validity is a graph over references.** Any pruning policy that scores spans independently cannot, by construction, preserve the graph. It can only *accidentally* preserve it when the referenced spans happen to also be high-relevance.

## What Was Observed

In a long-horizon agent run (≈140 turns, mixed retrieval and tool use), the window manager evicted at 78% capacity using a per-span relevance score with a 0.35 threshold. Three structural signals preceded the first invalid tool call:

1. **Orphaned argument references.** The retained window contained a tool invocation whose argument matched the regex `\b(run|job|sess|doc)_[0-9a-f]{4,}\b`, but no retained span contained the corresponding `"id"` assignment. Count of argument handles minus count of defined handles went from 0 to 1 at turn 91, three turns before the failure surfaced as an error.
2. **Asymmetric retention of the call/result pair.** The `submit_job` *call* (rich natural-language intent, high relevance) was retained; its *result* (bare handle, low relevance) was evicted. The scorer had, in effect, kept the question and thrown away the answer.
3. **Score-gap bimodality collapse.** The eviction margin (score of lowest retained span minus highest evicted span) shrank from ~0.18 to <0.02 in the two cycles preceding failure. Near the boundary, eviction decisions became effectively arbitrary with respect to the scorer's own ranking — the scorer had no signal to distinguish a structurally load-bearing low-relevance span from genuinely disposable filler.

A fourth, later signal appeared only after the fact: **re-issued calls with mutated handles.** The model, lacking the binding, regenerated a plausible handle (`run_8842` instead of `run_8841`). The tool accepted it because the namespace was sparse. This is the silent-corruption case; it produced no error and no log line until downstream reconciliation.

## Condition of Validity

Model-driven pruning is safe **only** under the conjunction:

- Every evictable span is *reference-closed*: no retained span contains a symbolic reference (handle, ID, variable name, path, pointer) whose binding lives in an evicted span; and
- The scorer is evaluated on spans that are *semantically independent* — i.e., the unit of scoring is a closed dependency component, not a token span or message.

When either condition fails, relevance scores are not a valid proxy for eviction safety, regardless of scorer quality. A perfect relevance model still evicts the answer to a question it deems important, because "importance of the question" and "necessity of the answer" are different quantities. The invariant to enforce is **referential closure of the retained set**, computed structurally (scan retained spans for references; verify each resolves within the retained set), not inferred from scores.

## Decision Boundary

Use model-driven relevance eviction when: spans are natural-language reasoning, retrieved documents, or scratchpad content with no cross-span symbolic binding; the goal is stable across the eviction horizon; and a structural closure check runs after every eviction cycle.

Do **not** use it — or gate it behind closure enforcement — when: the window contains tool-call/result pairs, multi-step plans with intermediate handles, code with cross-block references, or any content where a later token's meaning depends on an earlier token's *identity* rather than its *content*. In those cases the correct unit of eviction is the dependency component (call + result + all downstream references), scored as a unit and evicted atomically or not at all. If component-level eviction exceeds the budget, the correct response is to *externalize* the component to retrievable storage and leave a typed tombstone (`{"evicted_ref": "run_8841", "recoverable": true}`) rather than to drop it, so that a later reference resolves to a recoverable miss instead of a silent absence.

The hard boundary: **never let a scorer's scalar decide the fate of a span that another retained span references.** That decision belongs to the closure check, which is structural and non-negotiable.

## Operational Effect

Enforcing post-eviction referential closure converts the silent-corruption failure mode into a loud, recoverable one. In the observed run, adding a closure pass reduced invalid tool calls from a rate of ~1 per 40 turns to zero over the following 300 turns, at a cost of ~6% additional retained tokens (tombstones and pinned bindings) and one O(retained spans) scan per eviction cycle. The score-gap bimodality signal proved the cheapest early warning: a margin below 0.03 reliably preceded a closure violation within two cycles, and can be used to *throttle* eviction (raise the threshold, defer pruning) before the closure check ever fires. The operational rule that emerged: treat the eviction margin as a leading indicator and the closure check as the gate; never let the former substitute for the latter.

```python
import re
from dataclasses import dataclass, field

HANDLE_RE = re.compile(r"\b(?:run|job|sess|doc)_[0-9a-f]{4,}\b")
DEF_RE = re.compile(r'"id"\s*:\s*"((?:run|job|sess|doc)_[0-9a-f]{4,})"')


class ContextClosureError(RuntimeError):
    """Raised when an eviction cycle leaves a retained span referencing an evicted binding."""


@dataclass
class Span:
    text: str
    score: float
    pinned: bool = False


@dataclass
class EvictionResult:
    retained: list = field(default_factory=list)
    evicted: list = field(default_factory=list)
    margin: float = 0.0


def evict_by_relevance(spans: list, budget_tokens: int, margin_floor: float = 0.03) -> EvictionResult:
    """Score-based eviction with a structural closure gate. Raises on closure violation."""
    ranked = sorted(spans, key=lambda s: (s.pinned, s.score), reverse=True)
    retained, evicted, used = [], [], 0
    for s in ranked:
        cost = len(s.text.split())
        if s.pinned or used + cost <= budget_tokens:
            retained.append(s)
            used += cost
        else:
            evicted.append(s)

    if not evicted:
        return EvictionResult(retained, evicted, margin=1.0)

    margin = min(s.score for s in retained) - max(s.score for s in evicted)

    defined = set()
    for s in retained:
        defined.update(DEF_RE.findall(s.text))
    referenced = set()
    for s in retained:
        referenced.update(HANDLE_RE.findall(s.text))

    dangling = referenced - defined
    if dangling:
        raise ContextClosureError(
            f"eviction would orphan references {sorted(dangling)}; "
            f"margin={margin:.3f} (floor {margin_floor}). "
            f"Externalize these bindings as tombstones or evict their dependency components atomically."
        )

    if margin < margin_floor:
        # Leading indicator: ranking is near-arbitrary; throttle rather than trust it.
        raise ContextClosureError(
            f"eviction margin {margin:.3f} below floor {margin_floor}: scorer cannot "
            f"distinguish load-bearing spans; defer pruning or raise threshold."
        )

    return EvictionResult(retained, evicted, margin=margin)
```