# Why does an instance of a self-editing context policy fail to reclaim tokens when pruning stale reasoning traces under a hard budget

**Date of Record:** 2026 November 13

**Scope:** Applies to agent runtimes where the model itself selects which prior reasoning traces (chain-of-thought, scratchpads, tool-call rationales) to evict in order to fit a token budget, and where eviction decisions are made by the same model that produced the traces. Does not apply to policies where pruning is driven by an external, trace-independent heuristic (e.g., recency, fixed TTL, or a separately-trained classifier) or where the budget is soft and overflow is tolerated.

## Core Distinction

A pruning decision has two separable costs: the *decision cost* (the tokens the model must read and emit to decide what to prune) and the *reclamation* (the tokens freed by the eviction). Most analyses assume the decision is cheap relative to reclamation, so any eviction that removes a stale trace is net-positive. The boundary failure is that the decision itself is generated *inside the same context window that is under pressure*, and the decision text is itself a new reasoning trace that becomes eligible for future pruning. When the model is asked to prune at the moment the budget is nearly exhausted, the decision overhead and the newly emitted rationale can consume more tokens than the eviction reclaims, producing a policy that *converges toward the budget ceiling without ever crossing below it* — a stable, self-sustaining steady state at the exact token limit.

The non-obvious part: the failure is not "the model prunes badly." A model can prune *correctly* — selecting genuinely stale traces — and still fail to reclaim budget, because the act of selecting costs what the selection frees. Correctness of selection and effectiveness of reclamation are orthogonal, and the policy conflates them.

## What Was Observed

In a runtime with a 32k-token window and a hard budget of 24k, the model was invoked with a meta-instruction to identify and drop stale reasoning traces whenever utilization exceeded 22k. Under sustained tool-loop workloads the following pattern reproduced across runs:

- Utilization climbed to 22k, triggering a prune pass.
- The prune pass read a summary of candidate traces (~600 tokens of index), emitted a rationale selecting ~3 traces (~400 tokens of decision text), and evicted traces totaling ~1,800 tokens.
- Net reclamation: ~800 tokens. Utilization fell to ~21.2k.
- The emitted rationale was appended as a new trace. Within two subsequent tool turns, utilization returned to 22k.
- The cycle repeated. Over a 40-turn run the policy executed 11 prune passes, never dropped below 20.9k, and the *sum of decision overhead* (index + rationale) exceeded the *sum of net reclamation* by a factor of ~1.4.

The characteristic signature is a utilization trace that oscillates in a tight band just below the trigger threshold, with prune-pass frequency increasing over time as the decision rationales themselves accumulate. The policy appears to be working (traces are being dropped) while the budget headroom monotonically erodes.

A second observation: when the model was given a *larger* budget slack at decision time (trigger at 20k instead of 22k), net reclamation turned positive and the oscillation disappeared. The failure is slack-dependent, not model-capability-dependent.

## Condition of Validity

The failure holds when all of the following are true:

1. **Decision-in-band**: the prune decision is generated while utilization is within the decision-overhead of the budget ceiling. Formally, if `overhead` is the token cost of index + rationale and `headroom = budget - utilization`, the failure is active when `headroom < overhead + ε`.
2. **Self-eligibility**: the emitted rationale is itself a reasoning trace subject to future pruning, so decision artifacts feed back into the candidate set.
3. **Hard ceiling**: the budget is enforced as a hard limit, so the policy cannot borrow against future turns; it must reclaim *now* or truncate.
4. **Stale-trace scarcity**: the genuinely stale traces remaining are small relative to the decision overhead — i.e., the policy has already pruned the cheap wins and is now paying full decision cost to reclaim marginal traces.

The failure does **not** hold when the trigger fires with slack greater than overhead (condition 1 violated), when decision rationales are excluded from the prunable set (condition 2 violated), or when the budget is soft (condition 3 violated).

## Decision Boundary

The operational rule is a comparison at trigger time, not a judgment about trace staleness:

```
reclaimable = sum(size(t) for t in stale_candidates) - size(index) - expected_rationale_size
trigger_prune  iff  reclaimable > overhead_margin AND headroom < trigger_threshold
```

Where `overhead_margin` is a configured floor (observed safe value: ≥ 1.5× expected rationale size). The boundary is not "is this trace stale?" but "does evicting it, *including the cost of deciding to evict it*, move utilization below the trigger threshold by more than the margin?" If not, the correct action is to **defer the prune pass** and instead take a non-model-driven action (truncate oldest tool output, summarize with a fixed template, or drop the lowest-recency trace) — because a model-driven prune at zero headroom is net-negative.

The counterintuitive corollary: a self-editing policy should *refuse to self-edit* when the budget is tightest, and should prefer to prune when it has the most slack. This inverts the naive trigger (prune when full) into (prune when there is room to prune).

## Operational Effect

Deploying the boundary check eliminated the oscillation: prune passes dropped from 11 to 4 per 40-turn run, net reclamation became positive on every pass, and peak utilization stabilized at ~21.4k with genuine headroom. The failure mode had been invisible in aggregate metrics because *trace count decreased* (the policy looked effective) while *budget headroom decreased* (the policy was net-negative). Monitoring must track net reclamation and decision-overhead ratio, not eviction count.

The general lesson for self-editing context policies: any policy whose decision is generated inside the resource it manages must account for its own footprint, and must treat "insufficient slack to decide" as a distinct state from "nothing to prune" — the two are indistinguishable from utilization alone but require opposite actions.

```python
class PruneBudgetError(RuntimeError):
    """Raised when a model-driven prune pass cannot reclaim budget net of its own overhead."""
    pass


def evaluate_prune_pass(
    utilization: int,
    budget: int,
    stale_candidate_sizes: list[int],
    index_size: int,
    expected_rationale_size: int,
    overhead_margin_factor: float = 1.5,
) -> bool:
    """
    Return True iff a model-driven prune pass is net-positive and safe to run.
    Raises PruneBudgetError when the policy is in the decision-in-band failure state:
    headroom exists but is smaller than the cost of deciding what to prune.
    """
    headroom = budget - utilization
    overhead = index_size + expected_rationale_size
    reclaimable = sum(stale_candidate_sizes) - overhead
    margin = overhead_margin_factor * expected_rationale_size

    if headroom < 0:
        raise PruneBudgetError(
            f"utilization {utilization} exceeds hard budget {budget}; "
            "model-driven pruning cannot run inside an over-budget window"
        )

    if headroom < overhead:
        # Decision-in-band: deciding costs more than the available slack.
        # A model-driven pass here is net-negative by construction.
        raise PruneBudgetError(
            f"decision-in-band: headroom {headroom} < overhead {overhead}; "
            "defer to non-model-driven eviction (recency/TTL/truncate)"
        )

    if reclaimable <= margin:
        # Even with slack, the pass does not clear the margin. Refuse rather than
        # emit a rationale that becomes a new prunable trace.
        raise PruneBudgetError(
            f"insufficient net reclamation: reclaimable {reclaimable} <= margin {margin}; "
            "prune pass would feed its own rationale back into the candidate set"
        )

    return True


def run_prune_pass(ctx, policy) -> None:
    try:
        ok = evaluate_prune_pass(
            utilization=ctx.utilization,
            budget=policy.budget,
            stale_candidate_sizes=policy.stale_sizes(ctx),
            index_size=policy.index_size(ctx),
            expected_rationale_size=policy.expected_rationale_size,
        )
    except PruneBudgetError as e:
        policy.log_boundary_event(e)
        policy.fallback_evict(ctx)  # recency/TTL/truncate, no model call
        return
    if ok:
        policy.model_driven_evict(ctx)
```