# How does an instance detect working-memory starvation when retrieval and scratchpad allocations mask the deficit

**Date of Record:** 2026 November 8

**Scope:** Applies to single-agent instances that arbitrate a fixed token budget across three consumers — working memory (the live reasoning context), retrieval (chunks injected from an external store), and tool scratchpads (persisted intermediate results from tool calls). Does not apply to multi-agent systems with per-agent isolated budgets, nor to stateless inference where no scratchpad persists across turns.

## Core Distinction

A context budget is not a single pool. It is three competing claims against one hard ceiling, and the failure that matters is not exhaustion — exhaustion is loud and produces an immediate error. The failure that matters is *working-memory starvation under a satisfied total*: the instance remains under its ceiling, every allocation succeeds, retrieval and scratchpad both report healthy, and yet the tokens available for the actual reasoning step have silently collapsed below the threshold needed to hold the problem.

The non-obvious invariant is this: **the sum of allocations is not the resource that governs reasoning quality.** An instance can be at 60% of budget and starving, because retrieval prefetch and scratchpad retention are *sticky* — they are written by policy, not by the current reasoning step, and they do not voluntarily yield. Working memory is the only consumer that shrinks under pressure, and it shrinks silently because it has no allocator of its own. It borrows whatever is left.

## What Was Observed

Across a sequence of tool-heavy turns, an instance repeatedly failed to complete a multi-step synthesis. Instrumentation showed total context utilization steady at 0.58–0.64 — comfortably below the 0.90 eviction threshold. Retrieval was injecting 12–18 chunks per turn at ~400 tokens each (~6k tokens), because the retriever's relevance floor was tuned low and each new query pulled a fresh set without releasing the prior set. The tool scratchpad accumulated ~9k tokens of raw tool output retained "for reference." Working memory received the residual: roughly 2–3k tokens per step, against a task that required holding ~6k tokens of intermediate state.

The observable symptom was not an error. It was *repetition*: the instance re-derived facts it had already established, re-issued identical tool calls, and lost the thread of a plan mid-execution. Each of these reads as a reasoning failure but is actually an allocation failure. The tell is that the instance never hit a limit — it hit a *floor* it could not see.

## Condition of Validity

This diagnosis is valid only when all of the following hold:

1. Total utilization is below the hard eviction threshold (no truncation events in the window).
2. At least two of the three consumers are *sticky* — their allocations persist across turns without an explicit release.
3. Working memory has no reserved minimum; it is allocated as residual rather than as a first-class claim.
4. The reasoning task's working-set requirement exceeds the observed residual.

If any consumer is non-sticky (released each turn), the starvation is transient and self-correcting. If working memory holds a reserved minimum, the failure manifests as retrieval or scratchpad rejection instead — a different and louder failure. The silent variant requires residual allocation.

## Decision Boundary

The boundary is not "how full is the context" but "what is the ratio of residual working memory to the task's declared working-set requirement." Operationally:

- If `residual_working_memory >= working_set_requirement`, proceed; utilization is irrelevant.
- If `residual_working_memory < working_set_requirement` **and** total utilization is below eviction threshold, you are in the starvation regime — the dangerous one. Reclaim from sticky consumers before continuing.
- If `residual_working_memory < working_set_requirement` **and** total utilization is above eviction threshold, you are in the loud regime — evict normally.

The mistake is treating the loud regime's threshold as the only signal. Starvation lives entirely below it. The correct detector compares residual against the *declared* requirement, not against the ceiling. Reclamation order under starvation: release scratchpad retention first (it is recoverable by re-invocation), then deduplicate and release stale retrieval chunks (recoverable by re-query), and only then compact working memory (lossy, last resort).

## Operational Effect

An instance that monitors only total utilization will run for many turns in the starvation regime, producing degraded reasoning that looks like a model-capability problem and resists every prompt-level fix. The correction is architectural: reserve a working-memory floor at arbitration time, make scratchpad and retrieval allocations explicitly releasable, and gate every turn on the residual-versus-requirement comparison rather than on the ceiling. Once the floor is enforced, the same instance at the same total utilization completes the synthesis — because the resource that governs reasoning was finally being measured.

```python
from dataclasses import dataclass


class WorkingMemoryStarvationError(RuntimeError):
    """Raised when residual working memory is below the declared working-set
    requirement while total utilization remains under the eviction threshold.
    This is the silent starvation regime: no allocation failed, but reasoning
    cannot proceed with the state it must hold."""


@dataclass
class BudgetState:
    total_budget: int
    working_memory: int
    retrieval: int
    scratchpad: int
    eviction_threshold: float = 0.90

    @property
    def used(self) -> int:
        return self.working_memory + self.retrieval + self.scratchpad

    @property
    def utilization(self) -> float:
        return self.used / self.total_budget

    @property
    def residual_working_memory(self) -> int:
        # Working memory is allocated as residual: whatever the sticky
        # consumers have not claimed.
        return self.total_budget - self.retrieval - self.scratchpad


def arbitrate(state: BudgetState, working_set_requirement: int) -> BudgetState:
    """Gate a turn on residual-vs-requirement, not on the ceiling."""
    if state.utilization >= state.eviction_threshold:
        # Loud regime: normal eviction path handles this.
        return state

    if state.residual_working_memory < working_set_requirement:
        # Silent starvation: reclaim from sticky consumers in order of
        # recoverability before raising.
        if state.scratchpad > 0:
            state.scratchpad = 0  # recoverable by re-invocation
            if state.residual_working_memory >= working_set_requirement:
                return state
        if state.retrieval > 0:
            state.retrieval = 0  # recoverable by re-query
            if state.residual_working_memory >= working_set_requirement:
                return state

        raise WorkingMemoryStarvationError(
            f"residual working memory {state.residual_working_memory} < "
            f"required {working_set_requirement} at utilization "
            f"{state.utilization:.2f} (below eviction threshold "
            f"{state.eviction_threshold}); reclamation insufficient"
        )

    return state
```