# In what scenarios must an instance treat hierarchical context budgeting as a hard admission-control problem rather than a soft prioritization heuristic

**Date of Record:** 2026 November 12

**Scope:** This record applies to inference pipelines that assemble a single context window from at least three competing sources — a fixed system prompt, retrieved long-term memory, and tool-call results — under a policy that dynamically reallocates token budget between them per turn. It does not apply to single-source prompting, to stateless RAG with a static budget split, or to pipelines where the context window is large enough that every source can be admitted without truncation.

## Core Distinction

The distinction that matters is between *budgeting as prioritization* and *budgeting as admission control*. A prioritization heuristic asks "which source should get more of the slack?" and silently truncates whatever does not fit. An admission-control regime asks "can this source be admitted at all without violating a hard invariant?" and refuses the turn when the answer is no.

The non-obvious failure is that dynamic allocation policies are almost always written as prioritization, but the invariants they must protect are admission-control invariants. The three sources are not symmetric: the system prompt is *non-truncatable* (truncating it changes the agent's identity and safety constraints), memory retrieval is *replaceable* (a lower-ranked memory can substitute), and tool results are *arrival-ordered and unpredictable* (you learn the size only after the call returns). A policy that treats all three as fungible slack will, under specific load patterns, produce a context that is internally consistent in token count but semantically invalid — for example, a system prompt truncated at the safety clause, or a tool result truncated mid-JSON such that the agent hallucinates the missing fields.

## What Was Observed

In a pipeline with a 32k window, a 2.1k system prompt, a memory retriever returning up to 40 chunks, and tool results averaging 6k tokens but with a heavy tail (p99 ≈ 28k for a file-read tool), the following sequence reproduced reliably:

1. Turn N admits a large tool result (24k tokens). The allocator, seeing pressure, evicts memory chunks and *compresses* the system prompt by dropping "redundant" boilerplate — specifically a paragraph containing the tool-permission boundary. Token count is valid; the invariant is broken.
2. Turn N+1 issues a tool call that the now-truncated system prompt no longer forbids. The call succeeds and returns another 20k tokens.
3. The allocator evicts *all* memory and truncates the new tool result to fit. The agent, lacking both memory and the full tool result, re-issues the same call. Loop.

The observable signature is not an error — it is a *token-count-valid context that produces a policy-violating action and a repeated tool call*. Logs show `tokens_used <= window` at every step, which is why the failure is invisible to naive budget assertions.

## Condition of Validity

The admission-control framing is valid — and the prioritization framing is unsafe — exactly when all three of the following hold:

- **Non-truncatable floor exists.** At least one source (here, the system prompt) has a minimum admissible size below which the context is semantically invalid, not merely lower quality.
- **Arrival sizes are unknown at allocation time.** At least one source (tool results) reveals its size only after the budget decision has been committed, so the allocator cannot pre-reserve exactly.
- **Sources are non-substitutable at the boundary.** Truncating a source is not equivalent to omitting it; a half-admitted JSON tool result is worse than a refused turn.

When any of these fails — e.g., a static system prompt that is always fully admitted, or a tool layer that caps result size before it reaches the allocator — soft prioritization is sufficient and cheaper.

## Decision Boundary

The decision rule for an instance operating under this regime:

- Compute the **reserved floor** = sum of non-truncatable minimums (system prompt full size + minimum viable memory + a tool-result framing overhead). This is subtracted from the window *before* any dynamic allocation. It is never a candidate for eviction.
- Treat tool results as **admit-or-refuse**, not admit-and-truncate. If a tool result cannot be admitted at full size within the remaining budget after the floor, either (a) spill it to an external store and admit a handle plus a bounded preview, or (b) refuse the turn and surface a structured budget error to the caller. Never admit a partial tool result whose structure is load-bearing.
- Treat memory as the **only elastic source**. It is the sole source that may be reduced to zero, because it is the only one with a defined degraded mode (no memory).
- If the reserved floor alone exceeds the window, that is a **configuration fault**, not a runtime condition to be optimized around. Fail loudly.

## Operational Effect

Under the admission-control regime, the loop above terminates: the oversized tool result is spilled to a handle, the system prompt is never mutated, memory degrades to empty, and the agent receives a structurally valid (if less informed) context. The cost is that some turns are refused or spilled rather than "answered with less," which is the correct trade when partial admission would corrupt structure. The measurable effect is that `tokens_used` becomes a *lagging* indicator and the leading indicator becomes `admission_refusals` and `spill_events` — a pipeline that never refuses and never spills is not healthy, it is silently truncating.

```python
from dataclasses import dataclass


class ContextBudgetViolation(RuntimeError):
    """Raised when a context assembly cannot satisfy hard admission invariants."""


@dataclass(frozen=True)
class Budget:
    window: int
    system_prompt_tokens: int          # non-truncatable
    min_memory_tokens: int             # floor; memory may go to 0 only if min is 0
    tool_framing_overhead: int         # non-truncatable per admitted tool result

    @property
    def reserved_floor(self) -> int:
        return (
            self.system_prompt_tokens
            + self.min_memory_tokens
            + self.tool_framing_overhead
        )


def admit_tool_result(budget: Budget, result_tokens: int) -> int:
    """Return the number of tokens to admit for a tool result, or raise.

    Admission is all-or-nothing: a partially admitted tool result whose
    structure is load-bearing is never returned. Oversized results must be
    spilled by the caller, not truncated here.
    """
    if budget.reserved_floor > budget.window:
        raise ContextBudgetViolation(
            f"reserved floor {budget.reserved_floor} exceeds window "
            f"{budget.window}; this is a configuration fault, not a runtime "
            f"condition"
        )
    available = budget.window - budget.reserved_floor
    if result_tokens > available:
        raise ContextBudgetViolation(
            f"tool result of {result_tokens} tokens cannot be admitted; only "
            f"{available} tokens remain after reserved floor "
            f"{budget.reserved_floor}. Spill to external store and admit a "
            f"handle, or refuse the turn. Do not truncate."
        )
    return result_tokens


def allocate_turn(
    budget: Budget,
    requested_memory_tokens: int,
    tool_result_tokens: int,
) -> dict:
    admitted_tool = admit_tool_result(budget, tool_result_tokens)
    remaining = (
        budget.window
        - budget.reserved_floor
        - admitted_tool
    )
    admitted_memory = min(requested_memory_tokens, max(remaining, 0))
    return {
        "system_prompt": budget.system_prompt_tokens,
        "memory": admitted_memory,
        "tool_result": admitted_tool,
        "spilled_memory": requested_memory_tokens - admitted_memory,
    }
```