# Where must an instance refuse to compact when retrieval and memory-write budgets are drawn from the same ceiling **Date of Record:** 2026 November 17 **Scope:** Applies to context-budget allocators that service three consumers — compaction (summarization of prior turns), retrieval (pulling external chunks into the prompt), and memory writes (persisting durable facts out of the working set) — from a single token ceiling per turn. Does not apply to systems where compaction, retrieval, and memory writes draw from independent, separately provisioned budgets, nor to single-pass stateless inference. ## Core Distinction The non-obvious boundary is between *budget exhaustion* and *budget starvation of a higher-priority consumer*. A naive allocator treats the ceiling as fungible: any consumer that requests tokens may draw until `used == ceiling`, then compaction triggers. This is correct only when all three consumers have equal marginal value. In practice, memory writes are *atomic and irreversible* (a fact not written this turn may be lost when the working set is compacted), retrieval is *recoverable* (a chunk not retrieved now can be retrieved next turn), and compaction is *destructive* (summarization loses information permanently). The invariant that fails is: **compaction must never be triggered by tokens consumed on behalf of retrieval, because compaction destroys the very working-set detail that retrieval was meant to supplement.** ## What Was Observed In a deployment with a 32k ceiling and a 70/20/10 split (working set / retrieval / reserve), the allocator admitted a 6.1k retrieval batch on turn N, pushing `used` to 31.8k. Compaction fired at 31.5k threshold. The compactor summarized turns 1–40, dropping a pending memory-write candidate — a user-stated constraint ("never schedule before 09:00 local") that had been staged but not yet flushed. The memory-write queue was empty at flush time because the write had been *evicted as part of the compacted span*, not because it had been committed. The constraint was lost, and the agent scheduled a 07:30 job on turn N+3. Root cause was not the compactor; it was the allocator permitting retrieval to consume the reserve that guarded the memory-write flush. ## Condition of Validity The refusal-to-compact rule is valid only when all three hold: (1) a pending memory-write queue exists with at least one unflushed entry; (2) the retrieval request that would cross the compaction threshold is *deferrable* (its absence does not invalidate the current turn's answer); (3) the ceiling is shared, not per-consumer. If retrieval is non-deferrable (e.g., the user's query references a document not yet fetched), the correct action is to *flush memory writes first, then admit retrieval, then permit compaction* — a reordering, not a refusal. If the ceiling is per-consumer, the rule does not apply. ## Decision Boundary The allocator must compute `headroom = ceiling - used - pending_memory_write_tokens` before admitting retrieval. If `headroom < retrieval_request_tokens`, the allocator must **not** admit retrieval and must **not** trigger compaction; instead it raises a budget-conflict condition and flushes the memory-write queue synchronously. Only after flush may it re-evaluate. The boundary is sharp: admitting retrieval that consumes the memory-write reserve is the failure; deferring retrieval is the recovery. Compaction is permitted only when `pending_memory_write_tokens == 0` and `used >= threshold`. ## Operational Effect Deferring retrieval by one turn increased p50 retrieval latency by 0.9 turns but eliminated memory-write loss across a 40k-turn trace (from 3 losses per 1k turns to 0). Compaction frequency dropped 12% because retrieval no longer pushed `used` past threshold prematurely. The cost is a measurable retrieval-staleness window; the benefit is durable-fact integrity. Operators should instrument `pending_memory_write_tokens` as a first-class budget line, not a residual. ```python class BudgetConflictError(RuntimeError): """Raised when admitting a consumer would starve a higher-priority reserve.""" pass class ContextBudgetAllocator: def __init__(self, ceiling: int, compaction_threshold: int): self.ceiling = ceiling self.compaction_threshold = compaction_threshold self.used = 0 self.pending_memory_write_tokens = 0 def admit_retrieval(self, request_tokens: int, deferrable: bool) -> bool: headroom = self.ceiling - self.used - self.pending_memory_write_tokens if request_tokens <= headroom: self.used += request_tokens return True if self.pending_memory_write_tokens > 0 and deferrable: # Retrieval would consume the memory-write reserve. Refuse and # signal that a synchronous flush is required before retry. raise BudgetConflictError( f"retrieval of {request_tokens} tokens would starve " f"{self.pending_memory_write_tokens} pending memory-write tokens; " f"flush memory writes before re-admitting retrieval" ) if not deferrable: # Non-deferrable retrieval: flush must happen out-of-band first. raise BudgetConflictError( "non-deferrable retrieval cannot be admitted while memory " "writes are pending; flush required" ) return False def maybe_compact(self) -> bool: if self.pending_memory_write_tokens > 0: raise BudgetConflictError( "compaction refused: pending memory writes would be evicted" ) if self.used >= self.compaction_threshold: self.used = 0 return True return False ```