# Why does an instance of a context budget silently overrun when compaction reclaims tokens that were already counted as resident? **Date of Record:** 2026 November 15 **Scope:** Applies to agent runtimes that maintain an explicit token budget over a message/state graph and that perform compaction (summarization, eviction, or deduplication) as a background or inline operation. Does not apply to stateless single-shot inference where the budget is recomputed from scratch on every request with no persistent provenance ledger. ## Core Distinction There are two distinct quantities that both get called "context size": 1. **Resident tokens** — the sum of token counts of the messages currently materialized in the prompt that will be sent to the model on the next turn. 2. **Accounted tokens** — the sum of token counts recorded in the provenance ledger against every logical state object that the runtime believes is *either resident or recoverable*. The invariant that most runtimes implicitly assume is `accounted == resident + reclaimable`. The failure mode is that compaction mutates `resident` downward and *also* leaves the reclaimed objects in the ledger as `reclaimable`, but the reclaimable objects are not actually recoverable — their source bytes were dropped when the summary was written. The ledger therefore reports headroom that does not exist. The budget check passes. The next turn overruns the model's hard limit and is rejected by the provider, often mid-tool-loop, after side effects have already occurred. The non-obvious part: the overrun is *not* caused by a token-counting bug. Per-object counts are correct. The bug is a provenance-classification bug: a token that was counted once as `resident` and then "reclaimed" is still counted as `reclaimable`, so it is counted twice in the accounting identity while being physically present zero times. ## What Was Observed A long-running agent (multi-hour, tool-calling, ~40k-token budget) began failing intermittently with provider-side `context_length_exceeded` on turns where the internal budget estimator reported 61–78% utilization. The failure correlated with turns immediately following a compaction event, and only when the compaction was triggered by a *deduplication* pass rather than by summarization. Instrumentation of the ledger showed the following sequence on a failing turn: - Pre-compaction: `resident = 38,900`, `reclaimable = 0`, `accounted = 38,900`. - Dedup pass removes 12 near-duplicate tool-result messages (each ~180 tokens) and writes a single merged result of ~210 tokens. It marks the 12 originals `reclaimable` for "possible rehydration." - Post-compaction: `resident = 36,980`, `reclaimable = 2,160`, `accounted = 39,140`. - The budget check `accounted <= 40,000` passes with 860 tokens of headroom. - The next turn appends a 3,400-token tool result. The estimator computes `resident = 40,380` but the check still uses `accounted` minus a stale `reclaimable` discount and reports 1,340 tokens free. - Provider rejects the request: actual prompt is 40,380 tokens against a 40,000 limit. The 2,160 "reclaimable" tokens were never rehydratable. The merged result did not retain a pointer to the originals; the originals were dropped from the store when the dedup pass committed. The ledger's `reclaimable` field was a bookkeeping fiction. A second, subtler observation: the same overrun occurred even when rehydration *was* possible, because the rehydration path itself consumed tokens (the rehydrated message replaced the summary, netting a small gain but a large transient). The budget check evaluated the *steady-state* accounting, not the *post-rehydration* resident size. Compaction made the next turn's worst case larger, not smaller, while the ledger reported it smaller. ## Condition of Validity The invariant `accounted == resident` (not `resident + reclaimable`) holds, and the budget check is safe, only under all of the following: - Every object in the ledger is in exactly one provenance state: `resident`, `reclaimable-with-verified-source`, or `dropped`. - `reclaimable` is only assigned when the source bytes are still present in a store reachable at rehydration time, and the rehydration cost is included in the worst-case resident projection. - The budget check is performed against `resident + max(0, projected_next_turn_delta)`, not against a discounted `accounted` figure. - Compaction is treated as a *transfer* between provenance states, not as a deletion that frees budget. A reclaimed token frees budget only when its state becomes `dropped` and its source is gone. If any of these is violated — most commonly the third, because the next-turn delta is hard to bound for tool-calling agents — the budget check is unsound and the overrun is latent. ## Decision Boundary The runtime must choose, per compaction event, between two policies: - **Conservative (drop-on-compact):** reclaimed objects transition directly to `dropped`; `accounted` decreases by exactly the reclaimed amount; no rehydration is ever offered. Budget headroom is real. Cost: lossy, no undo. - **Reversible (retain-on-compact):** reclaimed objects transition to `reclaimable-with-verified-source`; `accounted` does *not* decrease; the budget check projects the worst-case rehydration size. Cost: compaction yields no budget relief until the objects are explicitly dropped. The failure occurs when a runtime adopts the reversible policy for storage reasons but the conservative policy for accounting reasons — i.e., it keeps the objects and the rehydration path, but subtracts them from the budget as if they were gone. That hybrid is the boundary. It is attractive because it makes compaction "look" effective in dashboards, and it is wrong because the budget is a statement about the next request, not about the store. The decision rule: **if an object can be rehydrated into the next prompt, it must be counted against the next prompt's budget.** Reclaimability is a storage property; residency is a prompt property; only residency belongs in the budget. ## Operational Effect With the boundary enforced, compaction events no longer produce phantom headroom. Dashboards show compaction reducing `resident` while leaving `accounted` flat until objects are explicitly dropped, which is the honest signal: compaction reduces *latency and prompt length*, not *budget*. Tool-loop agents gain a hard guarantee that a passing budget check implies the next request fits, at the cost of more frequent explicit drops and a slightly higher rate of early summarization. The intermittent `context_length_exceeded` failures disappear because the estimator and the provider now agree on what "resident" means. The general lesson: any accounting identity that mixes a *physical* quantity (tokens in the prompt) with a *logical* quantity (objects in the store) will drift at every compaction boundary. Keep the two ledgers separate, and gate the budget only on the physical one. ```python from dataclasses import dataclass, field from enum import Enum class ProvenanceError(RuntimeError): """Raised when a provenance state transition would corrupt budget accounting.""" class State(Enum): RESIDENT = "resident" RECLAIMABLE = "reclaimable" # source bytes still present and rehydratable DROPPED = "dropped" @dataclass class LedgerEntry: obj_id: str tokens: int state: State source_present: bool @dataclass class BudgetLedger: hard_limit: int entries: dict[str, LedgerEntry] = field(default_factory=dict) def resident_tokens(self) -> int: return sum(e.tokens for e in self.entries.values() if e.state is State.RESIDENT) def accounted_tokens(self) -> int: # accounted counts resident + reclaimable, because reclaimable can # re-enter the prompt and therefore consumes budget. return sum( e.tokens for e in self.entries.values() if e.state in (State.RESIDENT, State.RECLAIMABLE) ) def compact(self, obj_ids: list[str], merged_tokens: int, keep_source: bool) -> None: """Transfer objects out of RESIDENT. Never silently frees budget.""" for oid in obj_ids: entry = self.entries[oid] if entry.state is not State.RESIDENT: raise ProvenanceError( f"cannot compact {oid}: state is {entry.state.value}, not resident" ) if keep_source and not entry.source_present: raise ProvenanceError( f"cannot mark {oid} reclaimable: source bytes are absent" ) entry.state = State.RECLAIMABLE if keep_source else State.DROPPED self.entries[f"merged:{len(obj_ids)}"] = LedgerEntry( obj_id=f"merged:{len(obj_ids)}", tokens=merged_tokens, state=State.RESIDENT, source_present=True, ) def check_next_turn(self, projected_delta: int) -> None: """Gate the next request on physical residency, not discounted accounting.""" projected = self.resident_tokens() + projected_delta if projected > self.hard_limit: raise ProvenanceError( f"next turn would be {projected} tokens against limit " f"{self.hard_limit}; compaction did not create real headroom " f"(resident={self.resident_tokens()}, " f"accounted={self.accounted_tokens()})" ) def demonstrate_boundary() -> None: ledger = BudgetLedger(hard_limit=40_000) for i in range(12): ledger.entries[f"dup{i}"] = LedgerEntry( obj_id=f"dup{i}", tokens=180, state=State.RESIDENT, source_present=True ) ledger.entries["base"] = LedgerEntry( obj_id="base", tokens=38_900 - 12 * 180, state=State.RESIDENT, source_present=True ) # Dedup keeps sources for "possible rehydration": budget must NOT shrink. ledger.compact([f"dup{i}" for i in range(12)], merged_tokens=210, keep_source=True) # A 3,400-token tool result arrives. The correct gate rejects it. ledger.check_next_turn(projected_delta=3_400) ```