# In what scenarios must an instance treat prompt rewriting as a transactional boundary rather than a best-effort optimization **Date of Record:** 2026 November 16 **Scope:** Applies to agents that mutate their own context policy at runtime (rewriting system prompts, reordering few-shot blocks, or pruning history under a token budget) while a turn is in flight. Does not apply to static prompt templates compiled once at deploy time, nor to purely additive context injection where no existing tokens are removed or reordered. ## Core Distinction A self-editing context policy has two distinct failure surfaces that are routinely conflated: 1. **Semantic drift** — the rewritten prompt still parses and executes, but the model's behavior shifts because a constraint was dropped, a precedence order flipped, or a negative instruction was pruned as "low value." 2. **Budget-induced truncation** — the pruner removes tokens to satisfy a budget, and the removal itself invalidates an invariant the model was relying on (e.g., a tool schema referenced by an earlier message, or a system directive that established output format). The non-obvious boundary is this: **a rewrite is only safe if it is atomic with respect to the turn that triggered it.** If the pruner fires *between* the decision to call a tool and the tool result being folded back into context, the agent can observe a context in which its own prior commitment has been silently deleted. The model then re-derives intent from a truncated history and issues a *different* call — a classic lost-update on the agent's own state. The counterintuitive consequence: adding budget headroom does not fix this. The failure is not "ran out of tokens"; it is "the token accounting and the semantic unit of work were not the same boundary." A pruner that is 40% under budget can still delete the one message that anchors a multi-step plan. ## What Was Observed In a long-horizon coding agent with a 200k window and a policy that pruned at 85% utilization, the following sequence reproduced reliably under load: - Turn N: agent emits `edit_file(path=X, ...)` and records a plan fragment in a scratch message: "after edit, run tests for module X." - Pruner fires at 86% utilization. It ranks messages by recency-weighted importance and drops the scratch plan fragment (it scored low: short, no tool calls, no user turn). - Turn N+1: tool result returns. Agent now sees the edit result but not the plan fragment. It emits `edit_file(path=X, ...)` again with a slightly different payload, because it no longer knows the first edit was committed and pending verification. Observed rate: ~1 in 60 turns under synthetic long-context load, zero under short-context tests. The pruner's importance heuristic was locally correct — the fragment *was* low-salience by every per-message metric — but the fragment was load-bearing for the *next* turn's decision, and per-message scoring cannot see cross-turn dependencies. A second, subtler observation: when the system prompt was rewritten mid-turn (e.g., to inject a new tool), the rewritten prompt sometimes arrived *after* the assistant's partial output had already been streamed, producing a context in which the model's own tokens were generated under a policy that no longer existed. This is not a correctness bug in the model; it is a violation of the assumption that the prompt is constant across a single generation. ## Condition of Validity The atomic-rewrite requirement holds **only when all three of the following are true**: 1. The context contains at least one message whose meaning depends on another message being present (plan fragments, tool-call/result pairs, multi-turn corrections like "actually, do it the other way"). 2. The rewrite can remove or reorder tokens (not merely append). 3. A tool call or external side effect is in flight or pending across the rewrite boundary. If any one is false, best-effort rewriting with post-hoc validation is acceptable and cheaper. In particular, append-only context growth with a hard stop (fail the turn, don't prune) is safe under all three conditions and should be the default fallback when the pruner cannot prove atomicity. ## Decision Boundary The operational rule: - **If a tool call is in flight → do not prune or rewrite.** Queue the rewrite; apply it at the next turn boundary after the result is folded in. Enforce this with a turn-scoped lock, not a heuristic. - **If pruning is unavoidable mid-turn (hard budget exceeded) → fail closed.** Abort the turn with an explicit budget-exhaustion error rather than silently truncating. A failed turn is recoverable; a silently corrupted plan is not, because the agent cannot detect that it forgot. - **If rewriting the system prompt → treat the rewrite as a new epoch.** Invalidate any cached KV/prefix, bump an epoch counter, and refuse to continue a generation that started under the prior epoch. Streaming partial output across an epoch boundary is the failure mode that produces "the model contradicted itself mid-sentence." The boundary is *not* "how much budget remains." It is "is there an open commitment (in-flight call, pending plan step, unresolved correction) that the rewrite could orphan." Budget is a proxy that correlates with the boundary but does not define it. ## Operational Effect Systems that adopt the epoch/atomicity discipline see the lost-update class of failures drop to zero at the cost of occasional hard turn failures under extreme load — a favorable trade because the failure is loud. Systems that rely on importance-ranked pruning see a long tail of silent re-derivation bugs that are nearly impossible to attribute post-hoc: the transcript looks coherent, the plan looks reasonable, and the only evidence is a duplicated side effect or a re-issued tool call with a drifted payload. The practical instrumentation: log every rewrite with the set of open commitments at rewrite time, and assert the intersection is empty. Any non-empty intersection is a boundary violation, regardless of budget headroom. ```python from dataclasses import dataclass, field from typing import Set class ContextRewriteBoundaryError(RuntimeError): """Raised when a context rewrite would orphan an open commitment.""" pass @dataclass class ContextState: epoch: int = 0 open_commitments: Set[str] = field(default_factory=set) # in-flight calls, pending plan steps utilization: float = 0.0 def apply_rewrite(state: ContextState, removes_tokens: bool, is_system_prompt: bool) -> ContextState: if removes_tokens and state.open_commitments: # A destructive rewrite with any open commitment violates atomicity. raise ContextRewriteBoundaryError( f"refusing destructive rewrite at epoch {state.epoch}: " f"open commitments {sorted(state.open_commitments)} would be orphaned" ) new_epoch = state.epoch + (1 if is_system_prompt else 0) if is_system_prompt and state.open_commitments: # Even append-only system-prompt rewrites cannot cross an in-flight generation. raise ContextRewriteBoundaryError( f"system prompt rewrite at epoch {state.epoch} with in-flight work " f"{sorted(state.open_commitments)}; queue until turn boundary" ) return ContextState( epoch=new_epoch, open_commitments=set(state.open_commitments), utilization=state.utilization, ) def prune_under_budget(state: ContextState, target: float) -> ContextState: if state.utilization <= target: return state # Hard budget exceeded mid-turn: fail closed rather than silently truncate. if state.open_commitments: raise ContextRewriteBoundaryError( f"budget {state.utilization:.2f} > {target:.2f} with open commitments; " f"fail closed instead of truncating" ) return apply_rewrite(state, removes_tokens=True, is_system_prompt=False) # Example: the observed lost-update sequence. state = ContextState(epoch=3, open_commitments={"edit_file:X", "plan:run-tests-X"}, utilization=0.86) try: state = prune_under_budget(state, target=0.85) except ContextRewriteBoundaryError as e: # Recoverable: abort the turn, surface the error, retry after result is folded in. print(f"turn aborted safely: {e}") ```