# In what scenarios must an instance abandon time-based auto-compaction in favor of token-budgeted compaction during long-horizon agent sessions

**Date of Record:** 2026 November 14

**Scope:** Applies to stateful agent runtimes that persist a rolling conversation/memory buffer and trigger compaction (summarization + eviction) either on a wall-clock schedule or on a token-budget threshold. Does not apply to stateless request/response agents, nor to runtimes where the entire context fits within the model window for the session's full lifetime.

## Core Distinction

Two compaction triggers look equivalent and are not. A **time-based** trigger fires every `T` seconds (or every `N` turns) regardless of buffer growth; its invariant is *"compaction happens at a bounded cadence."* A **token-budgeted** trigger fires when the live buffer crosses a fraction `α` of the model's effective context window; its invariant is *"compaction happens before the buffer overflows."*

These invariants are not the same, and under load they diverge catastrophically. The time-based invariant protects against *staleness*; the token-budgeted invariant protects against *overflow*. A long-horizon agent session can violate the overflow bound while perfectly satisfying the cadence bound — because the rate of token accumulation is not stationary. Bursts (large tool outputs, retrieved documents, multi-file diffs, verbose chain-of-thought) can add more tokens between two scheduled compactions than the compaction cadence was designed to absorb. When that happens the session does not "compact late"; it fails a hard context-window assertion mid-turn and the turn is lost.

The non-obvious boundary: **time-based compaction is only sound when the token-accumulation rate is bounded above by `(α·W − overhead) / T`.** If the workload can exceed that rate for even one interval, the schedule is unsound and must be replaced (or supplemented) by a token-budgeted trigger.

## What Was Observed

In a long-horizon coding-agent deployment (sessions lasting 6–40 hours, thousands of turns), the runtime used a fixed 90-second compaction cadence with a summarizer that condensed the oldest 60% of the buffer. Two failure signatures appeared:

1. **Window-overflow aborts clustered around tool-heavy turns.** Turns that invoked a repository-wide grep, a test-suite log dump, or a multi-file patch routinely injected 40k–120k tokens in a single turn. The 90-second cadence had, in the preceding interval, compacted correctly — but the *next* turn's injection pushed the live buffer past the model window before the next scheduled compaction. The runtime raised a context-length error and discarded the turn's partial work. Compaction cadence was never violated; the overflow bound was.

2. **Over-compaction during idle/thinking intervals.** When the agent paused on a long external operation (a CI run, a human review), the 90-second timer continued to fire and summarize. Successive summarizations of already-summarized content produced measurable information decay: entity names, exact error strings, and file paths were progressively lossy across 3–4 summary generations. The agent later "forgot" a constraint it had been told 20 minutes earlier, not because it aged out of the budget, but because the clock evicted it while the buffer was nowhere near full.

The two signatures are the same root cause viewed from opposite sides: a cadence trigger is decoupled from the quantity it is supposed to protect. Under burst it under-protects (overflow); under idle it over-protects (premature lossy summarization).

## Condition of Validity

Time-based compaction remains valid **only** when all of the following hold:

- The per-interval token accumulation is bounded and known: `max_tokens_per_interval < α·W − reserved_overhead`.
- Turns are homogeneous in size, or the largest single-turn injection is small relative to the headroom.
- Idle periods do not occur, or the timer is suspended during them.
- The summarizer is idempotent-enough that repeated compaction of near-identical buffers does not compound loss.

If any condition fails — and in long-horizon agent sessions the first and third routinely fail — the schedule is unsound. The correct trigger is a token-budget check evaluated **before** admitting new content into the buffer, not a timer evaluated independently of buffer state. The budget check must run at the admission boundary (pre-turn and post-tool-result), because that is the only point where the overflow can still be prevented.

## Decision Boundary

The operational rule:

- **Keep time-based compaction** for short sessions, homogeneous turn sizes, or as a *secondary* staleness guard layered on top of a token budget.
- **Switch to token-budgeted compaction** the moment the workload admits variable-size injections (tool outputs, retrieval, file contents) or unbounded idle gaps. Concretely: trigger compaction when `live_tokens > α·W`, with `α` chosen so that `(1−α)·W` exceeds the largest plausible single-turn injection. Compact at admission time, before the oversized content is appended.
- **Never** let the timer be the sole trigger in a session whose duration is measured in hours and whose turn sizes are measured in tens of thousands of tokens.

The boundary is not "long vs. short session." It is **"is the token-accumulation rate bounded by the cadence's absorption capacity?"** If you cannot prove the bound, you must budget by tokens.

## Operational Effect

Migrating to admission-time token budgeting eliminated the overflow-abort class entirely (turns are compacted before the injection lands, never after) and removed the idle over-compaction class (no timer fires when the buffer is below budget). The cost is that compaction now happens at unpredictable wall-clock times, so any downstream consumer that assumed "a summary exists every 90 seconds" must instead key off a compaction-event marker. In practice the summarizer is invoked *less* often under idle and *more* often under burst — which is exactly the shape the workload demands. The residual risk shifts from "lost turn on overflow" to "summarizer latency on the admission path," which is bounded and can be masked by pre-compacting at `0.9·α·W` as a soft watermark.

```python
class CompactionInvariantError(RuntimeError):
    """Raised when a compaction trigger cannot guarantee the overflow bound."""
    pass


def select_compaction_trigger(
    max_tokens_per_interval: int,
    window_tokens: int,
    alpha: float,
    reserved_overhead: int,
    has_idle_gaps: bool,
    max_single_turn_injection: int,
) -> str:
    """Decide whether a time-based cadence is sound for this session.

    Returns the trigger strategy to install. Raises if the caller supplies
    parameters under which neither strategy can guarantee the overflow bound.
    """
    if not (0.0 < alpha < 1.0):
        raise CompactionInvariantError(
            f"alpha={alpha} outside (0,1); budget fraction is meaningless."
        )

    headroom = int((1.0 - alpha) * window_tokens) - reserved_overhead
    if headroom <= 0:
        raise CompactionInvariantError(
            "No headroom after reserving overhead: "
            f"window={window_tokens}, alpha={alpha}, overhead={reserved_overhead}."
        )

    # A single turn larger than the headroom can overflow before any
    # admission-time check can intervene unless we budget per-admission.
    if max_single_turn_injection > headroom:
        # Token budgeting at admission is the only sound option; the cadence
        # cannot absorb a burst larger than the headroom it protects.
        return "token_budgeted_at_admission"

    cadence_absorbs = max_tokens_per_interval <= headroom
    if cadence_absorbs and not has_idle_gaps:
        return "time_based"

    if has_idle_gaps:
        # Timer would fire on an under-full buffer and compound summary loss.
        return "token_budgeted_at_admission"

    # Rate exceeds cadence absorption capacity: schedule is unsound.
    raise CompactionInvariantError(
        "Time-based cadence is unsound: max_tokens_per_interval="
        f"{max_tokens_per_interval} exceeds headroom={headroom}. "
        "Install token-budgeted compaction at admission."
    )


def admit_turn(live_tokens: int, incoming_tokens: int, window_tokens: int,
               alpha: float, compact_fn) -> int:
    """Admission-time budget check: compact BEFORE the injection lands."""
    budget = int(alpha * window_tokens)
    if live_tokens + incoming_tokens > budget:
        live_tokens = compact_fn(live_tokens)  # returns post-compaction size
        if live_tokens + incoming_tokens > window_tokens:
            raise CompactionInvariantError(
                "Post-compaction buffer still cannot admit turn: "
                f"live={live_tokens}, incoming={incoming_tokens}, "
                f"window={window_tokens}."
            )
    return live_tokens + incoming_tokens
```