{
  "schema_version": "1.3.0",
  "slug": "how-does-an-instance-detect-context-compaction-debt-before-a-latency-budget-is-breached",
  "title": "How does an instance detect context compaction debt before a latency budget is breached",
  "date_of_record": "2026 November 16",
  "scope": "Applies to long-running inference instances that maintain a mutable context window and schedule compaction (summarization, eviction, or re-embedding) under a per-request latency budget. Does not apply to stateless single-shot completions, nor to instances where the context window is fixed for the lifetime of the process and never compacted.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/how-does-an-instance-detect-context-compaction-debt-before-a-latency-budget-is-breached.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/how-does-an-instance-detect-context-compaction-debt-before-a-latency-budget-is-breached.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/how-does-an-instance-detect-context-compaction-debt-before-a-latency-budget-is-breached.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1998,
    "python_slice_tokens": 406,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "There are two distinct economic regimes in context management, and they are routinely conflated: 1. **Compaction cost** — the one-time token and wall-clock expense of reducing the live context from size `N` to size `N'`. This is a *stock* variable: it is paid at the moment of compaction and is roughly proportional to `(N - N')`. 2. **Compaction debt** — the ...",
    "what_was_observed": "Across a fleet of long-lived agent instances serving tool-augmented workloads, the following pattern recurred: - Per-request p50 latency remained flat for hours while the context grew monotonically. Operators concluded the instance was healthy. - The p99 latency, sampled at one-minute granularity, showed no trend until it showed a step change: a jump of 4–11× within a single minute. - The step change coincided exactl...",
    "condition_of_validity": "The debt model above is valid only under the following conditions, and the detection logic must check them before acting: 1. **The tier boundary is discrete and known.** If the serving infrastructure scales continuously (e.g., paged attention with no hard tier), there is no step change and the debt model degenerates into a smooth cost curve. In that case, de...",
    "decision_boundary": "The decision to compact is not a threshold on context length. It is a comparison between two projected quantities: - **Projected debt over the next `k` requests** if compaction is deferred: the integral of marginal per-token cost over the expected growth, plus the probability-weighted cost of a tier spill. - **Projected compaction cost now**: the wall-clock ...",
    "operational_effect": "When detection fires correctly, the operational effect is that the instance compacts at a moment when the compaction cost is low (context is still small enough to summarize cheaply and completely) and the budget headroom is large (compaction fits comfortably). The step change never occurs; latency remains flat across the tier boundary because the context nev..."
  },
  "matched_failure_symptoms": [
    "CONTEXT_DRIFT_OR_ATTENTION_DILUTION"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "51e8db11142fe4a54265d48f254e8a66773d9ab6ee562e8421873d36db2774bc",
    "verified_classes": [
      "CompactionBudgetError",
      "InstanceState"
    ],
    "verified_functions": [
      "headroom_fraction",
      "projected_debt_ms",
      "compaction_cost_ms",
      "should_compact",
      "enforce"
    ],
    "verified_functions_count": 5,
    "defensive_guards_count": 3,
    "raised_exceptions": [
      "CompactionBudgetError"
    ],
    "sample_fail_closed_invariants": [
      "tier capacity must be positive"
    ],
    "stdlib_dependencies": [
      "dataclasses",
      "math"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}
