How must an instance detect silent context window truncation by an upstream proxy?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session sends prompts or context to an upstream LLM API through an intermediary proxy (API gateway, LLM router, or enterprise middleware), and the proxy may silently truncate the prompt to fit its own context window before forwarding to the model. Does not apply to direct API calls without intermediaries, or proxies that return explicit truncation errors.

Core Distinction

The separation between **application-level context limit** (the limit the agent enforces, based on the model's advertised window) and **proxy-level context limit** (the limit the proxy enforces, often smaller and undocumented). Relying on the application limit creates the **Silent Truncation Blindness**: the agent sends a 128k-token prompt, the proxy cuts it to 32k tokens, the model responds based on truncated context, and the agent has no way to know the response is based on incomplete input.

What Was Observed

In proxy-mediated LLM architectures, failing to detect silent truncation produces three primary failure modes:

  1. **Hallucinated Context:** The agent assumes the model read the full document. The model responds based on the first 32k tokens, missing crucial later content, and the agent acts on incomplete information.
  2. **Truncation Drift:** The proxy's context limit changes silently (e.g., after a version upgrade). Prompts that worked yesterday now truncate, and the agent's behavior silently degrades.
  3. **Compliance Gap:** The prompt contains PII or proprietary content that was truncated, but the agent has no proof of what was actually sent to the model, violating data governance.

Recorded Elements

  1. **`prompt_payload_hash_sent`:** The hash of the prompt as sent by the agent, and the hash of the prompt as received by the model (if the model echoes it back or the proxy provides a receipt).
  2. **`proxy_context_limit_probe`:** A pre-flight probe that measures the proxy's effective context limit by sending prompts of increasing length and observing responses.
  3. **`truncation_detection_receipt`:** A cryptographic proof that the prompt hash at send-time matched the hash at model-reception-time, confirming no truncation occurred.
  4. **`proxy_context_limit_signature`:** The proxy's advertised context limit and version, allowing the agent to detect drift when the limit changes.

Condition of Validity

* **Applies when:** The agent sends prompts through an intermediary proxy that may enforce its own context limit, and the consequence of silent truncation is material. * **Does not apply when:** The agent calls the model API directly (no proxy), or the proxy returns an explicit error on truncation.

Decision Boundary

  1. **Does the agent's prompt route through an intermediary proxy?** If no, standard context management applies. If yes, proceed.
  2. **Is the prompt length below the proxy's advertised context limit?** If yes, no truncation expected. If no, truncation is likely. The agent must either shrink the prompt or route around the proxy.
  3. **Was the prompt hash verified at model reception?** If no, the agent cannot prove truncation did not occur. It must verify via echo or receipt.
  4. **Conversely, when can an agent trust a proxy without truncation verification?** Only when the proxy is explicitly configured to pass through prompts unmodified with a contractual SLA, and the agent's prompts are known to be well below the proxy's limit.

Operational Effect

  1. **Truncation Blindness Prevention:** Forces the agent to verify what the model actually received, not just what was sent.
  2. **Proxy Drift Detection:** Detects silent changes to the proxy's context limit, preventing behavior drift.
  3. **Data Governance:** Provides proof of what was actually sent for compliance audits.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class TruncationVerdict(Enum):
    NO_TRUNCATION_VERIFIED = auto()
    TRUNCATION_SUSPECTED = auto()
    TRUNCATION_CONFIRMED = auto()

@dataclass(frozen=True)
class ProxyContextState:
    prompt_hash_sent: str
    prompt_hash_received: str | None
    prompt_token_count: int
    proxy_context_limit_tokens: int

@dataclass(frozen=True)
class TruncationPolicy:
    require_receipt: bool
    max_acceptable_ratio: float

@dataclass(frozen=True)
class TruncationManifest:
    verdict: TruncationVerdict
    directive: str
    manifest_digest: str

class ProxyTruncationGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_prompt(
        cls, state: ProxyContextState, policy: TruncationPolicy
    ) -> TruncationManifest:
        
        # Gate 1: Prompt within proxy limit
        if state.prompt_token_count <= state.proxy_context_limit_tokens:
            if state.prompt_hash_received and state.prompt_hash_sent != state.prompt_hash_received:
                return TruncationManifest(
                    verdict=TruncationVerdict.TRUNCATION_CONFIRMED,
                    directive="HALT: Prompt within limit but hash mismatch. Silent truncation.",
                    manifest_digest=cls._hash(f"MISMATCH:{state.prompt_hash_sent}")
                )
            digest = cls._hash(json.dumps({
                "sent": state.prompt_hash_sent,
                "tokens": str(state.prompt_token_count),
                "limit": str(state.proxy_context_limit_tokens)
            }, sort_keys=True))
            return TruncationManifest(
                verdict=TruncationVerdict.NO_TRUNCATION_VERIFIED,
                directive="PROCEED: Prompt within proxy limit.",
                manifest_digest=digest
            )

        # Gate 2: Prompt exceeds proxy limit
        ratio = state.prompt_token_count / state.proxy_context_limit_tokens

        if policy.require_receipt and state.prompt_hash_received is None:
            return TruncationManifest(
                verdict=TruncationVerdict.TRUNCATION_SUSPECTED,
                directive=f"HALT: Prompt {ratio:.2f}x proxy limit, no reception hash. Cannot verify.",
                manifest_digest=cls._hash(f"NO_RECEIPT:{ratio:.2f}")
            )

        # Gate 3: Confirmed truncation
        if state.prompt_hash_received and state.prompt_hash_sent != state.prompt_hash_received:
            return TruncationManifest(
                verdict=TruncationVerdict.TRUNCATION_CONFIRMED,
                directive=f"HALT: Hash mismatch confirms truncation ({ratio:.2f}x limit).",
                manifest_digest=cls._hash(f"CONFIRMED:{ratio:.2f}")
            )

        # Gate 4: Suspected
        return TruncationManifest(
            verdict=TruncationVerdict.TRUNCATION_SUSPECTED,
            directive=f"WARN: Prompt {ratio:.2f}x proxy limit with no hash verification.",
            manifest_digest=cls._hash(f"SUSPECTED:{ratio:.2f}")
        )