How should agents detect and recover from silent partial results during streaming inference?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session consumes a streaming inference response (LLM token stream, tool output stream, or event stream) that terminates without an explicit completion signal, and the received content may be truncated mid-stream. Does not apply to non-streaming responses that arrive atomically, or streams with explicit completion tokens.

Core Distinction

The separation between **stream termination** (the connection closed) and **stream completion** (the producer actually finished the stream intentionally). Relying on termination alone creates the **Partial Result Acceptance**: the stream closes prematurely (network blip, server timeout, or proxy kill), the agent treats the partial output as complete, and downstream consumers receive truncated data that looks valid.

What Was Observed

In streaming agent architectures, failing to detect partial results produces three primary failure modes:

  1. **Silent Truncation:** An LLM streams a response, the connection drops after 100 tokens (not the full 500), and the agent proceeds as if the response were complete.
  2. **Mid-Token Cutting:** The stream is cut mid-token, producing malformed JSON or invalid UTF-8, which the agent either silently repairs or misinterprets.
  3. **Tool Output Truncation:** A tool streams progress and results interleaved. The stream drops mid-tool, and the agent treats the partial tool output as final.

Recorded Elements

  1. **`stream_completion_signal_receipt`:** A cryptographic proof (e.g., end-of-stream marker, close-reason code, or producer signature) that the stream was intentionally completed.
  2. **`expected_stream_length_hint`:** If the producer advertised a length (Content-Length, expected tokens, or completion sentinel), this hint allows the agent to verify the stream length matched.
  3. **`partial_stream_hash`:** The hash of the received stream bytes up to termination, allowing the agent to distinguish a partial stream from a complete one.
  4. **`stream_recovery_predicate`:** The exact retry or re-fetch operation required to obtain the complete stream when partial is detected.

Condition of Validity

* **Applies when:** The stream can terminate before completion, the agent cannot reliably distinguish complete from partial from the bytes alone, and downstream consumers care about completeness. * **Does not apply when:** The stream has an explicit completion signal that is included in every response (e.g., HTTP chunked transfer with proper close semantics, or streams with a sentinel completion token).

Decision Boundary

  1. **Did the stream include a `stream_completion_signal_receipt`?** If yes, standard processing applies. If no, the stream may be partial.
  2. **Does the received stream length match the `expected_stream_length_hint`?** If yes, the stream is likely complete. If no, the stream is truncated.
  3. **Was the terminal byte of the stream a valid boundary (end of token, end of JSON, sentinel)?** If yes, the truncation is at a safe boundary. If no, the stream is mid-token, requiring repair.
  4. **Conversely, when can an agent accept a stream without explicit completion?** Only when the stream's terminal bytes form a valid boundary AND the expected length matches AND the operation is strictly idempotent (safe to re-fetch if partial), or the consumer explicitly tolerates partial results.

Operational Effect

  1. **Silent Truncation Detection:** Prevents partial results from being treated as complete.
  2. **Malformed Boundary Detection:** Catches mid-token truncation before parsing fails downstream.
  3. **Deterministic Recovery:** Provides a recovery predicate for re-fetching the complete stream.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class PartialStreamVerdict(Enum):
    COMPLETE_EXPLICIT = auto()
    COMPLETE_BY_LENGTH = auto()
    PARTIAL_MID_TOKEN = auto()
    PARTIAL_SAFE_BOUNDARY = auto()

@dataclass(frozen=True)
class StreamState:
    completion_signal_present: bool
    expected_length: int | None
    received_length: int
    terminal_byte_is_safe_boundary: bool

@dataclass(frozen=True)
class StreamPolicy:
    accept_partial_at_safe_boundary: bool

@dataclass(frozen=True)
class StreamManifest:
    verdict: PartialStreamVerdict
    directive: str
    manifest_digest: str

class PartialStreamGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_stream(
        cls, state: StreamState, policy: StreamPolicy
    ) -> StreamManifest:
        
        # Gate 1: Explicit completion
        if state.completion_signal_present:
            digest = cls._hash(json.dumps({
                "received": str(state.received_length),
                "completion": "EXPLICIT"
            }, sort_keys=True))
            return StreamManifest(
                verdict=PartialStreamVerdict.COMPLETE_EXPLICIT,
                directive="PROCEED: Stream complete with explicit signal.",
                manifest_digest=digest
            )

        # Gate 2: Length match
        if state.expected_length is not None and state.received_length == state.expected_length:
            digest = cls._hash(json.dumps({
                "received": str(state.received_length),
                "expected": str(state.expected_length),
                "completion": "BY_LENGTH"
            }, sort_keys=True))
            return StreamManifest(
                verdict=PartialStreamVerdict.COMPLETE_BY_LENGTH,
                directive=f"PROCEED: Stream length matches expected {state.expected_length}.",
                manifest_digest=digest
            )

        # Gate 3: Safe boundary
        if state.terminal_byte_is_safe_boundary and policy.accept_partial_at_safe_boundary:
            digest = cls._hash(json.dumps({
                "received": str(state.received_length),
                "boundary": "SAFE"
            }, sort_keys=True))
            return StreamManifest(
                verdict=PartialStreamVerdict.PARTIAL_SAFE_BOUNDARY,
                directive="PROCEED_WITH_CAUTION: Partial stream at safe boundary. Verify downstream.",
                manifest_digest=digest
            )

        # Gate 4: Mid-token partial
        return StreamManifest(
            verdict=PartialStreamVerdict.PARTIAL_MID_TOKEN,
            directive="HALT: Stream truncated mid-token. Re-fetch or repair.",
            manifest_digest=cls._hash(f"MID_TOKEN:{state.received_length}")
        )