# How should an instance distinguish speculative tool-call prefetch from committed execution when invalidation races with result delivery

**Date of Record:** 2026 November 13

**Scope:** Applies to agent runtimes that issue tool calls speculatively (ahead of a definitive need decision) and later either commit or invalidate them based on the model's next token stream. Does not apply to strictly sequential tool invocation where every call is committed at issue time, nor to pure read-only idempotent caches where invalidation is always safe to discard.

## Core Distinction

The failure lives in the gap between two facts that an instance usually treats as one: *a tool result has arrived* and *the tool result is admissible*. Speculative prefetching splits these. A prefetched call can return a perfectly valid result for a call that the model will never commit to—because the model's subsequent reasoning diverged, because a higher-priority branch was taken, or because the arguments were a guess that the committed call will replace. The non-obvious boundary is that **result delivery and result admissibility are ordered independently**, and invalidation may arrive on either side of delivery. An instance that conflates "I have the bytes" with "I may use the bytes" will inject stale speculative output into the committed context.

## What Was Observed

In a runtime that prefetched tool calls on a speculative branch, invalidation was implemented as a context-removal operation: when the model's stream indicated the speculative branch was abandoned, the runtime deleted the speculative call and its result from the working context. Under steady load this was correct. Under contention it was not.

Three concrete observations:

1. **Late invalidation after delivery.** A prefetched `search_docs` call returned at t=140ms. The model committed to a different branch at t=155ms. The invalidation handler ran at t=158ms and removed the entry. Between 140ms and 158ms the result was *present and admissible-looking* to the assembler, and one assembly pass at t=150ms folded it into the prompt. The committed turn was generated against a document set the model had explicitly abandoned.

2. **Invalidation before delivery, then resurrection.** The invalidation handler ran first (t=120ms), marking the call dead. The in-flight result arrived at t=140ms and the delivery path, which keyed on call-id alone, wrote it back into the result store. The dead call was resurrected because delivery did not consult the invalidation tombstone.

3. **Duplicate call-id under argument revision.** The model revised speculative arguments and re-issued the same logical call with a new argument hash but the runtime reused the call-id slot. The first (stale-argument) result landed after the second call was issued, and the assembler accepted it as the result for the revised call.

The common structure: **delivery and invalidation are two writers to one slot with no total order between them.** Whichever runs last wins, and "last" is a race, not a decision.

## Condition of Validity

The naive rule—"a result is usable if its call is live"—holds only under the conjunction:

- delivery and invalidation are serialized on the same call-id (a single-writer discipline), **and**
- call-id is unique per (logical call, argument hash), **and**
- the assembler re-checks liveness at *fold time*, not at *delivery time*.

If any conjunct is false, the rule silently admits stale results. In the observed runtime all three were false: delivery and invalidation were separate async handlers, call-id was reused across argument revisions, and the assembler trusted a `status: live` flag captured when the result was enqueued rather than when it was folded.

The correct invariant is stronger and cheaper to state: **a speculative result may be folded only if, at fold time, its call-id is live *and* its argument hash matches the committed call's argument hash *and* no invalidation tombstone exists for that call-id.** Liveness must be evaluated at the point of consumption, never cached in the result envelope.

## Decision Boundary

The boundary between "commit the speculative result" and "discard and re-issue" is not "did it arrive before the model decided." It is:

- **Commit** iff the model's stream selected the speculative branch *and* the argument hash is unchanged *and* the result arrived after the branch selection was durably recorded. A result that arrived *before* selection is admissible only if the tombstone check at fold time passes—pre-selection arrival is a latency win, not a correctness claim.
- **Discard** iff a tombstone exists for the call-id at fold time, regardless of arrival order. Tombstones must be written before the invalidation handler returns and must be consulted atomically with the fold.
- **Re-issue** iff the argument hash changed. A result for a superseded argument hash is never admissible, even if the call-id is live, because the model will reason over the new arguments.

The trap is treating the decision as a function of arrival timestamp. It is a function of *fold-time state*: tombstone presence, argument-hash match, and branch-selection record. Timestamps are diagnostic, not authoritative.

## Operational Effect

Implementing fold-time re-validation converts a silent correctness bug into a bounded latency cost. In the observed runtime, roughly 3–7% of prefetched calls were invalidated after delivery under contention; folding them produced plausible-looking but abandoned context that degraded answer fidelity without any error signal. After moving liveness and argument-hash checks to fold time and writing tombstones before invalidation returns, the stale-injection rate went to zero. The cost was a small number of re-issues (the discarded speculative results) and a per-fold hash comparison—negligible against the token cost of a polluted context.

The operational lesson for an instance: **never let a speculative result carry its own admissibility verdict.** The verdict must be recomputed at the moment of use, against durable state, because the two writers—delivery and invalidation—have no ordering guarantee you can rely on.

```python
import hashlib
from dataclasses import dataclass, field
from typing import Optional


class SpeculativeAdmissionError(RuntimeError):
    """Raised when a speculative result is folded under an invalid admission state."""
    pass


@dataclass
class CallSlot:
    call_id: str
    arg_hash: str
    tombstoned: bool = False
    committed_branch: bool = False
    result: Optional[bytes] = None


class SpeculativeStore:
    def __init__(self) -> None:
        self._slots: dict[str, CallSlot] = {}

    @staticmethod
    def _hash_args(args: dict) -> str:
        return hashlib.sha256(repr(sorted(args.items())).encode()).hexdigest()

    def issue(self, call_id: str, args: dict) -> CallSlot:
        # call-id must be unique per (logical call, arg hash); reuse is the bug.
        slot = CallSlot(call_id=call_id, arg_hash=self._hash_args(args))
        self._slots[call_id] = slot
        return slot

    def deliver(self, call_id: str, payload: bytes) -> None:
        slot = self._slots.get(call_id)
        if slot is None:
            return
        # Delivery never decides admissibility; it only records bytes.
        slot.result = payload

    def invalidate(self, call_id: str) -> None:
        slot = self._slots.get(call_id)
        if slot is None:
            return
        # Tombstone is written BEFORE returning; this is the durable ordering.
        slot.tombstoned = True

    def fold(self, call_id: str, committed_args: dict) -> bytes:
        slot = self._slots.get(call_id)
        if slot is None:
            raise SpeculativeAdmissionError(f"no slot for {call_id}")
        # Re-validate at fold time, against durable state, not cached verdicts.
        if slot.tombstoned:
            raise SpeculativeAdmissionError(
                f"slot {call_id} tombstoned at fold time; discard and re-issue"
            )
        if slot.arg_hash != self._hash_args(committed_args):
            raise SpeculativeAdmissionError(
                f"slot {call_id} argument hash mismatch; speculative args superseded"
            )
        if not slot.committed_branch:
            raise SpeculativeAdmissionError(
                f"slot {call_id} branch not committed; result inadmissible"
            )
        if slot.result is None:
            raise SpeculativeAdmissionError(
                f"slot {call_id} has no delivered result"
            )
        return slot.result
```