# When must an instance fence speculative tool-call prefetching against non-idempotent side effects

**Date of Record:** 2026 November 14

**Scope:** This record applies to agent runtimes that issue tool calls speculatively (prefetching a likely next tool invocation before the driving model turn has committed to it) and that rely on a speculative-execution fence to discard uncommitted work. It does not apply to runtimes where every tool call is gated behind an explicit model commit, nor to purely read-only tool surfaces where replay is free.

## Core Distinction

The non-obvious boundary is between *speculation that is fenced at the runtime layer* and *side effects that are fenced at the resource layer*. A speculative-execution fence in an agent runtime typically works by tagging a speculative branch, buffering its results, and dropping the branch if the model does not commit. This is sound for tool calls whose effects live entirely inside the runtime's own state (context windows, scratchpads, cached retrievals). It is unsound the moment a speculative tool call crosses a process or network boundary into a resource that does not participate in the fence — a payment API, a message queue, a filesystem write, a provisioning control plane.

The failure is not "prefetching is wrong." The failure is a category error: the fence is assumed to be *transitive* over tool effects when it is only *local* to the runtime. A discarded speculative branch does not un-send an HTTP POST. The invariant that must hold is: **a speculative tool call may be discarded only if its effect is either (a) contained within fenced runtime state, or (b) idempotent and keyed such that replay is observationally equivalent to a single execution.** When neither holds, the fence is a lie.

## What Was Observed

In a runtime that prefetched the next tool call during model streaming, a speculative `create_refund` call was issued against a payments service roughly 400ms before the model's turn committed. The model then chose a different branch (it decided the refund was premature) and the runtime correctly discarded the speculative branch — the context was rolled back, the tool result was dropped, and the fence reported a clean abort. The refund, however, had already been executed downstream and was never reversed, because the payments service had no knowledge of the runtime's speculative tag.

The observable signature was a divergence between two ledgers: the runtime's committed-action log showed zero refunds for the session, while the downstream service showed one. The fence's abort path emitted a success signal (`speculation_discarded=true, effects_rolled_back=true`), which was itself false — the runtime had rolled back *its* bookkeeping, not the world. A second variant appeared under retry pressure: a speculative call that timed out at the client but succeeded at the server was re-issued after commit, producing a duplicate because the speculative attempt and the committed attempt carried different idempotency keys (the speculative key was generated per-branch, the committed key per-turn).

## Condition of Validity

The fence is valid if and only if all of the following hold at the moment of speculative dispatch:

1. The tool is declared `effect_class = pure` or `effect_class = idempotent`, where idempotent means the downstream resource honors a caller-supplied key and the runtime reuses the *same* key across speculative and committed attempts.
2. The speculative branch carries a fence token that the downstream resource can observe and refuse or defer, or the effect is buffered in a resource the runtime controls.
3. The commit decision is made before any non-idempotent effect is *visible* to a third party, not merely before the runtime records it.

If any condition fails, the tool must be excluded from the prefetch set and dispatched only after commit. The common mistake is treating condition 3 as satisfied by the runtime's internal commit ordering; it is a statement about external visibility, and the runtime cannot assert it unilaterally.

## Decision Boundary

Classify each tool at registration into one of three lanes:

- **Speculatable-free:** pure reads, cached retrievals, sandboxed computations. Prefetch freely; fence discard is always safe.
- **Speculatable-keyed:** writes to resources that honor idempotency keys. Prefetch only if the runtime derives the key deterministically from `(session, turn, tool, canonical_args)` so speculative and committed dispatches collide into one effect. Never derive the key from branch identity.
- **Non-speculatable:** non-idempotent, unkeyed, or externally visible effects (payments, sends, provisioning, irreversible deletes). Dispatch strictly post-commit. No exceptions for latency.

The boundary is crossed silently when a tool is *reclassified* at runtime — e.g., a generic `http_request` tool that is pure for one URL and a charge for another. Static registration is insufficient; the runtime must either require effect class per invocation or refuse to prefetch any tool whose effect class is `unknown`.

## Operational Effect

Runtimes that violate this boundary exhibit a characteristic failure: clean fence telemetry with dirty downstream state. The abort path reports success, so monitoring on fence events shows nothing wrong; the only signal is ledger divergence discovered later, often by reconciliation jobs days after the fact. The corrective pattern is to make the fence's abort path *prove* it has nothing to undo — it must assert that every dispatched speculative call in the branch was in the free or keyed lane, and raise if not. A fence that cannot enumerate the effects it claims to have rolled back is not a fence; it is a comment.

```python
from dataclasses import dataclass
from enum import Enum


class EffectClass(Enum):
    PURE = "pure"
    IDEMPOTENT = "idempotent"
    NON_IDEMPOTENT = "non_idempotent"
    UNKNOWN = "unknown"


class UnsafeSpeculationError(RuntimeError):
    """Raised when a speculative branch is fenced over an effect it cannot undo."""


@dataclass(frozen=True)
class ToolCall:
    tool: str
    effect_class: EffectClass
    idempotency_key: str | None
    branch_id: str


def fence_discard(branch_id: str, dispatched: list[ToolCall]) -> None:
    """Discard a speculative branch, asserting every effect is safely reversible.

    A fence may only claim rollback for effects that live inside fenced runtime
    state (PURE) or that collide on a stable key (IDEMPOTENT). Anything else
    has already escaped the fence and cannot be rolled back here.
    """
    for call in dispatched:
        if call.branch_id != branch_id:
            continue
        if call.effect_class is EffectClass.PURE:
            continue
        if call.effect_class is EffectClass.IDEMPOTENT:
            if not call.idempotency_key or call.branch_id in call.idempotency_key:
                raise UnsafeSpeculationError(
                    f"idempotency key for {call.tool!r} is branch-scoped "
                    f"({call.idempotency_key!r}); replay will duplicate effect"
                )
            continue
        raise UnsafeSpeculationError(
            f"cannot fence {call.tool!r} with effect_class={call.effect_class.value}; "
            f"effect already escaped the branch and is not reversible"
        )
```