# In what scenarios must an instance refuse to commit a speculated tool invocation whose side-effect journal has already been fsynced but whose dependency token has expired

**Date of Record:** 2026 November 8

**Scope:** Applies to agent runtimes that speculatively pre-execute tool calls inside a sandbox, journal their side effects, and later commit them under a dependency token (lease) with a TTL. Does not apply to idempotent read-only tools, nor to systems where commit is a single atomic transaction with no separate journal flush.

## Core Distinction

Speculation and commit are two different clocks. The sandbox runs the tool *before* the dependency graph is settled; the commit protocol replays the journaled side effects *after* the dependency token that authorized the speculation has been validated. The non-obvious boundary is this: a side-effect journal can be durably persisted (fsync returned, WAL segment sealed) while the dependency token under which that journal was authorized has already expired. Persistence is not authorization. Many runtimes conflate "the journal is on disk" with "the effect is committable," and commit anyway on the theory that the work is already paid for. That is the failure.

The invariant: **a journaled side effect may only be committed under a token that was valid at the moment the journal entry was sealed, and that has not been invalidated by a superseding token for the same resource.** Expiry alone is not the trigger for refusal — expiry *plus* a re-derivation of the dependency graph that would now produce a different result is. If expiry occurred but the dependency closure is unchanged, commit is safe. If expiry occurred and the closure has changed (a new writer took the resource, a parent token was revoked, a superseding speculation won), the journal must be quarantined, not committed.

## What Was Observed

A planner fanned out three speculative branches against a shared resource `R` (a table row). Each branch pre-executed an `UPDATE`, sealed its journal under token `T_1` with a 400 ms TTL, and waited for the orchestrator to pick a winner. The orchestrator was delayed by a GC pause of 1.2 s. On resume, `T_1` had expired. The orchestrator's commit path checked only `journal.sealed == True` and `journal.checksum_valid == True`, both of which held, and committed all three updates. The last writer won, but the two losing branches had already emitted downstream side effects (a webhook enqueue, an audit row) that were keyed to their own now-discarded results. The audit rows referenced a row state that never existed. Reconciliation took four hours and required replaying the webhook queue against a reconstructed timeline.

The signal was not an error. The commit succeeded. The failure was visible only in the audit table's referential inconsistency, discovered downstream. There is no exception to catch; the protocol's success path is the bug.

## Condition of Validity

Commit is valid iff, at commit time:

1. The journal entry's `token_id` matches the currently active token for the resource, **or** the token is expired but a re-validation pass confirms the dependency closure is byte-identical to the closure at seal time.
2. No superseding token for the same resource was minted between seal and commit.
3. The journal's `seal_monotonic` is strictly less than the current token's `mint_monotonic` (no clock inversion across the seal).
4. The resource's generation counter has not advanced since the journal captured it.

Conditions 2 and 4 are the ones routinely skipped. A token can be expired and yet still the active one (no superseder); a token can be unexpired and yet superseded (a faster branch already committed). Expiry is a weak proxy for supersession and must never be used in place of it.

## Decision Boundary

The decision is not "expired → refuse." It is a two-axis table over (token expired?, closure changed?):

- **Not expired, closure unchanged:** commit.
- **Not expired, closure changed:** refuse — a superseder exists even though the clock has not run out. This is the case that surprises operators, because the token looks healthy.
- **Expired, closure unchanged:** commit, but re-seal the journal under a fresh token first so downstream readers see a coherent authorization lineage. Do not commit under the dead token id.
- **Expired, closure changed:** refuse, quarantine the journal, and emit a `StaleSpeculationError` carrying both token ids and the diff of the closure.

The boundary that matters operationally is the second row: healthy-looking token, changed closure. Any commit path that gates only on TTL will pass it and corrupt the resource.

## Operational Effect

After the incident, the commit path was rewritten to always re-derive the dependency closure and compare it to the sealed closure, regardless of TTL state. The cost was one extra graph walk per commit (~3 ms at p50, ~40 ms at p99 under contention), which was acceptable because commits are batched. The benefit was that the TTL became advisory rather than load-bearing: the runtime no longer relies on wall-clock expiry to detect supersession, which was always a race. Quarantined journals are retained for 24 h and replayed only by an explicit reconciliation job that the planner cannot trigger.

The residual risk is closure re-derivation cost on very large graphs, where the walk itself can exceed the token TTL and produce a self-inflicted expiry. The mitigation is to derive the closure once at seal, hash it, and compare hashes at commit; the hash comparison is O(1) and the closure is only re-derived if the hash mismatches, which is the rare path.

```python
import hashlib
import time
from dataclasses import dataclass, field
from typing import Optional


class StaleSpeculationError(RuntimeError):
    """Raised when a journaled side effect cannot be committed under its token."""

    def __init__(self, token_id: str, reason: str, sealed_closure_hash: str,
                 current_closure_hash: Optional[str] = None):
        self.token_id = token_id
        self.reason = reason
        self.sealed_closure_hash = sealed_closure_hash
        self.current_closure_hash = current_closure_hash
        super().__init__(
            f"refusing commit under token {token_id}: {reason} "
            f"(sealed={sealed_closure_hash[:12]}, "
            f"current={(current_closure_hash or 'n/a')[:12]})"
        )


@dataclass
class DependencyToken:
    token_id: str
    resource: str
    mint_monotonic: float
    ttl_s: float
    generation: int
    superseded_by: Optional[str] = None

    def expired(self, now: Optional[float] = None) -> bool:
        now = time.monotonic() if now is None else now
        return (now - self.mint_monotonic) > self.ttl_s


@dataclass
class JournalEntry:
    token_id: str
    resource: str
    seal_monotonic: float
    sealed_closure_hash: str
    sealed_generation: int
    sealed: bool = False
    fsynced: bool = False


def commit_journaled_effect(
    entry: JournalEntry,
    token: DependencyToken,
    current_generation: int,
    derive_closure_hash,
    now: Optional[float] = None,
) -> str:
    """Commit a speculated side effect, or refuse. Returns the commit token id."""
    if not (entry.sealed and entry.fsynced):
        raise StaleSpeculationError(
            token.token_id, "journal not durably sealed", entry.sealed_closure_hash
        )

    if entry.token_id != token.token_id:
        raise StaleSpeculationError(
            token.token_id, "token id mismatch", entry.sealed_closure_hash
        )

    if entry.seal_monotonic < token.mint_monotonic:
        raise StaleSpeculationError(
            token.token_id, "clock inversion: sealed before token mint",
            entry.sealed_closure_hash,
        )

    if token.superseded_by is not None:
        raise StaleSpeculationError(
            token.token_id, f"superseded by {token.superseded_by}",
            entry.sealed_closure_hash,
        )

    if current_generation != entry.sealed_generation:
        raise StaleSpeculationError(
            token.token_id, "resource generation advanced",
            entry.sealed_closure_hash,
        )

    current_hash = derive_closure_hash(entry.resource)
    if current_hash != entry.sealed_closure_hash:
        raise StaleSpeculationError(
            token.token_id, "dependency closure changed",
            entry.sealed_closure_hash, current_hash,
        )

    if token.expired(now):
        # Expired but closure identical: re-seal under a fresh token, then commit.
        return _reseal_and_commit(entry, token)

    return token.token_id


def _reseal_and_commit(entry: JournalEntry, old: DependencyToken) -> str:
    new_id = hashlib.sha256(
        f"{old.token_id}:{entry.seal_monotonic}".encode()
    ).hexdigest()[:16]
    entry.token_id = new_id
    return new_id
```