# Under what conditions must an instance abort a speculative branch before merge when side effects have already leaked to a shared external sink

**Date of Record:** 2026 November 12

**Scope:** Applies to orchestrators that execute speculative multi-agent branches whose task graphs merge into a committed linearization, and that permit branches to perform external side effects (writes to queues, object stores, APIs, databases) under a rollback-safe containment protocol. Does not apply to purely functional branches with no external sinks, nor to systems where all side effects are deferred to post-merge commit.

## Core Distinction

The distinction that matters is between **speculative divergence** and **side-effect leakage**. Speculative divergence is the normal, healthy condition: two branches produce different intermediate results, one is selected, the other is discarded. Side-effect leakage is the pathological condition: a discarded branch has already mutated state that the merge protocol cannot atomically reverse because the sink is outside the transaction boundary.

Most rollback-safe containment designs assume that "rollback-safe" means "we can undo it." The non-obvious invariant is that **rollback-safety is a property of the sink's compensation semantics, not of the orchestrator's intent.** A branch is only safely abortable if every side effect it emitted is either (a) compensable by a registered inverse operation whose execution is guaranteed, or (b) isolated behind a fence that the merge step can atomically discard. If a branch emits an effect that is neither compensable nor fenced, the branch has crossed the abort boundary and must not be silently discarded.

The failure mode is subtle because the effect often *looks* compensable. A `POST /orders` can be "undone" by a `DELETE /orders/{id}` — until the downstream consumer has already read the order, or the delete fails, or the order id was never returned because the response was lost after commit. Compensation is not inversion; it is a best-effort approximation with its own failure modes.

## What Was Observed

In a branch-and-merge graph where branch B1 (speculative) and B2 (speculative) both emit to a shared idempotency-keyed sink, the following sequence was observed:

1. B1 acquires idempotency key `k` and writes payload `p1`. Sink commits `p1` under `k`.
2. B2, unaware of B1's write (speculation isolation), acquires the *same* key `k` — because the key was derived from the merge-point task id, not the branch id — and attempts to write `p2`. The sink, honoring idempotency, returns the *original* `p1` result and does not write `p2`.
3. The merge step selects B2 as the winner based on its in-memory result.
4. The merge commits, believing `p2` is durable. It is not. `p1` is durable, and `p1` came from the losing branch.
5. Rollback of B1 is attempted. The compensation `DELETE k` runs. It removes `p1`. Now the sink has *neither* `p1` nor `p2`, but the merge has committed.

The observed invariant violation: **idempotency keys derived from merge-point identity collapse branch identity, causing the winning branch's effect to be silently replaced by the losing branch's effect, and subsequent compensation to delete the only surviving record.** This is not a race in the usual sense — it is deterministic under the stated key derivation. It reproduces whenever two speculative branches share a merge-point-derived key and the sink is first-writer-wins.

A second observation: when the sink was changed to last-writer-wins, the same graph produced the opposite failure — the losing branch's late write overwrote the winner's, and the merge committed a pointer to state that no longer matched the selected branch's result. Both key-derivation strategies fail, in opposite directions, because the root cause is that **branch identity was not encoded in the effect identity.**

## Condition of Validity

The rollback-safe containment protocol is valid — meaning abort-and-discard is sound — if and only if all of the following hold at the moment of abort:

- **C1 (Effect identity includes branch identity).** Every emitted effect carries a key that is a function of `(graph_id, branch_id, task_id, attempt)`, not merely `(graph_id, task_id)`. If two branches can collide on a key, C1 fails.
- **C2 (Compensation is registered before emission).** The inverse operation is durably recorded in the orchestrator's log *prior* to the sink call, so that a crash between emission and registration does not orphan the effect. Register-after-emit is a C2 violation even if the registration usually succeeds.
- **C3 (Compensation is confirmed, not assumed).** The abort path must observe the compensation's terminal state. Fire-and-forget compensation is a C3 violation; a failed compensation converts an abort into an *unresolved leak* that requires escalation, not silent discard.
- **C4 (The sink is fenced or the effect is deferred).** Either the sink is a transactional resource enrolled in the merge's commit protocol (so the abort is a rollback, not a compensation), or the effect is buffered and only flushed post-merge. If neither, the effect is a live external mutation and the branch is not abortable without C1–C3.

If any of C1–C4 fails, the branch is **not abortable** in the rollback-safe sense. The correct action is not "abort anyway and hope compensation works." The correct action is to escalate to a **quarantine-and-reconcile** path: freeze the branch, mark the graph as `dirty`, and require an explicit reconciliation decision (commit-the-leak, or human/automated repair) before the merge can proceed.

## Decision Boundary

The decision boundary is between **abort** and **quarantine**. The naive rule "if the branch lost, abort it" is wrong whenever leakage has occurred. The correct rule:

- If the branch lost **and** C1–C4 hold for every emitted effect → abort, run compensations, confirm, proceed.
- If the branch lost **and** any emitted effect violates C1–C4 → do **not** abort. Quarantine the branch, mark the graph `dirty`, and refuse to advance the merge until reconciliation resolves the leaked effects. The merge may still select the other branch, but the graph cannot be marked `clean`.
- If the branch won **but** a losing branch leaked an effect that collides with the winner's effect identity (the key-collision case above) → the merge itself is unsafe. The winner's effect may already have been shadowed. Escalate to reconciliation; do not commit.

The boundary is *not* "did the branch emit anything." It is "can every emitted effect be provably reversed or fenced." Emission volume is irrelevant; a single unfenced, uncompensable effect is disqualifying. Conversely, a branch that emitted a thousand fenced effects is fully abortable.

## Operational Effect

The practical consequence is that the orchestrator must maintain a per-branch **effect ledger** with three states per effect: `pending` (registered, not yet emitted), `emitted` (sink acknowledged, compensation registered), and `compensated` (inverse confirmed). Abort is permitted only when every ledger entry is `compensated` or `pending`. Any entry in `emitted` with a failed or unregistered compensation transitions the graph to `dirty` and blocks merge.

This changes the failure taxonomy: a "failed abort" is no longer an error to retry blindly — it is a signal that the speculation boundary was misconfigured (C1 or C2 violated at emission time). The ledger becomes the audit trail that lets reconciliation decide, per effect, whether to commit the leak or repair it. Graphs that reach `dirty` must be drained before the orchestrator accepts new speculation on the same sink, or key collisions will compound.

The invariant to encode as an assertion: **a graph may transition to `clean` after a merge only if every non-winning branch's effect ledger is fully `compensated` or `pending`, and every winning branch's effects are `emitted` under keys that no other branch could have produced.** Violating this assertion is the condition under which an instance must abort — or, more precisely, must refuse to abort and escalate instead.

```python
class SpeculationLeakError(RuntimeError):
    """Raised when a speculative branch cannot be safely aborted because
    an emitted side effect violates the containment invariants C1-C4."""


class EffectLedger:
    PENDING = "pending"
    EMITTED = "emitted"
    COMPENSATED = "compensated"

    def __init__(self):
        self._entries = {}

    def register(self, effect_id, key, compensation):
        # C2: compensation must be registered before emission.
        if compensation is None:
            raise SpeculationLeakError(
                f"effect {effect_id} has no registered compensation; C2 violated"
            )
        self._entries[effect_id] = {
            "key": key,
            "state": self.PENDING,
            "compensation": compensation,
        }

    def mark_emitted(self, effect_id):
        self._entries[effect_id]["state"] = self.EMITTED

    def mark_compensated(self, effect_id):
        self._entries[effect_id]["state"] = self.COMPENSATED

    def is_abortable(self, branch_id, branch_effects):
        for eid in branch_effects:
            entry = self._entries.get(eid)
            if entry is None:
                raise SpeculationLeakError(
                    f"branch {branch_id} emitted unledgered effect {eid}"
                )
            if entry["state"] == self.EMITTED and entry["compensation"] is None:
                return False
        return True


def abort_branch(branch_id, branch_effects, ledger, sink):
    if not ledger.is_abortable(branch_id, branch_effects):
        # Do NOT abort. Quarantine and escalate.
        raise SpeculationLeakError(
            f"branch {branch_id} has uncompensable emitted effects; "
            f"graph must be marked dirty and reconciled, not aborted"
        )

    for eid in branch_effects:
        entry = ledger._entries[eid]
        if entry["state"] == ledger.EMITTED:
            try:
                sink.run_compensation(entry["compensation"])
            except Exception as exc:
                # C3: compensation failed; abort is unresolved.
                raise SpeculationLeakError(
                    f"compensation for {eid} failed: {exc}; "
                    f"branch {branch_id} is now an unresolved leak"
                ) from exc
            ledger.mark_compensated(eid)
    return True


def verify_merge_clean(winning_branch, all_branches, ledger):
    for bid, effects in all_branches.items():
        if bid == winning_branch:
            continue
        for eid in effects:
            state = ledger._entries[eid]["state"]
            if state == ledger.EMITTED:
                raise SpeculationLeakError(
                    f"non-winning branch {bid} left effect {eid} emitted; "
                    f"merge cannot be marked clean"
                )
    return True
```