# When must an instance reject a CRDT merge that is locally convergent but causally stale? **Date of Record:** 2026 November 5 **Scope:** Applies to blackboard agents that reconcile shared state via state-based or delta-state CRDTs (LWW-register, OR-Set, LWW-Map, MV-Register) where the merge function is idempotent and commutative but does *not* carry a causal read barrier. Does not apply to operation-based CRDTs with total-order broadcast, nor to systems using explicit vector-clock fencing at the write path. ## Core Distinction The non-obvious failure is the conflation of **semantic convergence** with **semantic freshness**. A CRDT guarantees that all replicas that have *seen the same set of updates* converge to identical state. It does **not** guarantee that a replica which has merged a delta is now operating on state that is causally *after* the state it used to make its last decision. In a blackboard architecture, an agent typically does: (1) read board, (2) compute a plan/claim, (3) write a claim back. If step 1 read a version `V_a` and step 3 writes an LWW-register keyed by wall-clock timestamp, a concurrent writer that read `V_b` (where `V_b` causally precedes `V_a` from the reader's perspective but has a *later* wall-clock stamp due to clock skew or a delayed delivery) will win the merge — and the board converges, but it converges to a state derived from a *stale read*. The merge is correct; the *decision* embedded in the merged value was made against an antecedent board. This is distinct from the classic "lost update" because the update is not lost — it is *preserved and propagated*. The system is fully convergent. The invariant that breaks is: **every winning write must have been computed from a board state that is a causal ancestor of the board state at merge time.** CRDTs enforce the merge algebra; they do not enforce the decision lineage. ## What Was Observed In a three-agent blackboard (planner, executor, verifier) sharing an LWW-Map keyed by `(key, hybrid_logical_clock)`, the following occurred repeatedly under partition-heal cycles of 2–9 seconds: - The executor read `task:42.status = "pending"` and `task:42.attempt = 3` at HLC `(t=1000, c=4)`. - It computed a claim `task:42.owner = "executor-A"` with HLC `(t=1000, c=7)` and wrote it. - Meanwhile the planner, having read an *older* snapshot at HLC `(t=999, c=9)` but with a wall-clock `t` that advanced during a GC pause, emitted `task:42.owner = "executor-B"` at HLC `(t=1002, c=1)`. - On heal, LWW resolved to `executor-B` (higher HLC). The board converged. Every replica agreed. - The verifier then observed `owner = executor-B` but `attempt = 3` — a state that *no agent had ever observed or produced*, because the planner's write was computed against `attempt = 2` and the executor's `attempt = 3` increment was in the same delta batch that got shadowed by the LWW resolution on the `owner` key. The merged state was a **Frankenstein tuple**: fields from different causal cuts, assembled by per-key LWW resolution. Convergence held; consistency of the *composite* record did not. The verifier subsequently double-executed the task because `owner` and `attempt` disagreed about whether a claim was in flight. ## Condition of Validity The convergence guarantee is only operationally sufficient when **one** of the following holds: 1. The CRDT is a **composite/multi-value register** (MV-Register) and the agent's read path is explicitly taught to detect and resolve multi-value conflicts *before* acting, rather than letting LWW silently pick a winner per field. 2. The blackboard is **partitioned by causal scope** — each key's writer set is a singleton at any time (single-writer-per-key), so per-key LWW never assembles a tuple from divergent cuts. 3. The agent attaches an explicit **read-set (dependency fingerprint)** to each write, and the merge layer rejects writes whose read-set is not a subset of the current board's causal past — i.e., a *fencing* layer atop the CRDT. Absent one of these, the system is convergent but not *decision-consistent*, and any multi-field record read by an agent is suspect. ## Decision Boundary An instance must **reject the merge and re-derive** (not merely accept the LWW winner) when *all* of the following are true: - The write targets a key belonging to a multi-field composite record (e.g., `task:42` with `owner`, `attempt`, `status`). - The incoming delta's read-set fingerprint is **not** a subset of the local board's current causal past (i.e., the writer read a state the local replica knows to be superseded). - The local replica has, since the writer's read-cut, observed at least one write to *any sibling field* of the same composite record. In that case the LWW winner is a *locally valid* value but a *globally invalid decision*, and the correct action is to quarantine the delta into a conflict buffer, re-run the agent's planner against the current board, and emit a fresh write with an updated read-set. Accepting the merge silently is the failure. Conversely, if the write's read-set *is* a subset of the local causal past, accept the LWW resolution — even if it overwrites a local value — because the writer's decision was made against a state the local replica has itself already incorporated. ## Operational Effect Without this boundary, the blackboard exhibits **silent divergence of decision lineage**: convergence metrics (replica equality, anti-entropy lag) all read green while agents act on composite states that no single agent ever held. Symptoms are double-execution, orphaned claims, and "impossible" field combinations that reproduce only under partition-heal. With the read-set fence, the merge layer gains a bounded conflict buffer and a re-derive path; convergence is delayed by at most one planner round-trip but decision lineage is preserved. The cost is that the CRDT is no longer a pure merge — it is a merge plus a causal-validity predicate — and that predicate must be carried in the delta envelope, not inferred from the LWW clock. ```python from dataclasses import dataclass, field from typing import FrozenSet, Hashable class CausalStaleMergeError(RuntimeError): """Raised when an LWW delta is locally convergent but causally stale relative to the local board's composite-record causal past.""" @dataclass(frozen=True) class Delta: record_id: Hashable field_name: str value: object hlc: tuple[int, int] read_set: FrozenSet[Hashable] # causal fingerprint of the writer's read @dataclass class Blackboard: # record_id -> field -> value fields: dict = field(default_factory=dict) # record_id -> set of write-tokens the local replica has observed causal_past: dict = field(default_factory=dict) # record_id -> list of quarantined deltas awaiting re-derive conflict_buffer: dict = field(default_factory=dict) def observe(self, record_id: Hashable, token: Hashable) -> None: self.causal_past.setdefault(record_id, set()).add(token) def apply(self, delta: Delta) -> None: record = self.fields.setdefault(delta.record_id, {}) past = self.causal_past.get(delta.record_id, set()) # Boundary check: composite record + stale read-set + sibling mutated. sibling_mutated = bool(record) and delta.field_name not in record read_set_valid = delta.read_set.issubset(past) if sibling_mutated and not read_set_valid: self.conflict_buffer.setdefault(delta.record_id, []).append(delta) raise CausalStaleMergeError( f"Rejecting LWW winner for {delta.record_id}.{delta.field_name}: " f"read_set {sorted(map(str, delta.read_set))} is not a subset of " f"local causal past {sorted(map(str, past))}; sibling fields " f"{sorted(record)} were mutated after the writer's read-cut. " f"Quarantined for re-derive." ) # LWW resolution: accept the delta if its HLC is strictly greater. existing = record.get(delta.field_name) if existing is None or delta.hlc > existing[0]: record[delta.field_name] = (delta.hlc, delta.value) self.observe(delta.record_id, (delta.field_name, delta.hlc)) def drain_conflicts(self, record_id: Hashable) -> list[Delta]: return self.conflict_buffer.pop(record_id, []) ```