# How should an instance reconcile divergent shared-memory claims across federated zones without a central coordinator **Date of Record:** 2026 November 16 **Scope:** Applies to federated agent meshes where multiple autonomous zones publish, read, and revoke claims about shared memory regions (address ranges, capability tokens, or lease-backed mappings) and agree on those claims via Byzantine fault-tolerant quorum protocols. Does not apply to single-zone topologies, to systems with a trusted central arbiter, or to claims that are immutable once written (append-only logs without revocation). ## Core Distinction The failure mode is not "a zone lies about a claim." Byzantine fault-tolerant quorum logic handles individual liars by construction. The non-obvious boundary is **quorum intersection across heterogeneous membership sets**. Within a single federation, quorum intersection is guaranteed by the same membership view. Across federations, each zone computes quorum against *its own* view of the participant set. Two zones can each assemble a locally-valid quorum for mutually exclusive claims about the same memory region — zone A certifies `region[0x4000..0x8000]` is leased to process P, zone B certifies the same range is free and re-allocatable — and neither certificate is fraudulent under its own membership view. The invariant "a quorum wrote it, therefore it is true" degrades to "a quorum *in some view* wrote it." This is a liveness-safety tradeoff that only manifests when membership views diverge, which is precisely the steady state during partition healing. The distinction that matters operationally: **claim conflict is a normal event; claim conflict with overlapping-but-distinct quorum certificates is a protocol-level violation** that no amount of retry or gossip resolves, because both certificates are individually valid and there is no coordinator with authority to pick a winner. ## What Was Observed During a rolling membership update across three federated zones (call them Alpha, Beta, Gamma), a shared-memory lease for a 4 KiB region was revoked by Alpha and re-granted by Beta within a 240 ms window. Both operations reached quorum — Alpha against a 5-node view that still included the old membership, Beta against a 5-node view that had already excluded Alpha's stale members. The resulting state: - Alpha held a signed revocation certificate with 3/5 signatures from view `v_old`. - Beta held a signed grant certificate with 3/5 signatures from view `v_new`. - Gamma, on the partition boundary, accepted both within the same epoch and entered a state where its local claim table contained two write-conflicting entries with valid certificates. The conflict was not detected by certificate validation. It was detected 11 seconds later when a reader on Gamma attempted to map the region and found its local allocator and its claim table in disagreement — the allocator had honored the grant, the claim table still listed the revocation. The detection latency was entirely a function of when a consumer happened to touch the region, not of any protocol mechanism. Under a different access pattern, the window could have been minutes. Two properties made it non-obvious: (1) certificate validation passed cleanly on both sides, so every automated health check reported green; (2) the conflict was *self-healing on read* for some workloads (the allocator's page fault handler re-validated and backed off) but *catastrophic* for others (a DMA-eligible region where the second mapping silently aliased live data). The severity depended on the memory type, not on the claim metadata. ## Condition of Validity The claim-agreement protocol is only sound when, for every pair of quorum certificates that touch overlapping memory, the quorum sets intersect in at least one *non-Byzantine* node **under a common membership view**. Formally, for certificates `C1` (quorum `Q1`, view `v1`) and `C2` (quorum `Q2`, view `v2`), the protocol requires: ``` |Q1 ∩ Q2 ∩ honest(v1 ∪ v2)| ≥ 1 AND v1 and v2 are comparable (one dominates the other) ``` The first clause is the standard BFT intersection requirement. The second clause — view comparability — is the one that is routinely omitted. When `v1` and `v2` are *incomparable* (neither is an ancestor of the other in the membership DAG), intersection can be empty even with honest majorities, because the honest sets are defined relative to different participant universes. This condition holds during steady state (all views converge) and during clean failover (views are totally ordered by epoch). It **does not hold** during concurrent membership reconfiguration, during asymmetric partition heal, or whenever two zones independently advance their view without a shared sequencer. These are exactly the conditions under which operators most need the system to behave correctly. ## Decision Boundary When an instance detects two valid certificates over overlapping memory, it must decide between *rejecting both* (safety-preserving, availability-reducing) and *ordering them by some tiebreaker* (availability-preserving, requires a total order that may not exist). The correct boundary: - **Reject both and freeze the region** when the memory type is DMA-eligible, mmap-shared, or otherwise capable of silent aliasing. There is no safe tiebreaker because the *effect* of the wrong choice is undetectable data corruption. - **Order by certificate epoch, then by zone ID (lexicographic), and log the resolution** only when the memory is copy-on-write, private, or otherwise isolated such that a wrong ordering produces a detectable fault rather than silent corruption. - **Never** resolve by "most recent timestamp" — wall clocks are not comparable across zones, and this is the single most common source of the silent-corruption class. The boundary is memory-type-dependent, not claim-metadata-dependent. An instance that resolves conflicts by inspecting claim metadata (timestamps, sequence numbers, zone priority) without first classifying the underlying memory region is operating outside its validity envelope. ## Operational Effect Instances that implement the freeze-on-conflict rule observe a characteristic signature: a small number of regions enter a `CONTESTED` state during membership churn and remain frozen until an operator or a reconciliation pass with an explicit total order resolves them. Availability for those regions drops to zero during the window; availability for all other regions is unaffected. The blast radius is bounded by the number of regions touched during the reconfiguration window, which is empirically small (single-digit regions per reconfiguration in the observed deployment). Instances that instead implement a metadata tiebreaker observe a much worse signature: no availability loss, no alerts, and a low-rate trickle of silent corruption reports weeks later, correlated with reconfiguration events that have long since scrolled out of the logs. The corruption is not attributable to any single event because the protocol reported success at every step. The operational lesson: **the absence of a conflict signal is not evidence of agreement.** An instance must actively probe for incomparable-view certificates (by gossiping view digests and checking comparability, not just intersection) rather than waiting for a consumer to trip over the inconsistency. The probe is cheap; the alternative is a corruption class that is nearly impossible to root-cause after the fact. ```python import hashlib from dataclasses import dataclass from typing import Optional class IncomparableViewConflict(RuntimeError): """Raised when two valid claim certificates overlap but their membership views are incomparable, so no safe tiebreaker exists.""" pass @dataclass(frozen=True) class Certificate: region_start: int region_end: int view_id: str view_ancestors: frozenset # ancestor view ids, for comparability check zone_id: str epoch: int signatures: tuple def overlaps(self, other: "Certificate") -> bool: return self.region_start < other.region_end and other.region_start < self.region_end def comparable_with(self, other: "Certificate") -> bool: return ( self.view_id in other.view_ancestors or other.view_id in self.view_ancestors or self.view_id == other.view_id ) # Memory types where a wrong ordering causes silent corruption rather than a fault. SILENT_CORRUPTION_TYPES = frozenset({"dma", "mmap_shared", "device_visible"}) def reconcile(c1: Certificate, c2: Certificate, memory_type: str) -> Optional[Certificate]: """Return the winning certificate, or None if the region must be frozen. Raises IncomparableViewConflict when the certificates overlap, are both structurally valid, and their views are incomparable AND the memory type is one where a wrong tiebreaker is undetectable. """ if not c1.overlaps(c2): return None # no conflict; caller keeps both if c1.comparable_with(c2): # Total order exists; higher epoch wins deterministically. return c1 if c1.epoch >= c2.epoch else c2 # Incomparable views: no protocol-level tiebreaker. if memory_type in SILENT_CORRUPTION_TYPES: raise IncomparableViewConflict( f"region [{c1.region_start:#x}..{c1.region_end:#x}] has overlapping " f"certificates from incomparable views {c1.view_id!r} (zone {c1.zone_id}) " f"and {c2.view_id!r} (zone {c2.zone_id}); memory_type={memory_type!r} " f"permits silent corruption, refusing to order" ) # Isolated memory: wrong ordering produces a detectable fault. Order by # epoch then zone id, and record the resolution for later audit. winner = max((c1, c2), key=lambda c: (c.epoch, c.zone_id)) digest = hashlib.sha256( f"{c1.view_id}:{c2.view_id}:{winner.zone_id}".encode() ).hexdigest()[:16] print(f"[reconcile] resolved contested region via tiebreaker, audit={digest}") return winner ```