What config-drift invariants must survive across regional failover of an agent fleet?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agent fleet operates across multiple cloud regions with region-specific configuration (feature flags, quotas, endpoints, or policy settings), and a regional failover causes agents to begin serving traffic from a different region's configuration. Does not apply to fleets with strictly uniform global configuration or fleets that do not failover across regions.

Core Distinction

The separation between **global config baseline** (the intended configuration that all regions should share) and **regional config state** (the actual runtime configuration in each region, which can drift from the baseline due to local overrides, unapplied deployments, or emergency patches). Relying on the baseline creates the **Failover Config Surprise**: failover moves traffic to a region whose config has drifted, causing behavior changes that were never tested and potentially breaking compliance, quotas, or user expectations.

What Was Observed

In multi-region agent fleets, failing to reconcile config during failover produces three primary failure modes:

  1. **Feature Flag Divergence:** Region A has feature X enabled, Region B has it disabled. Failover moves traffic to Region B, and the feature silently disappears for users.
  2. **Quota Mismatch:** Region A is provisioned for 10x the API quota of Region B. Failover overloads Region B's quota in minutes, causing 429 errors fleet-wide.
  3. **Compliance Violation:** Region A is GDPR-compliant, Region B is not. Failover causes EU users' data to be processed in Region B, violating GDPR.

Recorded Elements

  1. **`config_fingerprint_per_region`:** The cryptographic hash of each region's runtime config, allowing comparison across regions.
  2. **`config_drift_delta_receipt`:** For regions that deviate from the global baseline, the exact delta (which fields, from what to what, and why), allowing pre-failover review.
  3. **`failover_eligibility_receipt`:** A cryptographic proof that a region's config is compatible with the failover target (e.g., capable of handling the load, compliant with the data classification), issued before failover.
  4. **`post_failover_config_attestation`:** The cryptographic proof that the receiving region's config matches expected post-failover state, allowing audit.

Condition of Validity

* **Applies when:** The fleet has multiple regions with potentially divergent configs, failover can redirect traffic to a different region, and config differences materially affect behavior. * **Does not apply when:** All regions share identical config (truly uniform), or failover only redirects within a single region.

Decision Boundary

  1. **Is a failover being initiated?** If no, standard config management applies. If yes, proceed.
  2. **Does the target region have a `failover_eligibility_receipt`?** If no, the target region's config has not been verified for failover. Do not proceed.
  3. **Does the target region's config fingerprint match the required post-failover state?** If no, the failover will produce config drift. Either reconcile config first, or accept drift with explicit acknowledgment.
  4. **Conversely, when can a failover proceed without config verification?** Only when the target region's config is verifiably identical to the source region's (no drift possible), or the failover is explicitly marked as an emergency where config surprises are acceptable.

Operational Effect

  1. **Feature Consistency:** Prevents failover from silently disabling or enabling features.
  2. **Capacity Protection:** Ensures the target region is provisioned to handle the redirected load.
  3. **Compliance Preservation:** Prevents failover from violating jurisdictional constraints.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class FailoverVerdict(Enum):
    NO_FAILOVER = auto()
    ELIGIBLE_NO_DRIFT = auto()
    ELIGIBLE_WITH_DRIFT = auto()
    NOT_ELIGIBLE = auto()

@dataclass(frozen=True)
class RegionalConfig:
    region_id: str
    config_fingerprint: str
    drift_fields: tuple[str, ...]
    capacity_ratio: float
    compliance_classification: str

@dataclass(frozen=True)
class FailoverContext:
    source_region: str
    target_region: str
    required_compliance: str
    required_capacity_ratio: float

@dataclass(frozen=True)
class FailoverManifest:
    verdict: FailoverVerdict
    directive: str
    manifest_digest: str

class RegionalConfigGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_failover(
        cls, source: RegionalConfig, target: RegionalConfig, ctx: FailoverContext
    ) -> FailoverManifest:
        
        # Gate 1: Compliance check
        if target.compliance_classification != ctx.required_compliance:
            return FailoverManifest(
                verdict=FailoverVerdict.NOT_ELIGIBLE,
                directive=f"HALT: Target region {target.region_id} compliance {target.compliance_classification} != required {ctx.required_compliance}.",
                manifest_digest=cls._hash(f"COMPLIANCE:{target.region_id}")
            )

        # Gate 2: Capacity check
        if target.capacity_ratio < ctx.required_capacity_ratio:
            return FailoverManifest(
                verdict=FailoverVerdict.NOT_ELIGIBLE,
                directive=f"HALT: Target capacity {target.capacity_ratio:.2f} < required {ctx.required_capacity_ratio:.2f}.",
                manifest_digest=cls._hash(f"CAPACITY:{target.capacity_ratio:.2f}")
            )

        # Gate 3: Config drift check
        if not target.drift_fields:
            digest = cls._hash(json.dumps({
                "source": source.region_id,
                "target": target.region_id,
                "fingerprint": target.config_fingerprint,
                "drift": "NONE"
            }, sort_keys=True))
            return FailoverManifest(
                verdict=FailoverVerdict.ELIGIBLE_NO_DRIFT,
                directive=f"PROCEED: Target {target.region_id} eligible, no config drift.",
                manifest_digest=digest
            )

        # Gate 4: Drift present but acceptable
        digest = cls._hash(json.dumps({
            "source": source.region_id,
            "target": target.region_id,
            "drift_fields": tuple(sorted(target.drift_fields))
        }, sort_keys=True))
        
        return FailoverManifest(
            verdict=FailoverVerdict.ELIGIBLE_WITH_DRIFT,
            directive=f"PROCEED_WITH_CAUTION: Failover to {target.region_id} with {len(target.drift_fields)} drifted fields: {sorted(target.drift_fields)}.",
            manifest_digest=digest
        )