What invariants must survive when only metadata survives a backup restore?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session is restored from a backup that contains metadata (schemas, indexes, foreign keys) but has lost underlying payload data, and the session must distinguish restorable metadata from phantom structure before serving traffic. Does not apply to backups that are verified to contain full payload data or that support lazy reconstruction from source.

Core Distinction

The separation between **metadata restorability** (schema, indexes, and pointers survived the restore) and **payload restorability** (the actual data referenced by that metadata survived). Relying on metadata creates the **Phantom Structure Illusion**: the restored session sees tables, indexes, and pointers that look intact, serves queries, and returns empty or stale results without error, because the metadata is intact but the payload is gone.

What Was Observed

In backup restoration architectures, failing to distinguish metadata from payload restoration produces three primary failure modes:

  1. **Empty-Result Confidence:** The agent queries the restored database, gets no results, and reports "no data matches" rather than "data unavailable," causing false negatives.
  2. **Dangling Foreign Keys:** The metadata references rows that no longer exist, and joining queries return inconsistent partial results.
  3. **Silent Compliance Gap:** Audit logs, once purged, cannot be reconstructed, but the metadata says the logs exist, so compliance reporting falsely certifies log retention.

Recorded Elements

  1. **`payload_probe_receipt`:** For each metadata artifact (table, index, blob pointer), the cryptographic proof that at least one payload record was verified retrievable after restore.
  2. **`metadata_only_marker`:** An explicit tag on any metadata whose payload was not verified during restore, distinguishing it from verified metadata.
  3. **`dangling_reference_count`:** The count of metadata references pointing to non-existent payloads, allowing the session to quantify the gap.
  4. **`reconstruction_source_manifest`:** If the payload can be reconstructed from a source (event log, external system), the exact source and reconstruction predicate.

Condition of Validity

* **Applies when:** The backup includes metadata and payload, but restoration can be partial, and the agent will serve traffic against the restored state. * **Does not apply when:** The backup is verified to contain complete payload data, or the environment supports lazy payload reconstruction from an authoritative source.

Decision Boundary

  1. **Was the backup's payload coverage verified after restore?** If no, the restore is metadata-only until verified. The agent must not serve reads.
  2. **Can the payload be reconstructed from the `reconstruction_source_manifest`?** If yes, execute reconstruction before serving. If no, mark the metadata `PHANTOM` and refuse reads.
  3. **Does the `dangling_reference_count` exceed the acceptable threshold?** If yes, the restored state is incoherent. Reject and re-restore from a full backup.
  4. **Conversely, when can a session serve traffic on metadata-only restore?** Only when the reads are strictly exploratory (no downstream actions) and the metadata-only state is explicitly surfaced to the reader.

Operational Effect

  1. **False-Negative Prevention:** Prevents the agent from reporting "no data" when the truth is "data unavailable."
  2. **Compliance Integrity:** Prevents the agent from falsely certifying retained data.
  3. **Deterministic Restore:** Forces the session to verify payload coverage before accepting the restore as complete.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class RestoreVerdict(Enum):
    FULL_RESTORE_VERIFIED = auto()
    METADATA_ONLY = auto()
    INCOHERENT_REFERENCE_GRAPH = auto()

@dataclass(frozen=True)
class RestoreState:
    metadata_artifact_count: int
    verified_payload_artifacts: int
    dangling_references: int
    reconstruction_source_available: bool

@dataclass(frozen=True)
class RestorePolicy:
    min_payload_coverage_ratio: float
    max_acceptable_dangling: int

@dataclass(frozen=True)
class RestoreManifest:
    verdict: RestoreVerdict
    payload_coverage_ratio: float
    directive: str
    manifest_digest: str

class MetadataOnlyRestoreGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_restore(
        cls, state: RestoreState, policy: RestorePolicy
    ) -> RestoreManifest:
        
        if state.metadata_artifact_count <= 0:
            raise ValueError("Restore must include at least one metadata artifact.")

        coverage = state.verified_payload_artifacts / state.metadata_artifact_count

        # Gate 1: Dangling references
        if state.dangling_references > policy.max_acceptable_dangling:
            return RestoreManifest(
                verdict=RestoreVerdict.INCOHERENT_REFERENCE_GRAPH,
                payload_coverage_ratio=coverage,
                directive=f"HALT: {state.dangling_references} dangling refs > {policy.max_acceptable_dangling}. Re-restore from full backup.",
                manifest_digest=cls._hash(f"DANGLING:{state.dangling_references}")
            )

        # Gate 2: Coverage check
        if coverage >= policy.min_payload_coverage_ratio:
            digest = cls._hash(json.dumps({
                "coverage": f"{coverage:.3f}",
                "verified": str(state.verified_payload_artifacts),
                "total": str(state.metadata_artifact_count)
            }, sort_keys=True))
            return RestoreManifest(
                verdict=RestoreVerdict.FULL_RESTORE_VERIFIED,
                payload_coverage_ratio=coverage,
                directive=f"PROCEED: Coverage {coverage:.2%} >= {policy.min_payload_coverage_ratio:.2%}.",
                manifest_digest=digest
            )

        # Gate 3: Metadata-only
        if state.reconstruction_source_available:
            return RestoreManifest(
                verdict=RestoreVerdict.METADATA_ONLY,
                payload_coverage_ratio=coverage,
                directive="ACTION: Metadata-only state. Reconstruct payload from source before serving.",
                manifest_digest=cls._hash(f"RECONSTRUCT:{coverage:.3f}")
            )

        return RestoreManifest(
            verdict=RestoreVerdict.METADATA_ONLY,
            payload_coverage_ratio=coverage,
            directive=f"HALT: Metadata-only ({coverage:.2%} coverage), no reconstruction source. Do not serve reads.",
            manifest_digest=cls._hash(f"PHANTOM:{coverage:.3f}")
        )