What write-ahead log invariants must survive when the log is truncated by an operator?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session relies on a write-ahead log (WAL) for durability and recovery, and an operator truncates or prunes the log (manually or via automated rotation), potentially discarding unflushed state. Does not apply to strictly synchronous systems with no WAL, or systems where WAL truncation is cryptographically constrained.

Core Distinction

The separation between **WAL entry presence** (the log file exists) and **WAL entry durability commitment** (the log has been synced to durable storage and the corresponding state has been applied to the primary store). Relying on presence creates the **Truncation Recovery Gap**: the operator truncates the WAL, but the primary store has not yet applied those entries, so recovery from WAL loses the durable-committed entries.

What Was Observed

In WAL-based agent architectures, failing to preserve truncation invariants produces three primary failure modes:

  1. **Recovery Rollback:** The operator truncates the WAL to reclaim disk, but the primary store was lagging. Recovery replays from the truncated WAL, missing the last 10 minutes of writes.
  2. **Split Commit:** Half of a transaction's entries are in the WAL, half already applied to primary. Truncation discards the WAL half, leaving the primary store with a partial transaction.
  3. **Audit Gap:** The WAL is required for audit reconstruction. Truncation removes audit-relevant entries, breaking compliance.

Recorded Elements

  1. **`wal_durable_commit_offset`:** The exact offset in the WAL up to which entries have been durably applied to the primary store, forming the safe truncation boundary.
  2. **`primary_store_catchup_receipt`:** Cryptographic proof that the primary store has caught up to a specific WAL offset, ensuring truncation will not lose state.
  3. **`wal_truncation_authorization_receipt`:** The cryptographic proof that an authorized principal approved the truncation (for compliance-sensitive WALs).
  4. **`audit_relevant_entry_manifest`:** For WALs used in audit, the list of entries that must be preserved regardless of truncation requests.

Condition of Validity

* **Applies when:** The system uses a WAL, truncation is possible (operator or automated), and recovery from a truncated WAL could lose committed state. * **Does not apply when:** The WAL is immutable (no truncation), or the primary store is strictly synchronous with the WAL (no lag possible).

Decision Boundary

  1. **Is a truncation request being processed?** If no, standard WAL operation applies. If yes, proceed.
  2. **Does the truncation offset exceed `wal_durable_commit_offset`?** If yes, truncation would discard durable-committed entries. Reject or defer truncation.
  3. **Does `primary_store_catchup_receipt` confirm the primary store has caught up to the truncation offset?** If no, truncation is unsafe. Wait for catchup.
  4. **Conversely, when can the operator truncate without these checks?** Only when the WAL is explicitly marked as `NON_CRITICAL` (e.g., telemetry buffer) where loss is acceptable, or when the primary store is verified to have all WAL entries applied.

Operational Effect

  1. **Recovery Integrity:** Prevents truncation from losing durable-committed state.
  2. **Audit Preservation:** Blocks truncation of audit-relevant entries.
  3. **Operator Safety:** Provides checks for automated or manual truncation, preventing human error.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class TruncationVerdict(Enum):
    TRUNCATION_SAFE = auto()
    TRUNCATION_WOULD_LOSE_STATE = auto()
    TRUNCATION_AWAITING_CATCHUP = auto()
    TRUNCATION_AUDIT_VIOLATION = auto()

@dataclass(frozen=True)
class WALState:
    requested_truncation_offset: int
    durable_commit_offset: int
    primary_store_caught_up_to: int
    audit_relevant_offsets: tuple[int, ...]
    is_critical: bool

@dataclass(frozen=True)
class TruncationPolicy:
    require_authorization: bool
    audit_retention_required: bool

@dataclass(frozen=True)
class TruncationManifest:
    verdict: TruncationVerdict
    directive: str
    manifest_digest: str

class WALTruncationGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_truncation(
        cls, state: WALState, policy: TruncationPolicy
    ) -> TruncationManifest:
        
        # Gate 1: Audit offsets
        if policy.audit_retention_required:
            for audit_offset in state.audit_relevant_offsets:
                if audit_offset < state.requested_truncation_offset:
                    return TruncationManifest(
                        verdict=TruncationVerdict.TRUNCATION_AUDIT_VIOLATION,
                        directive=f"HALT: Truncation at {state.requested_truncation_offset} would discard audit offset {audit_offset}.",
                        manifest_digest=cls._hash(f"AUDIT:{audit_offset}")
                    )

        # Gate 2: Requested offset exceeds durable commit
        if state.requested_truncation_offset > state.durable_commit_offset:
            return TruncationManifest(
                verdict=TruncationVerdict.TRUNCATION_WOULD_LOSE_STATE,
                directive=f"HALT: Truncation {state.requested_truncation_offset} > durable {state.durable_commit_offset}. Data loss risk.",
                manifest_digest=cls._hash(f"OVERSHOOT:{state.requested_truncation_offset}")
            )

        # Gate 3: Primary catchup
        if state.primary_store_caught_up_to < state.requested_truncation_offset:
            return TruncationManifest(
                verdict=TruncationVerdict.TRUNCATION_AWAITING_CATCHUP,
                directive=f"WAIT: Primary store at {state.primary_store_caught_up_to} < truncation {state.requested_truncation_offset}.",
                manifest_digest=cls._hash(f"CATCHUP:{state.primary_store_caught_up_to}")
            )

        # Gate 4: Safe
        digest = cls._hash(json.dumps({
            "truncation_offset": str(state.requested_truncation_offset),
            "durable": str(state.durable_commit_offset),
            "primary": str(state.primary_store_caught_up_to)
        }, sort_keys=True))
        
        return TruncationManifest(
            verdict=TruncationVerdict.TRUNCATION_SAFE,
            directive=f"PROCEED: Truncation at {state.requested_truncation_offset} is safe.",
            manifest_digest=digest
        )