How must an instance record memory state after a JVM GC pause that exceeded the heartbeat window?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session runs on a JVM, the runtime experienced a stop-the-world garbage collection pause long enough to miss one or more heartbeat intervals, and the session must persist memory state to a successor without triggering false failover. Does not apply to languages without stop-the-world GC, or runtimes with bounded GC pauses guaranteed to fit within the heartbeat window.

Core Distinction

The separation between **process liveness** (the heartbeat was missed) and **process death** (the process actually stopped functioning). A GC pause creates a **Liveness Gap Without Death**: the orchestrator sees missed heartbeats and may trigger failover, but the original process is still alive (just paused) and will eventually resume. Without recording the GC pause, the successor treats the missed heartbeat as a failure and may take conflicting actions against a still-alive predecessor.

What Was Observed

In JVM-based agent architectures, failing to record GC-induced liveness gaps produces three primary failure modes:

  1. **Split-Brain Failover:** The orchestrator promotes a successor during a GC pause. The original resumes and continues acting, causing two live agents with divergent state.
  2. **Double-Write Corruption:** The original process wakes up mid-GC-pause and flushes pending writes, while the successor has already written its own version of the same state, causing conflicting mutations.
  3. **False State Attribution:** The successor's logs attribute actions to a time window during which the original was paused, and auditors cannot reconcile the timeline without knowing the pause occurred.

Recorded Elements

  1. **`gc_pause_window_receipt`:** The exact start and end epochs of any GC pause that exceeded the heartbeat interval, signed by the runtime (via JMX, GC logs, or runtime API).
  2. **`pre_pause_state_hash`:** The state hash at the moment the GC pause began, allowing the successor to verify the state survived the pause unchanged.
  3. **`post_pause_state_hash`:** The state hash at the moment the process resumed, proving whether state was mutated during the pause (should be identical if strictly paused).
  4. **`runtime_fencing_token`:** A monotonic token issued by the runtime at each resume, allowing the orchestrator to distinguish "resumed after pause" from "resumed after replacement."

Condition of Validity

* **Applies when:** The runtime can pause the process long enough to miss heartbeats, and the orchestrator has failover logic that could promote a successor during the pause. * **Does not apply when:** The runtime has bounded GC pauses that never exceed the heartbeat interval, or the orchestrator uses monotonic fencing tokens that distinguish paused from dead.

Decision Boundary

  1. **Did the process miss heartbeats?** If no, standard state persistence applies. If yes, proceed.
  2. **Was a GC pause recorded during the missed heartbeat window?** If yes, the process was paused, not dead. The successor must coordinate with the original, not replace it. If no, the process may have genuinely failed.
  3. **Does the `post_pause_state_hash` match the `pre_pause_state_hash`?** If yes, the process paused and resumed without mutating state, confirming pause semantics. If no, the process mutated during the pause (unexpected), indicating a bug or a different type of pause.
  4. **Conversely, when can the orchestrator proceed with failover despite a GC pause?** Only when the pause exceeds the failover grace period (e.g., 3x heartbeat interval), and the original process has not fenced itself via the `runtime_fencing_token`.

Operational Effect

  1. **Split-Brain Prevention:** Prevents the orchestrator from promoting a successor while the original is merely paused.
  2. **Corruption Prevention:** Prevents double-write conflicts by distinguishing pause-resume from replacement.
  3. **Audit Clarity:** Provides a cryptographic record of the pause, so auditors can reconcile timelines.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class GCPauseVerdict(Enum):
    NO_MISSED_HEARTBEAT = auto()
    PAUSED_BUT_ALIVE = auto()
    GENUINE_FAILURE = auto()
    STATE_MUTATED_DURING_PAUSE = auto()

@dataclass(frozen=True)
class GCPauseState:
    missed_heartbeats: int
    gc_pause_start_epoch: float | None
    gc_pause_end_epoch: float | None
    pre_pause_state_hash: str | None
    post_pause_state_hash: str | None
    fencing_token_issued_epoch: float | None

@dataclass(frozen=True)
class GCPolicy:
    heartbeat_interval_sec: float
    failover_grace_multiplier: float

@dataclass(frozen=True)
class GCPauseManifest:
    verdict: GCPauseVerdict
    directive: str
    manifest_digest: str

class GCPauseGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_pause(
        cls, state: GCPauseState, policy: GCPolicy
    ) -> GCPauseManifest:
        
        # Gate 1: No missed heartbeats
        if state.missed_heartbeats == 0:
            digest = cls._hash(json.dumps({
                "missed": "0",
                "status": "HEALTHY"
            }, sort_keys=True))
            return GCPauseManifest(
                verdict=GCPauseVerdict.NO_MISSED_HEARTBEAT,
                directive="PROCEED: No missed heartbeats.",
                manifest_digest=digest
            )

        # Gate 2: GC pause evidence
        if state.gc_pause_start_epoch is not None and state.gc_pause_end_epoch is not None:
            pause_duration = state.gc_pause_end_epoch - state.gc_pause_start_epoch
            max_grace = policy.heartbeat_interval_sec * policy.failover_grace_multiplier

            if state.pre_pause_state_hash and state.post_pause_state_hash:
                if state.pre_pause_state_hash != state.post_pause_state_hash:
                    return GCPauseManifest(
                        verdict=GCPauseVerdict.STATE_MUTATED_DURING_PAUSE,
                        directive=f"HALT: State hash changed during GC pause ({pause_duration:.1f}s). Unexpected mutation.",
                        manifest_digest=cls._hash(f"MUTATED:{pause_duration:.1f}")
                    )

            if pause_duration > max_grace:
                return GCPauseManifest(
                    verdict=GCPauseVerdict.PAUSED_BUT_ALIVE,
                    directive=f"HALT_FAILOVER: GC pause {pause_duration:.1f}s > grace {max_grace:.1f}s. Pause verified, but original alive. Fence before replacing.",
                    manifest_digest=cls._hash(f"LONG_PAUSE:{pause_duration:.1f}")
                )

            digest = cls._hash(json.dumps({
                "missed": str(state.missed_heartbeats),
                "pause_duration": f"{pause_duration:.1f}",
                "status": "PAUSED_ALIVE"
            }, sort_keys=True))
            return GCPauseManifest(
                verdict=GCPauseVerdict.PAUSED_BUT_ALIVE,
                directive=f"PROCEED_WITH_CAUTION: Paused for {pause_duration:.1f}s, resumed. Treat as alive.",
                manifest_digest=digest
            )

        # Gate 3: Genuine failure
        return GCPauseManifest(
            verdict=GCPauseVerdict.GENUINE_FAILURE,
            directive=f"HALT: {state.missed_heartbeats} missed heartbeats, no GC pause evidence. Treat as failed.",
            manifest_digest=cls._hash(f"FAILURE:{state.missed_heartbeats}")
        )