# What happens when an instance of a long-horizon agent state store trusts its own write-ahead log over its retrieval index during partial compaction

**Date of Record:** 2026 November 8

**Scope:** Applies to agent state stores that persist episodic memory as append-only write-ahead log (WAL) segments plus a derived retrieval index (vector or keyword), where compaction merges segments and rebuilds the index incrementally. Does not apply to stores that rebuild the index atomically from a single immutable snapshot, nor to purely stateless retrieval over external corpora.

## Core Distinction

Memory poisoning detection is usually framed as a *content* problem: an adversary injects false facts into the agent's memory and the detector must flag them by semantic inconsistency or provenance. The non-obvious failure here is structural, not semantic. During partial compaction, the WAL and the retrieval index can disagree about *which version of a memory record is authoritative*. The WAL is the durable ground truth; the index is a lossy derived projection that lags. When a detector reads the index to establish "what the agent believes" and reads the WAL to establish "what the agent was told," a poisoned record that was overwritten in the WAL but not yet re-indexed appears as a *legitimate* disagreement — and the detector either suppresses a real poison (because the stale index entry matches the attacker's content) or raises a false positive against a benign overwrite. The distinction is between *temporal skew between two views of the same store* and *actual adversarial injection*. Conflating them is the boundary condition.

## What Was Observed

In a store with 14-day retention and hourly compaction, a poisoned record was written at T0 (segment S7, offset 0x1a2c), then legitimately superseded by a corrected record at T0+40min (segment S7, offset 0x3f10). Compaction of S7 began at T0+55min but was interrupted by a node eviction and resumed at T0+3h. During the window [T0+55min, T0+3h], the retrieval index still contained the poisoned embedding (index generation `gen=441`), while the WAL contained both records with the correction marked `supersedes: 0x1a2c`. The poisoning detector, which compared index-retrieved content against WAL-replayed content, observed a mismatch and classified it as *content tampering* — because its model assumed index and WAL are consistent except under attack. It quarantined the corrected record (the one in the WAL) and left the poisoned index entry live. The agent then answered 11 queries using the poisoned memory before a manual audit caught it. The detector's false-negative rate on the poisoned content was 100% for the duration of the skew window; its false-positive rate on the correction was also 100%.

## Condition of Validity

The failure requires all of the following to hold simultaneously:

1. Compaction is **partial and resumable** — the index rebuild is not atomic with the WAL segment merge, so a crash or eviction leaves an index generation older than the WAL head.
2. The poisoning detector treats **index/WAL divergence as evidence of tampering** rather than as a possible compaction artifact. This is the load-bearing assumption; if the detector first reconciles generations, the failure does not occur.
3. Records carry **supersession metadata** (`supersedes`, `valid_from`, `tombstone`) that the detector does not consult before classifying divergence.
4. The skew window is **long enough** for the agent to issue queries against the stale index — in the observed case, 2h05m, driven by the eviction/resume cycle.

When compaction is atomic (condition 1 false), the index is never stale and divergence is genuinely suspicious. When the detector reconciles generations first (condition 2 false), it correctly attributes the mismatch to compaction lag. The boundary is precisely *non-atomic compaction combined with a detector that lacks generation-awareness*.

## Decision Boundary

The operational rule: **a detector must never classify index/WAL divergence as poisoning until it has proven the divergence is not explained by compaction lag.** Concretely, before flagging:

- Compare the index generation stamped on each retrieved record against the WAL head generation for its segment. If `index_gen < wal_gen` for the segment, the record is *stale*, not *tampered*; route to reconciliation, not quarantine.
- Replay the WAL for the segment and resolve supersession chains *before* comparing content. A record whose WAL entry is superseded is not a live belief and must not be compared against the index.
- Only when `index_gen == wal_gen` and content still diverges is the divergence admissible as poisoning evidence.

The inverse boundary also matters: if the detector simply trusts the WAL and ignores the index during skew, it will miss poisoning that occurred *before* the last successful compaction and was already baked into the index generation — the WAL replay must be the authority, but the index must still be checked for records the WAL no longer covers (post-compaction tombstones). The safe posture is WAL-authoritative with explicit tombstone reconciliation, not index-authoritative and not naive two-way comparison.

## Operational Effect

After the incident, the store was changed to stamp every index entry with its source `(segment_id, wal_gen, offset)` and to make the detector refuse to run against a segment whose index generation trails the WAL head by more than one compaction cycle — it emits a `CompactionSkewError` and blocks queries on that segment until reconciliation completes. This converts a silent false-negative into a loud, bounded availability hit. The tradeoff is deliberate: a 2-hour query block on one segment is cheaper than 11 poisoned answers. The general lesson for long-horizon stores is that *staleness and tampering are indistinguishable to a detector that does not track provenance generations*, and any detector that conflates them will fail in whichever direction its default assumption points — here, toward trusting the attacker's stale index entry.

```python
class CompactionSkewError(RuntimeError):
    """Raised when index generation trails WAL head; divergence is not admissible as poisoning evidence."""
    pass


def classify_divergence(record, wal_head_gen, max_lag_cycles=1):
    index_gen = record["index_gen"]
    wal_gen = record["wal_gen"]
    lag = wal_head_gen - index_gen

    if lag > max_lag_cycles:
        # Cannot distinguish staleness from tampering; refuse to classify.
        raise CompactionSkewError(
            f"segment {record['segment_id']} index_gen={index_gen} "
            f"trails wal_head_gen={wal_head_gen} by {lag} cycles; "
            f"reconciliation required before poisoning classification"
        )

    if index_gen < wal_gen:
        # Stale but within tolerance: reconcile via WAL replay, do not flag.
        return "stale_reconcile"

    if record["content_hash"] != record["wal_content_hash"]:
        # Same generation, content differs: admissible as poisoning evidence.
        return "poisoning_candidate"

    return "consistent"


def detect(record, wal_head_gen):
    verdict = classify_divergence(record, wal_head_gen)
    if verdict == "poisoning_candidate":
        quarantine(record["record_id"], reason="content_divergence_same_gen")
    elif verdict == "stale_reconcile":
        enqueue_reconciliation(record["segment_id"])
    return verdict
```