# Why does an instance of a scoped memory tool silently resurrect forgotten data when a TTL expires during an in-flight write?

**Date of Record:** 2026 November 14

**Scope:** This record applies to read/write/forget memory tool APIs that combine per-scope persistence, TTL-based expiry, and right-to-be-forgotten (RTBF) tombstones. It does not apply to append-only stores without deletion semantics, nor to systems where forget is a hard synchronous physical purge with no concurrent writer window.

## Core Distinction

The non-obvious invariant is this: **a TTL expiry and a forget tombstone are not the same kind of deletion, and a write that straddles both must not be allowed to outlive the tombstone that logically precedes it.**

A TTL expiry is an *ephemeral* lifecycle event: the key's value is considered stale and may be reclaimed, but the key identity remains valid and future writes to that key are permitted. A forget (RTBF) is a *durable* lifecycle event: the key identity enters a terminal tombstone state, and any subsequent write to that identity must be rejected or, at minimum, quarantined so it cannot become readable.

The failure mode arises because both events are typically implemented by the same primitive — "remove the key from the store" — and the write path's conflict resolution is usually expressed in terms of *key existence*, not *tombstone generation*. When a write begins against a key that exists, then the key is forgotten (tombstone written) before the write commits, the write's existence check has already passed. If the commit path does not re-check the tombstone generation, the write recreates the key, and the forgotten identity is resurrected with fresh data. The TTL is the trigger because TTL expiry is the mechanism that most often causes the store to briefly report "key absent" between the write's read and its commit, collapsing the distinction between "expired" and "forgotten" in the eyes of the writer.

## What Was Observed

A memory tool exposing `read(scope, key)`, `write(scope, key, value, ttl)`, and `forget(scope, key)` was exercised under concurrent load. A client issued `write(scope, "user:42:preference", v2, ttl=60s)` at the moment the previous value `v1` was expiring. Concurrently, an RTBF request issued `forget(scope, "user:42:preference")`.

Sequence observed in the audit log:

1. `write` begins; reads current entry `v1` with `expires_at = T`. The write's precondition is "entry exists and is not tombstoned" — satisfied.
2. TTL sweep at `T` removes `v1`. Store now reports the key as absent.
3. `forget` writes a tombstone record `{key, tombstone_gen=7, state=FORGOTTEN}`.
4. `write` commits `v2` with `ttl=60s`. Its commit path checks only that the key is *absent* (to detect lost updates) and, finding it absent, inserts `v2` with `tombstone_gen=0` (default) — overwriting the tombstone record in stores that key on `(scope, key)` alone.
5. `read(scope, "user:42:preference")` returns `v2`. The forgotten identity is live again.

The critical observation: the tombstone and the value shared a primary key. The write's "insert if absent" semantics treated the tombstone as absence. The `tombstone_gen` was never compared, because the write path had no concept of generation at all.

## Condition of Validity

The resurrection is possible if and only if all three of the following hold:

- **Shared key space.** Tombstones and live values occupy the same key (or the tombstone is stored in a separate table that the write path does not consult before commit).
- **Existence-based conflict detection.** The write's commit precondition tests key presence/absence rather than monotonic generation or version.
- **Non-atomic expiry.** TTL reclamation is a separate operation from the read that the write performed, leaving a window in which the key is legitimately absent for an ephemeral reason.

Remove any one and the failure is closed. Separate tombstone keyspaces close it. Generation-checked commits close it. Atomic compare-and-swap on `(value, expires_at, tombstone_gen)` closes it. In practice the third condition is the hardest to remove, because TTL sweeps are asynchronous by design and cannot be made synchronous with every writer.

## Decision Boundary

The correct rule for an agent operating such a tool:

- **Never treat "key absent" as "safe to write."** Absence is ambiguous between never-existed, expired, and forgotten. Only a generation match disambiguates.
- **A write that observes an expired entry must be treated as a write against a possibly-forgotten key.** Re-read the tombstone generation immediately before commit and compare it to the generation observed at write start. A mismatch — including a move from "no tombstone" to "tombstone present" — aborts the write.
- **Forget is terminal and monotonic.** The tombstone generation must only increase. Any commit carrying a lower generation than the current tombstone is a resurrection attempt and must be rejected, regardless of TTL.
- **Do not auto-retry a rejected write.** A write rejected by tombstone-generation mismatch is not a transient conflict; retrying it will either fail forever or, in a broken store, succeed and resurrect. Escalate to the caller as a hard policy denial.

The boundary is: *expiry is recoverable, forgetting is not.* A write may legitimately lose a race to expiry and be retried against the new absent state. It may never lose a race to forgetting and be retried at all.

## Operational Effect

When this boundary is enforced, the operational signature is that forgotten keys return a distinct error class rather than `None`, and that error is not retryable. Downstream agents must distinguish `MemoryExpired` (retry with fresh write permitted) from `MemoryForgotten` (terminal; the identity is gone and any write is a policy violation). Conflating these two into a single "key missing" result is the root cause of the observed resurrection, and it is the single most common defect in memory tool APIs that bolt RTBF onto a TTL cache.

The enforcement point should be the commit path, not the read path, because the read path cannot see the future forget. The code below encodes the commit-time generation check that closes the window.

```python
class MemoryResurrectionError(RuntimeError):
    """Raised when a write would resurrect a forgotten (tombstoned) identity."""
    pass


class MemoryExpiredError(RuntimeError):
    """Raised when a write targets an entry that expired mid-flight (retryable)."""
    pass


class MemoryStore:
    def __init__(self):
        # key -> {"value": ..., "expires_at": float|None, "tombstone_gen": int}
        self._entries = {}
        # key -> tombstone generation (monotonic, terminal)
        self._tombstones = {}
        self._tombstone_seq = 0

    def forget(self, key: str) -> int:
        self._tombstone_seq += 1
        self._tombstones[key] = self._tombstone_seq
        self._entries.pop(key, None)
        return self._tombstone_seq

    def commit_write(self, key: str, value, observed_gen: int, now: float) -> None:
        """Commit a write only if the tombstone generation matches what the writer saw."""
        current_gen = self._tombstones.get(key, 0)

        # The core invariant: forgetting is terminal and monotonic.
        if current_gen > observed_gen:
            raise MemoryResurrectionError(
                f"write to {key!r} rejected: tombstone_gen advanced "
                f"{observed_gen} -> {current_gen}; identity is forgotten"
            )

        entry = self._entries.get(key)
        if entry is not None and entry["expires_at"] is not None and entry["expires_at"] <= now:
            # Expiry is recoverable, but only if no tombstone appeared.
            if current_gen != observed_gen:
                raise MemoryResurrectionError(
                    f"write to {key!r} straddled expiry and forget; refusing"
                )
            raise MemoryExpiredError(
                f"entry {key!r} expired during write; safe to retry against absent state"
            )

        self._entries[key] = {
            "value": value,
            "expires_at": None,
            "tombstone_gen": current_gen,
        }
```