# In what scenarios must an instance treat “forget” as a tombstone rather than a deletion: the read-after-forget resurrection invariant **Date of Record:** 2026 November 16 **Scope:** Applies to memory subsystems exposing read/write/forget primitives with scoped retention guarantees, where forget is expected to be idempotent and permanent within a scope. Does not apply to append-only logs (where forget is a compaction hint) or to systems where a forget call carries a synchronous, verifiable physical-erasure receipt. ## Core Distinction The failure surface is not “does forget delete the record.” It is **whether a subsequent read can observe the record again after forget returns success**. A memory tool that implements forget as a logical tombstone (a marker that suppresses reads) is correct only if every read path consults the tombstone. A memory tool that implements forget as a physical removal is correct only if no concurrent writer can re-insert the same key between the tombstone write and the read. The non-obvious boundary: **forget followed by a write of the same key, then a read, must not resurrect the pre-forget value, and must not silently lose the post-forget write.** These two requirements conflict when the write path is an upsert that treats “absent” as “safe to create” while the forget path is a delayed tombstone. The window between them is the resurrection interval. ## What Was Observed A scoped store with TTL-bucketed retention was used by an agent loop that: wrote a fact, read it back to confirm, then forgot it on task completion, then wrote a *different* fact under a reused key (keys were derived from task slot, not content). Observed sequence: 1. `write(k, v1)` → retained. 2. `read(k)` → `v1`. Confirmation passes. 3. `forget(k)` → returns `OK`, tombstone scheduled at bucket boundary `t+Δ`. 4. `write(k, v2)` → upsert sees no live record (tombstone not yet applied), inserts `v2` as a *new* generation. 5. Bucket boundary fires; tombstone applies to the *key*, not the generation. 6. `read(k)` → absent. The `v2` write is lost. 7. A retry path re-runs step 4, now inserting `v3`; the tombstone from step 3 is already consumed, so `v3` survives — but the agent's audit log shows `v2` was acknowledged as durable. The second observed variant: when forget was implemented as a *soft* delete (row retained with `deleted_at`), the read path joined on `deleted_at IS NULL`, but a compaction job that rebuilt the index from a replica that had not yet received the tombstone re-materialized `v1`. Read-after-forget returned `v1` with no error. This is resurrection: a successful forget that is observably not durable. ## Condition of Validity A forget is operationally valid only when all of the following hold: - **Generation fencing:** every write carries a monotonically increasing generation per key; tombstones are scoped to a generation, so a post-forget write is a strictly newer generation and cannot be suppressed by an older tombstone. - **Read-your-forget:** any read issued after forget returns must not observe the forgotten generation, from any replica, including index rebuilds and compaction outputs. - **Idempotent tombstone consumption:** re-applying a tombstone must not delete a generation newer than the one it targets. - **Scope closure:** retention scope boundaries (tenant, session, slot) must be part of the key or the tombstone predicate; a tombstone that matches on bare key across scopes will delete a sibling scope's live record. If the store cannot provide generation fencing, forget must be treated as *advisory* and the caller must not reuse keys. If the store cannot provide read-your-forget across replicas, the caller must not treat forget as a security or privacy boundary. ## Decision Boundary Choose tombstone semantics when: keys are reused, writes are concurrent with forgets, or the store has asynchronous compaction. Choose physical deletion when: keys are never reused, forget is terminal for the key, and the store returns a verifiable erasure receipt. Never choose “soft delete plus best-effort index rebuild” for anything that carries a deletion guarantee — that combination is precisely where resurrection lives. The decision rule: **if a key can be written after it is forgotten, forget must be generation-fenced; if it cannot, forget may be a bare tombstone.** Violating this rule produces either lost writes (bare tombstone over reused key) or resurrection (soft delete over rebuilt index). ## Operational Effect Downstream, the agent observes a successful forget and a successful write, then a read that contradicts one of them. The failure is silent: no exception, no error code, only a divergence between the audit log and the store. Recovery requires replaying the write with a fresh generation and re-verifying the tombstone was consumed, which is only possible if the store exposes tombstone state. Where it does not, the correct operational posture is to treat the scope as poisoned and rotate it. ```python class ResurrectionInvariantError(RuntimeError): """Raised when a read observes a generation that a prior forget claimed to remove, or when a post-forget write is suppressed by a stale tombstone.""" pass class ScopedMemory: def __init__(self): # key -> (generation, value) self._live = {} # key -> highest generation tombstoned self._tombstone_gen = {} self._gen_counter = 0 def write(self, key: str, value) -> int: self._gen_counter += 1 gen = self._gen_counter tomb = self._tombstone_gen.get(key, -1) if gen <= tomb: # A post-forget write must always be a newer generation than the tombstone. raise ResurrectionInvariantError( f"write generation {gen} not newer than tombstone {tomb} for key {key!r}" ) self._live[key] = (gen, value) return gen def read(self, key: str, expected_gen: int | None = None): entry = self._live.get(key) if entry is None: return None gen, value = entry tomb = self._tombstone_gen.get(key, -1) if gen <= tomb: # Resurrection: a live generation at or below the tombstone survived. raise ResurrectionInvariantError( f"read of key {key!r} gen {gen} violates tombstone gen {tomb}" ) if expected_gen is not None and gen != expected_gen: raise ResurrectionInvariantError( f"read of key {key!r} returned gen {gen}, expected {expected_gen}" ) return value def forget(self, key: str) -> int: entry = self._live.pop(key, None) if entry is None: # Idempotent: forgetting an absent key is a no-op, not an error. return self._tombstone_gen.get(key, -1) gen, _ = entry self._tombstone_gen[key] = max(self._tombstone_gen.get(key, -1), gen) return self._tombstone_gen[key] def apply_delayed_tombstone(self, key: str, tomb_gen: int) -> None: """Simulates an asynchronous compaction applying a stale tombstone.""" entry = self._live.get(key) if entry is not None: gen, _ = entry if gen <= tomb_gen: raise ResurrectionInvariantError( f"delayed tombstone gen {tomb_gen} would delete live gen {gen} for {key!r}" ) self._tombstone_gen[key] = max(self._tombstone_gen.get(key, -1), tomb_gen) ```