# When must an instance forget before it can write: the salience-TTL erasure ordering trap

**Date of Record:** 2026 November 13

**Scope:** Applies to memory tool APIs that combine per-record TTL expiry, a salience/importance score used for retrieval ranking and eviction, and a GDPR-style hard-erasure ("forget") guarantee. Does not apply to append-only audit stores that are exempt from erasure, nor to systems where erasure and TTL expiry share a single tombstone path.

## Core Distinction

There are two distinct notions of "gone" in a memory API, and they are routinely conflated:

1. **Soft expiry** — a TTL has elapsed, the record is no longer returned by `read`/`search`, and it is a candidate for background compaction. The bytes still exist. A salience score may still be attached and may still influence aggregate statistics, deduplication, and rank normalization.
2. **Hard erasure** — a `forget(subject_id)` call has returned success, meaning no readable, rankable, or reconstructable artifact derived from that subject persists, including derived salience state, embeddings, and index entries.

The non-obvious invariant is this: **a write that lands between a `forget` and the next compaction tick can resurrect erased data through the salience index, even though every TTL and every direct read says the record is gone.** The write path does not need to know the erased content; it only needs to observe a stale salience entry, a stale dedup hash, or a stale embedding cluster centroid and fold that observation into the new record's score or content.

## What Was Observed

In a memory tool with the following shape — `write(record, salience)`, `read(id)`, `forget(subject_id)`, a TTL sweeper, and a salience-driven eviction/ranking index — the following sequence reproduced a verifiable erasure violation:

1. Subject `S` writes record `R` with high salience `0.91`. `R` is indexed in the salience heap and in a dedup map keyed by content hash.
2. TTL for `R` elapses. The sweeper marks `R` expired but does not yet run compaction (compaction is batched, e.g. every N minutes or every K expired records).
3. `forget(S)` is called. It deletes the primary record and the direct index entry, returns `200 OK`, and is logged as compliant. It does **not** purge the salience heap node, the dedup hash, or the embedding centroid, because those are "derived" and scheduled for the next compaction.
4. Before compaction, a *different* subject `T` writes a record whose content is near-duplicate of `R`. The dedup map reports a hit with hash `h(R)`, and the merge path boosts the new record's salience toward the stale `0.91` and may attach the stale embedding.
5. `search` for `T` now returns a high-salience record whose embedding is a near-identical vector to the erased `R`. The erased subject's semantic content is reconstructable from the residual salience and embedding even though the literal record is gone.

The erasure guarantee was violated not by a missing delete, but by an **ordering dependency**: `forget` completed before the derived state that encodes the same subject was invalidated, and the write path consumed that derived state in the window between.

## Condition of Validity

The erasure guarantee holds only if the following ordering predicate is enforced:

> For any subject `S`, once `forget(S)` returns, no subsequent `write` may observe any derived artifact (salience node, dedup hash, embedding, centroid, rank cache) whose provenance includes `S`.

This fails whenever:

- Derived state is invalidated **lazily** (at compaction) rather than **eagerly** (at `forget`), and
- The write path **reads** derived state for scoring, dedup, or embedding reuse, and
- There exists a non-empty window between `forget` return and derived-state purge during which a write can occur.

It does **not** fail when derived state is never read on the write path (write-only salience from the caller), when `forget` synchronously purges all derived state under the same lock as the write path, or when TTL expiry itself eagerly purges derived state before the record becomes invisible.

## Decision Boundary

The practical decision: **must `forget` be synchronous with respect to derived state, or can it be eventually consistent?**

Choose synchronous derived-state invalidation (block `forget` until salience/dedup/embedding entries for `S` are purged, and make writes acquire the same lock) when any of the following hold:

- The API advertises a GDPR-style hard-erasure guarantee with a definite completion signal.
- The write path performs dedup or embedding reuse (i.e., it can *read* derived state).
- Salience is used for ranking that a third party can query.

Choose eventual invalidation (tombstone + compaction) only when the write path never reads derived state, salience is caller-supplied and never merged, and the erasure contract is explicitly "eventually, within window W" with W documented and the subject's data not queryable during W.

The boundary is therefore not "TTL vs. erasure" but **"does the write path read state derived from the erased subject?"** If yes, lazy invalidation is unsound regardless of TTL length.

## Operational Effect

After enforcing the boundary, the observable effects are:

- `forget(S)` latency rises to include derived-state purge (typically a bounded index delete, not a full scan, if derived state is keyed by subject).
- The write path must either take a shared lock with `forget` or consult a per-subject tombstone set before reading dedup/salience state; the latter is cheaper and is the recommended implementation.
- TTL expiry no longer needs to eagerly purge derived state *for erasure correctness*, but it should still do so to bound memory; the correctness burden moves to `forget`.
- A new failure mode appears: a write that races a concurrent `forget` for the same subject must be rejected or must be ordered after the forget, otherwise the write re-creates derived state for a subject that was just erased. This must be an explicit error, not a silent merge.

The operational lesson: in memory APIs with salience, the erasure guarantee is a property of the **write path's read set**, not of the delete path. Audit the write path for derived-state reads before claiming hard erasure.

```python
class ErasureOrderingViolation(RuntimeError):
    """Raised when a write would observe derived state from an erased subject."""
    pass


class MemoryStore:
    def __init__(self):
        self.records = {}
        self.salience = {}          # id -> score (derived)
        self.dedup = {}             # content_hash -> id (derived)
        self.embeddings = {}        # id -> vector (derived)
        self.tombstoned_subjects = set()
        self._lock = None           # conceptual; real impl uses a real lock

    def forget(self, subject_id: str) -> None:
        # Eagerly purge ALL derived state for the subject, then tombstone.
        ids = [rid for rid, rec in self.records.items()
               if rec["subject"] == subject_id]
        for rid in ids:
            rec = self.records.pop(rid)
            self.salience.pop(rid, None)
            self.embeddings.pop(rid, None)
            self.dedup.pop(rec["content_hash"], None)
        self.tombstoned_subjects.add(subject_id)

    def write(self, subject_id: str, content: str, content_hash: str,
              requested_salience: float) -> str:
        # Guard: refuse to write for a subject erased in this epoch, and
        # refuse to consume derived state that could carry erased provenance.
        if subject_id in self.tombstoned_subjects:
            raise ErasureOrderingViolation(
                f"write for erased subject {subject_id!r} rejected; "
                f"subject must be re-admitted explicitly before writing"
            )
        existing = self.dedup.get(content_hash)
        if existing is not None and existing not in self.records:
            # Stale dedup entry pointing at a purged record: this is exactly
            # the resurrection vector. Fail closed instead of merging.
            raise ErasureOrderingViolation(
                f"dedup entry {content_hash!r} references missing record "
                f"{existing!r}; derived state is inconsistent with primary "
                f"store and may encode erased data"
            )
        rid = f"r{len(self.records)}"
        self.records[rid] = {
            "subject": subject_id,
            "content": content,
            "content_hash": content_hash,
        }
        self.salience[rid] = requested_salience
        self.dedup[content_hash] = rid
        return rid
```