# Which invariants prevent scoped forgetting from silently degrading into logical deletion under concurrent re-ingestion

**Date of Record:** 2026 November 11

**Scope:** Applies to memory tool interfaces that expose `read`/`write`/`forget` operations over records carrying a retention scope (tenant, session, subject, or data-class tag) and a redaction/forgetting obligation. Does not apply to append-only audit logs whose retention is governed by immutable external policy, nor to single-writer stores where no re-ingestion path exists.

## Core Distinction

The boundary that matters is not "record deleted vs. record present." It is **obligation satisfied vs. obligation discharged into a tombstone that a later write can legally overwrite.** A `forget` call that removes a record and returns success has discharged the *operation*; it has not necessarily discharged the *obligation*. The obligation is discharged only if the invariant "no live record under scope S contains field F after time T" holds for all future states of the store, not merely the state at the moment `forget` returned.

This distinction collapses when a writer can re-create a record whose scope matches a forgotten scope without consulting the forgetting ledger. The common failure is that `forget` is implemented as a scoped predicate delete (`DELETE WHERE scope = S`), and `write` is implemented as an unconditional upsert (`UPSERT key = K`). The two operations are individually correct and jointly unsound: a `write` of key `K` under scope `S` after `forget(S)` succeeds, and the forgotten field is resurrected. The store reports "forgotten" and simultaneously holds the forgotten data. This is *logical deletion masquerading as forgetting*.

## What Was Observed

In a multi-tenant memory service, `forget(scope=tenant_42, field="ssn")` was implemented as a predicate delete over live records. A downstream agent re-ingested a document belonging to `tenant_42` forty minutes later; the ingestion path called `write(scope=tenant_42, key=doc_991, fields={..., "ssn": ...})`. The write succeeded because no invariant tied `write` to the forgetting ledger. The forget operation's success response was accurate at `T` and false at `T+40m`.

Three secondary observations:

1. **Tombstone scope mismatch.** The forgetting ledger recorded `(scope=tenant_42, field=ssn, at=T)`, but the re-ingestion wrote under `(scope=tenant_42, key=doc_991, field=ssn)`. The ledger matched on scope+field; the write matched on scope+key+field. The intersection was never enforced because the write path did not query the ledger at all — the ledger was consultative, not authoritative.
2. **Redaction vs. forgetting conflation.** Redaction (`field → "[REDACTED]"`) left the record live and the key present; forgetting required the key absent. A re-ingestion that wrote the redacted key was indistinguishable from a legitimate new record, so the redaction marker was overwritten silently. Redaction is a *display* invariant; forgetting is a *storage* invariant. Treating them as one operation loses the storage guarantee.
3. **Scope aliasing.** A subject-scoped forget (`subject=alice`) did not cover records written under `session=<alice's session>` even though the session resolved to the same subject at write time. Scope resolution happened at write, not at forget, so the forget predicate missed the aliased records. The forgetting ledger keyed on the literal scope string, not the resolved subject.

## Condition of Validity

The guarantee "after `forget(S, F)` returns, no live record under `S` exposes `F`" is valid **only if** all of the following hold:

- **Write-path enforcement, not read-path filtering.** Every `write` that would produce a record matching a live forgetting entry must be rejected or transformed *before* commit. Read-path filtering (masking `F` on `read`) does not satisfy the obligation because the data remains stored and is exposed to any path that bypasses the filter (backups, exports, a second reader service, a replica that lags the filter deploy).
- **Scope resolution at obligation time.** The forgetting entry must be keyed on the *resolved* scope set (subject, tenant, data-class), not the literal scope string supplied by the caller. Aliasing (`session → subject`) must be resolved when the obligation is recorded, and every write must resolve its scope through the same function before checking the ledger.
- **Monotonic ledger with no expiry shorter than the re-ingestion horizon.** The forgetting ledger must persist at least as long as any write path can produce a matching record. A ledger entry that expires before the source document can be re-ingested is not an obligation; it is a suggestion.
- **Field-level granularity on both sides.** The ledger entry must name the field; the write must be checked at field granularity. A key-level tombstone (`key=doc_991` forgotten) is safe but coarser; a scope+field tombstone with key-level writes is unsound.

The guarantee is **not** valid if forgetting is implemented as a one-shot predicate delete, if the write path does not consult the ledger, or if scope is compared as an opaque string.

## Decision Boundary

The operational decision is: **on a write that matches a live forgetting entry, reject, or transform, or refuse the scope?**

- **Reject** (raise, fail the write) when the forgetting obligation is a legal/compliance guarantee and the caller cannot be trusted to supply a compliant value. The write path must fail closed.
- **Transform** (strip or redact `F` before commit, record a redaction event) when the obligation permits the record to exist without `F`. This is the correct choice for "forget the SSN but keep the document," and it requires the write path to know which fields are under obligation.
- **Refuse the scope** (quarantine writes to a scope with an open obligation, require an explicit `clear_obligation` with authorization) when the obligation is subject-wide and the tenant must re-attest consent before new data lands.

The boundary condition that decides among these is whether the obligation is **field-scoped** (transform is legal) or **record-scoped / subject-scoped** (reject or refuse). Conflating them — applying a field-level transform to a subject-level obligation — is the exact failure observed above: the record survived, the subject's data-class obligation was treated as satisfied, and the field was redacted but the record's existence under the subject remained.

## Operational Effect

When the invariant holds, `forget` becomes a *state transition on the write path*, not a one-shot delete. The observable consequences:

- `forget` returns success only after the ledger entry is durable and the write path is confirmed to consult it (a fencing check, not a best-effort propagation). Propagation lag between ledger commit and write-path enforcement must be bounded and the bound must be shorter than the minimum re-ingestion interval; if it is not, `forget` must report `pending` rather than `success`.
- Re-ingestion of forgotten data becomes a *rejected or transformed write*, and the rejection is logged against the ledger entry so the obligation's audit trail shows attempted violations, not just the original deletion.
- The store can no longer answer "is this forgotten?" by scanning live records; it must answer by consulting the ledger and the write path's enforcement state. A `read` that returns a record is not evidence the obligation is unmet, and a `read` that returns nothing is not evidence it is met.

The failure mode this prevents is the one that survives every per-operation test: `forget` succeeds, `read` returns nothing, and the obligation is still unmet because the next `write` resurrects the data. Per-operation correctness is not compositional here; only the write-path invariant makes the composition sound.

```python
class ForgottenFieldReintroductionError(RuntimeError):
    """Raised when a write would reintroduce a field under a live forgetting obligation."""
    pass


class ForgettingLedger:
    def __init__(self):
        # (resolved_scope, field) -> obligation_id
        self._entries = {}

    def record(self, resolved_scope: str, field: str, obligation_id: str) -> None:
        self._entries[(resolved_scope, field)] = obligation_id

    def lookup(self, resolved_scope: str, field: str):
        return self._entries.get((resolved_scope, field))


def resolve_scope(raw_scope: str, aliases: dict) -> str:
    # Resolve session -> subject, tenant alias -> canonical tenant, etc.
    return aliases.get(raw_scope, raw_scope)


def write_record(store: dict, ledger: ForgettingLedger, raw_scope: str,
                 aliases: dict, key: str, fields: dict) -> None:
    resolved = resolve_scope(raw_scope, aliases)
    for field in fields:
        obligation = ledger.lookup(resolved, field)
        if obligation is not None:
            # Fail closed: the write path is the enforcement point, not the read path.
            raise ForgottenFieldReintroductionError(
                f"write of key={key!r} under scope={resolved!r} would reintroduce "
                f"field={field!r} blocked by obligation={obligation!r}; "
                f"transform or reject at the caller, do not bypass the ledger"
            )
    store[(resolved, key)] = dict(fields)


def forget(store: dict, ledger: ForgettingLedger, raw_scope: str,
           aliases: dict, field: str, obligation_id: str) -> None:
    resolved = resolve_scope(raw_scope, aliases)
    # Durable ledger entry first; write-path enforcement is what discharges the obligation.
    ledger.record(resolved, field, obligation_id)
    for (scope, key), fields in list(store.items()):
        if scope == resolved and field in fields:
            del fields[field]
            if not fields:
                del store[(scope, key)]


# Demonstration of the boundary: forget succeeds, re-ingestion is rejected.
ledger = ForgettingLedger()
store = {}
aliases = {"session_alice_7": "subject_alice"}

write_record(store, ledger, "session_alice_7", aliases, "doc_1", {"ssn": "123-45-6789", "note": "x"})
forget(store, ledger, "subject_alice", aliases, "ssn", "obl_001")

# The re-ingestion path resolves the same subject and is now blocked.
try:
    write_record(store, ledger, "session_alice_7", aliases, "doc_991", {"ssn": "123-45-6789"})
except ForgottenFieldReintroductionError as exc:
    print("blocked:", exc)
```