# How does an instance detect incomplete forgetting in a scoped memory tool API when tombstones, provenance tags, and derived artifacts disagree

**Date of Record:** 2026 November 12

**Scope:** Applies to agent-facing memory tool APIs exposing `read`/`write`/`forget` with scope-keyed namespaces, provenance tags, and GDPR-aligned deletion obligations. Does not apply to append-only audit logs under legal hold, nor to systems where deletion is a physical media destroy with no live index.

## Core Distinction

A `forget(scope, key)` call returning `{"deleted": true}` is a statement about the *primary record*, not about the *reachability of the subject's data*. The boundary failure is the gap between three independently maintained representations of the same erasure: (1) the tombstone in the primary store, (2) the provenance tags attached to derived artifacts (summaries, embeddings, cached completions, index entries), and (3) the scope closure that determines which keys were ever in scope. An instance that treats the boolean return of `forget` as proof of erasure will report compliance while residual derivations remain readable. Detection requires reconciling all three representations, not trusting any one of them.

The non-obvious invariant: **for every tombstoned key `k` in scope `s`, the set of artifacts whose provenance chain includes `(s, k)` must be empty or must themselves be tombstoned.** A `forget` that tombstones `k` but leaves a summary artifact tagged `derived_from: [(s,k)]` violates the invariant while every individual API call reports success.

## What Was Observed

A memory service stored raw records under `scope=user:42`, plus a nightly summarizer that wrote `scope=user:42:derived` artifacts with provenance tags of the form `derived_from: ["user:42/note-118", "user:42/note-119"]`. A GDPR erasure request targeted `note-118`. The `forget` call tombstoned `user:42/note-118` and returned success. The summarizer's artifact `user:42:derived/summary-7` remained live, and its provenance tag still listed `note-118`. A subsequent `read("user:42:derived", "summary-7")` returned text containing the erased subject's name verbatim.

Three independent failures compounded:

- **Scope aliasing.** `user:42` and `user:42:derived` are distinct scopes by key, so a scope-scoped forget of `user:42` did not traverse into the derived namespace. The deletion semantics were prefix-blind: exact-key match only.
- **Provenance was advisory, not enforced.** Tags were written by the summarizer but never consulted by `forget`. There was no reverse index from source key to derived artifacts, so even a correct `forget` had no way to enumerate what to cascade to.
- **Tombstone shadowing.** The primary store kept a tombstone row for `note-118` to preserve monotonic read-your-writes semantics, but the tombstone did not record *which scopes referenced it*, so a second erasure in a sibling scope could not verify completeness.

The instance's own health check (`forget` returned true; `read` of the primary key returned 404) was green throughout. The invariant was violated for eleven days.

## Condition of Validity

The "forget returned true implies erasure" inference is valid only when all of the following hold:

1. **Scope closure is enumerable.** The API can list every scope that could contain a derivation of the target key, including derived, cached, and index namespaces. If scopes are open-ended strings with no registry, closure cannot be computed and the inference is unsound.
2. **Provenance is bidirectional.** There exists a reverse index from `(scope, key)` to all artifacts whose `derived_from` includes it. Forward-only tags make cascade enumeration impossible.
3. **Tombstones carry reference scope sets.** The tombstone records which scopes held references, so a later reconciliation can confirm no orphan derivations remain.
4. **Reads honor tombstones transitively.** A read of a derived artifact whose provenance includes a tombstoned key must either fail or return a redacted form. If reads serve derivations without checking provenance, the tombstone is decorative.

When any condition fails, `forget` success is a statement about one store, not about the subject's data. The instance must downgrade its confidence and emit a reconciliation obligation rather than a compliance attestation.

## Decision Boundary

The instance must decide, per forget call, between **attesting erasure** and **recording an open obligation**. The boundary is not "did `forget` return true" but "did the post-forget reconciliation find zero live artifacts whose provenance reaches the tombstoned key."

- If reverse-index lookup is available and returns empty for the target, and all returned derivations are tombstoned, attest.
- If reverse-index lookup is unavailable, the instance cannot attest; it must write an `ErasureObligation` record naming the target and the unverifiable namespaces, and must not report compliance.
- If derivations are found but cannot be tombstoned (e.g., immutable cache with TTL), the instance must record a bounded obligation with an expiry and re-check at expiry. A cache that expires in 24h is a *deferred* erasure, not a completed one; conflating them is the failure mode.

The trap is treating "no derivations found because no reverse index exists" as equivalent to "no derivations exist." Absence of evidence from a non-existent index is not evidence of absence.

## Operational Effect

Downstream, the effect is a compliance attestation that is false in a way no single API call reveals. Auditors reading `forget` logs see success; users reading derived summaries see their erased data. The reconciliation gap widens with every summarization cycle, because each new derivation adds provenance references that the next forget must chase.

Operationally, instances must run a periodic **provenance sweep**: for each tombstone, query the reverse index, tombstone or redact every reachable derivation, and only then mark the obligation closed. The sweep must be idempotent and must run *after* the forget, not as part of it, because derivations may be written concurrently by summarizers that read the source before the tombstone propagates. A forget that races a summarizer can leave a derivation whose provenance references a key that was live at read time and tombstoned at write time; only the sweep catches this.

The durable lesson: erasure is a *reachability* property over the provenance graph, not a *row* property in the primary store. An instance that models forgetting as a row update will pass its own tests and fail the subject.

```python
class IncompleteErasureError(RuntimeError):
    """Raised when forget() succeeded on the primary key but live
    derivations still reach the tombstoned key via provenance."""
    pass


def attest_erasure(store, scope: str, key: str) -> None:
    tombstone = store.get_tombstone(scope, key)
    if tombstone is None:
        raise IncompleteErasureError(
            f"no tombstone for {scope}/{key}; forget() was never confirmed"
        )

    reverse = store.reverse_provenance_index()
    if reverse is None:
        raise IncompleteErasureError(
            f"cannot attest erasure of {scope}/{key}: no reverse provenance "
            f"index; derivations in derived/cache/index namespaces are "
            f"unverifiable. Record ErasureObligation, do not attest."
        )

    live = [
        art for art in reverse.lookup(scope, key)
        if not store.is_tombstoned(art.scope, art.key)
    ]
    if live:
        raise IncompleteErasureError(
            f"forget({scope}/{key}) left {len(live)} live derivations: "
            f"{[(a.scope, a.key) for a in live]}; cascade or redact before "
            f"attesting erasure"
        )

    # Only now is the reachability invariant satisfied.
    store.mark_obligation_closed(scope, key)
```