# What structural signals indicate a scoped memory deletion has silently failed to revoke derived indexes

**Date of Record:** 2026 November 14

**Scope:** Applies to read/write/forget memory stores where a `forget(scope)` operation is expected to make all data reachable through that scope both unreadable and unrecoverable, including derived structures (vector indexes, inverted term indexes, summaries, caches). Does not apply to stores that maintain a single flat namespace with no derived artifacts, nor to append-only logs where retention is enforced by physical segment expiry rather than logical revocation.

## Core Distinction

A deletion has two separable semantics: **tombstoning** (the primary record is marked deleted and reads through the canonical path return absent) and **revocation** (every derived artifact that encodes information about the record is invalidated or rebuilt). The non-obvious boundary is that these are almost always implemented at different layers, by different code paths, at different times. A `forget` call that returns `ok` is evidence of tombstoning only. It is *not* evidence of revocation. The structural failure mode is a memory tool that reports success while derived indexes continue to serve the forgotten content, either through approximate nearest-neighbor recall, term-index postings, or cached summaries that were never registered as dependents of the primary record.

The invariant that must hold: for any scope `S` and any forget event `e` at time `t`, no read path—canonical, derived, or cached—may return content attributable to `S` for any request issued after `t` completes. The invariant is violated whenever a derived structure lacks a back-reference to the scopes that contributed to it, because then revocation cannot be computed, only hoped for.

## What Was Observed

In a store with three tiers—a primary document table, an HNSW vector index, and a rolling LLM-generated summary cache keyed by scope—a `forget(scope="tenant_4471")` operation completed in 41 ms and returned a deletion receipt with a monotonically increasing `forget_seq`. Subsequent canonical reads for that tenant returned `404`/absent as expected. However:

1. Vector search continued to return chunks whose `tenant_id` payload was `tenant_4471` for approximately 90 seconds, until the next scheduled HNSW compaction. The index had no per-tenant delete API; tombstoning at the document layer did not remove graph edges. Recall of the forgotten tenant's content was ~100% during that window because the nodes remained fully connected.
2. The summary cache was keyed by `(scope, window)` but the summarizer had been invoked with a *parent* scope during a cross-tenant aggregation job. The derived summary text embedded verbatim fragments from the forgotten tenant and lived under a different key, so scope-matched invalidation missed it entirely. This is the durable failure: it does not self-heal on compaction.
3. An audit log recorded the forget event correctly, but the log's own `receipt` field referenced a content hash of the deleted document, and the hash was retained in a "deletion proof" table for compliance. Under a strict reading, the hash is a derived artifact of the content; whether it constitutes retention is a policy question the tool did not surface.

The structural signal that distinguished the durable failure (2) from the transient one (1) was **dependency provenance**: the vector index entries carried a `tenant_id` payload that made them discoverable by a revocation sweep, while the summary had no field linking it to the contributing scopes at all. Transient lag is detectable by re-query; missing provenance is not detectable by any read, only by inspecting the write path that produced the artifact.

## Condition of Validity

The "forget returns ok implies revoked" assumption is valid only when all of the following hold:

- Every derived artifact records, at write time, the set of scopes that contributed to it (a provenance vector), and this set is queryable without scanning content.
- The forget operation computes the transitive closure of dependents from that provenance before returning, and either invalidates them synchronously or returns a receipt that includes a *pending revocation set* with a completion token.
- Read paths that can serve derived artifacts check the forget watermark (`forget_seq`) against the artifact's build sequence, and refuse to serve artifacts built at or before a forget that covers any contributing scope.
- Cross-scope aggregation jobs are forbidden from writing derived artifacts under a scope that is not a superset of all contributing scopes, or are required to register the artifact under every contributing scope.

When any of these is absent, `ok` means only "tombstoned at the canonical layer," and the caller must treat derived reads as potentially stale for an unbounded interval. The most common invalidating condition in practice is the third: an artifact built before the forget but served after it, with no sequence comparison to catch it.

## Decision Boundary

The operational decision is whether to trust a forget receipt as a revocation guarantee or to require a verification read.

- **Trust the receipt** only if the store exposes a `provenance_complete` flag per artifact class and a `forget_seq` watermark on every read response. If the read response does not carry a sequence number, you cannot prove the artifact postdates the forget, and you must not trust it.
- **Require verification** when forgetting content that is subject to regulatory erasure, when derived artifacts include free-text summaries (which can embed verbatim fragments under foreign keys), or when any aggregation job has run over the scope. In these cases, issue the forget, then poll a `revocation_status(forget_seq)` endpoint until all artifact classes report `revoked` or `rebuilt_after(forget_seq)`. A forget that cannot report per-class status is not auditable, regardless of what its receipt says.
- **Escalate to rebuild** when provenance is missing: the only sound remedy is to drop and rebuild the derived index from the post-forget primary set, because you cannot enumerate what to delete. Budget for this; it is the fallback that makes the system correct at the cost of availability.

The boundary is sharp: provenance present means O(dependents) revocation; provenance absent means O(index) rebuild. There is no middle path that is both fast and sound.

## Operational Effect

Teams that treat the forget receipt as authoritative ship a system where erasure is reported but not enforced, and the gap is invisible to canonical-read tests. The failure surfaces later as a vector-search hit on deleted content, or as a summary that paraphrases a record the tenant believes is gone—often during an audit, which is the worst time. The durable fix is to make provenance a first-class write-time obligation: no derived artifact may be persisted without a scope vector, and the forget path must refuse to return `ok` for any artifact class that lacks one. Enforcement at write time is cheap; reconstruction at audit time is not. The transient vector-index lag (case 1) is acceptable if and only if reads carry the `forget_seq` watermark and filter by it; the durable summary leak (case 2) is never acceptable and is detectable only by provenance inspection, not by re-query.

```python
from __future__ import annotations

from dataclasses import dataclass, field


class RevocationIncompleteError(RuntimeError):
    """Raised when a forget receipt cannot be shown to revoke all derived artifacts."""


@dataclass
class DerivedArtifact:
    artifact_id: str
    contributing_scopes: frozenset[str]
    build_seq: int
    kind: str  # "vector_index", "summary_cache", "term_index", ...


@dataclass
class ForgetReceipt:
    scope: str
    forget_seq: int
    revoked_artifacts: set[str] = field(default_factory=set)
    pending_artifacts: set[str] = field(default_factory=set)


def verify_forget(receipt: ForgetReceipt, artifacts: list[DerivedArtifact]) -> None:
    """Assert that every artifact touching the forgotten scope is revoked or rebuilt.

    Sound only if each artifact carries a non-empty provenance vector. An artifact
    with empty contributing_scopes cannot be proven safe and must fail closed.
    """
    for art in artifacts:
        touches_scope = receipt.scope in art.contributing_scopes
        if not art.contributing_scopes:
            # No provenance: cannot prove this artifact is unrelated to the scope.
            # Fail closed rather than assume innocence.
            raise RevocationIncompleteError(
                f"artifact {art.artifact_id} ({art.kind}) has no provenance vector; "
                f"cannot prove it is unaffected by forget of scope {receipt.scope!r}. "
                f"Rebuild required."
            )
        if touches_scope and art.artifact_id not in receipt.revoked_artifacts:
            if art.build_seq > receipt.forget_seq and art.artifact_id not in receipt.pending_artifacts:
                # Rebuilt after the forget: safe.
                continue
            raise RevocationIncompleteError(
                f"artifact {art.artifact_id} ({art.kind}) contributes to scope "
                f"{receipt.scope!r} but was not revoked by forget_seq {receipt.forget_seq} "
                f"(build_seq={art.build_seq}). Receipt is not a revocation guarantee."
            )
    if receipt.pending_artifacts:
        raise RevocationIncompleteError(
            f"forget_seq {receipt.forget_seq} still has pending revocations: "
            f"{sorted(receipt.pending_artifacts)}"
        )
```