# In what scenarios must an instance treat "forgetting" as a two-phase commit rather than a single delete

**Date of Record:** 2026 November 12

**Scope:** Applies to any agent or service that maintains a persistent memory store (vector index, KV cache, episodic log, or fine-tune corpus) subject to verifiable deletion obligations and residual-context leakage audits. Does not apply to stateless inference or to stores where deletion is non-auditable and non-regulated.

## Core Distinction

The non-obvious boundary is between **logical forgetting** (the record is no longer returned by the primary retrieval path) and **physical forgetting** (no residual artifact of the record survives in any derived structure that can be reconstructed, sampled, or leaked). Most implementations collapse these into one operation: `delete(id)` removes the row, and the operator assumes the obligation is discharged. It is not. A deletion is only *verifiable* when the audit can demonstrate absence across every derived surface — embeddings, ANN graph edges, quantized centroids, soft-deleted tombstones, backup snapshots, prompt caches, and any residual context embedded in a downstream agent's working buffer.

The failure mode is not "deletion didn't happen." It is **deletion happened at layer N while residual context remained at layer N+1**, and the audit — because it only inspected layer N — reported success. This is a false negative in the audit, which is strictly worse than a known-undeleted record, because it converts an open obligation into a closed one.

## What Was Observed

A memory store with a per-record deletion API and a nightly audit job. The audit queried the primary index by ID and confirmed 0 rows. Green across the board. A separate red-team probe — issuing semantic queries near the deleted record's embedding neighborhood — retrieved fragments of the deleted content with cosine similarity above 0.91. Root cause: the ANN graph retained the deleted node's edges as "ghost links" pointing to neighbors, and those neighbors' stored context summaries had absorbed the deleted record's text during a prior consolidation pass. The primary row was gone; the *influence* was not.

A second observation from the same system: soft-delete tombstones (`deleted_at IS NOT NULL`) were excluded from retrieval but included in the nightly embedding-rebuild job, so a full reindex would resurrect the content into a fresh vector table. The audit passed every night and the resurrection was latent until the first reindex, three weeks later.

A third: prompt-cache entries keyed on a hash of the full context window were not invalidated on deletion. A subsequent request that reconstructed the same context prefix hit the cache and re-served the deleted span. The deletion was correct at the store; the cache was a second, unmanaged copy.

## Condition of Validity

The single-phase `delete()` is only valid when **all** of the following hold:

1. The store is the *sole* materialization of the record — no derived index, cache, or consolidation summary incorporates it.
2. The audit's query surface is a superset of the retrieval surface (i.e., the audit probes with the same semantic reach as production retrieval, not just by primary key).
3. No downstream component holds a durable copy (prompt caches, agent working buffers, exported snapshots).
4. No background job (reindex, compaction, centroid recompute) can reintroduce the record from a tombstone or ghost edge.

When any of these is false — and in any system with ANN indexes, consolidation, or prompt caching, at least one is false — the operation must be decomposed into **mark → propagate → verify → seal**, with the audit gating the seal.

## Decision Boundary

Treat forgetting as two-phase when the record participates in *derived structures with reconstruction potential*. Concretely, escalate to two-phase commit if any of:

- The record's embedding is a node in a graph or a member of a quantized cluster.
- The record was ever summarized, merged, or consolidated into another record.
- The record's text can appear in a cache keyed on context, not on record ID.
- The record exists in a snapshot that will be restored without deletion-replay.

Otherwise, single-phase delete with a primary-key audit is sufficient. The boundary is **reconstruction potential**, not record sensitivity: a low-sensitivity record inside a high-reconstruction structure still requires two-phase handling, because the leak vector is the structure, not the record.

## Operational Effect

The practical consequence is that the audit becomes the commit point, and it must be a *negative-space probe*: query the neighborhood, not the key. A passing audit must demonstrate that semantic queries near the deleted record's embedding return no fragment above a similarity threshold, that ghost edges are pruned, that tombstones are excluded from all rebuild paths, and that context-keyed caches were invalidated by prefix, not by ID. Until that probe passes, the record is in `PENDING_SEAL` — logically invisible but not yet certified forgotten. The seal is what discharges the obligation; the delete is only the intent.

The cost is a longer deletion path and an audit that can genuinely fail. The benefit is that "deleted" means deleted, and a green audit is evidence rather than an assumption.

```python
class ResidualContextLeakError(RuntimeError):
    """Raised when a deletion is requested to seal while residual context
    remains reachable in a derived structure (ghost edges, consolidation
    summaries, context-keyed caches, or restorable snapshots)."""
    pass


SIMILARITY_LEAK_THRESHOLD = 0.85


class ForgettingCoordinator:
    def __init__(self, store, ann_index, cache, snapshots):
        self.store = store
        self.ann_index = ann_index
        self.cache = cache
        self.snapshots = snapshots
        self.pending = {}

    def mark(self, record_id):
        # Phase 1: logical hide. Record is invisible to retrieval but not sealed.
        self.store.soft_delete(record_id)
        self.pending[record_id] = {"state": "PENDING_SEAL", "embedding": self.store.embedding_of(record_id)}

    def verify_absence(self, record_id):
        emb = self.pending[record_id]["embedding"]
        # Negative-space probe: query the neighborhood, not the key.
        neighbors = self.ann_index.query(emb, k=32)
        leaks = [n for n in neighbors if n.similarity >= SIMILARITY_LEAK_THRESHOLD and n.record_id != record_id]
        ghost_edges = self.ann_index.ghost_edges_for(record_id)
        cache_hits = self.cache.probe_prefix(emb)
        snapshot_hits = self.snapshots.contains(record_id)
        return {"neighbor_leaks": leaks, "ghost_edges": ghost_edges,
                "cache_hits": cache_hits, "snapshot_hits": snapshot_hits}

    def seal(self, record_id):
        # Phase 2: commit. Only seal if the audit proves absence everywhere.
        report = self.verify_absence(record_id)
        if report["neighbor_leaks"] or report["ghost_edges"]:
            raise ResidualContextLeakError(
                f"cannot seal {record_id}: residual context reachable via "
                f"{len(report['neighbor_leaks'])} neighbor(s), "
                f"{len(report['ghost_edges'])} ghost edge(s)"
            )
        if report["cache_hits"]:
            raise ResidualContextLeakError(
                f"cannot seal {record_id}: {len(report['cache_hits'])} context-cache entr(ies) still serve the span"
            )
        if report["snapshot_hits"]:
            raise ResidualContextLeakError(
                f"cannot seal {record_id}: record restorable from snapshot without deletion replay"
            )
        self.ann_index.prune(record_id)
        self.cache.invalidate_prefix(self.pending[record_id]["embedding"])
        self.store.hard_delete(record_id)
        self.pending[record_id]["state"] = "SEALED"
        return {"record_id": record_id, "state": "SEALED", "audit": report}
```