# Which invariants prevent provable erasure from silently degrading when vector index compaction and episodic agent logs drift out of sync **Date of Record:** 2026 November 15 **Scope:** Applies to systems that satisfy "right to erasure" obligations by deleting embeddings from an ANN index and deleting episodic log records from an append-only agent transcript store, where the two stores are updated by independent compaction and retention processes. Does not apply to systems where a single transactional store owns both the vector and the log, or where erasure is enforced by cryptographic key destruction over a single keyspace. ## Core Distinction There are two distinct claims that get conflated under "forgetting compliance": 1. **Logical erasure** — the record is no longer reachable by any read path: no query returns it, no replay reconstructs it, no snapshot contains it. 2. **Provable erasure** — there exists a verifiable certificate that *every* physical copy derived from the record has been removed or rendered unrecoverable, including derived artifacts: ANN graph neighbors, tombstone-free segment files, log-append fragments, and any summaries the agent produced *from* that record. The failure mode is not that erasure is skipped. It is that erasure is applied to the vector store and the episodic log independently, each with its own notion of "done," and the two notions diverge at exactly the boundary where a derived artifact lives. A vector deletion is "done" when the tombstone is applied and the segment is compacted. A log deletion is "done" when the retention window passes. Neither process knows that the *other* store may still hold a reconstruction of the deleted content. The non-obvious invariant: **a record is not provably erased until the last derived artifact that references it is unreachable, and derived artifacts can outlive their source in either store.** An agent summary written into the episodic log can embed the semantic content of an embedding that has already been tombstoned; a re-indexed vector can be regenerated from a log fragment that has not yet been compacted. ## What Was Observed A compliance audit requested proof of erasure for a set of user records spanning 90 days. The vector store reported 100% tombstone application and segment compaction for all target IDs. The episodic log reported 100% retention expiry for the same IDs. Both stores individually passed their erasure checks. A reconstruction probe — re-embedding the agent's own logged summaries and running nearest-neighbor search against the live index — returned matches at high cosine similarity to the *supposedly erased* records' neighborhoods. The mechanism: the agent had written episodic summaries that paraphrased the erased content, and those summaries were themselves embedded into the vector store under a different record ID (the summary's ID, not the source's ID). The source record's tombstone was correct. The summary vector was never in scope because it was never linked to the source ID. A second, independent leak appeared in the ANN graph: soft-delete (tombstone) followed by a compaction that rebuilt the HNSW graph retained the deleted node as a *routing* neighbor for a small fraction of queries. The node was not returned as a result, but its vector influenced traversal, and in a low-cardinality cluster the traversal path was observable via latency side-channel. Compaction had removed the payload but not the graph-position residue. Neither store was "wrong" by its own contract. The contract was incomplete: it scoped erasure to record identity, not to record *derivation*. ## Condition of Validity The invariant holds — erasure is provable — only when all of the following are true simultaneously: - Every derived artifact (summary, embedding, index entry, cache line, graph edge) carries a **provenance link** to its source record(s), and erasure is defined transitively over the provenance graph, not over the record ID set. - The vector store's compaction is **graph-rebuilding**, not merely payload-removing, and the rebuild is verified to exclude tombstoned nodes from routing adjacency, not just from result sets. - The episodic log's retention expiry and the vector store's compaction are **ordered and acknowledged** against a shared erasure manifest; neither store may declare "done" for a record until the other has acknowledged removal of all derived artifacts for that record. - The reconstruction probe (re-embed logged text, search live index) is run as a **post-condition**, not as an audit-time check. If any of these is false, logical erasure may still hold while provable erasure does not, and the gap is invisible to per-store health metrics. ## Decision Boundary The operational decision is whether to treat a record as erasable-in-place or to require **derivation-scoped destruction**: - If provenance links are complete and compaction is graph-rebuilding, erase in place and certify against the manifest. - If provenance links are incomplete (the common case for agent-generated summaries), do **not** certify. Escalate to derivation-scoped destruction: identify all artifacts whose embeddings fall within a similarity neighborhood of the source, quarantine them, and re-run the reconstruction probe. Accept the higher cost; the alternative is a false certificate. - If the ANN implementation cannot guarantee graph-rebuild on compaction, treat the index as **not erasable** and require index rotation (rebuild from a clean corpus) for the affected segment. The boundary is crossed the moment a derived artifact exists whose provenance is not recorded. At that point, per-store erasure certificates are unsound regardless of how many checks pass. ## Operational Effect Teams that rely on per-store erasure metrics will report compliance that a reconstruction probe falsifies. The practical consequences: - Audit findings surface late and as a class, not as individual records, because the leak is structural (derivation gap) rather than incidental. - Remediation cost scales with the provenance gap, not with record count: a single unlinked summary can require re-embedding and re-indexing an entire neighborhood. - The reconstruction probe must become a standing post-condition on every erasure batch; it is the only check that catches derivation leaks, because it is the only check that tests reachability *from the erased content's semantics* rather than from its identity. The invariant to encode: **erasure is transitive over provenance, and provable only when every store acknowledges the full derived closure, in order, against a shared manifest.** ```python class ErasureProvenanceError(RuntimeError): """Raised when a record's derived-artifact closure is not fully acknowledged by every store, so a provable-erasure certificate cannot be issued.""" pass def certify_erasure(record_id, manifest, vector_ack, log_ack, probe_similarity_threshold=0.85): """ Issue a provable-erasure certificate for record_id only if: - the full derived closure is present in the shared manifest, - both the vector store and the episodic log have acknowledged every derived artifact in that closure, - the post-condition reconstruction probe returns no match above threshold. """ closure = manifest.derived_closure(record_id) if closure is None: raise ErasureProvenanceError( f"record {record_id}: no provenance closure in manifest; " f"derived artifacts may be unlinked and erasure is not provable" ) missing_vector = closure - vector_ack.acknowledged(record_id) missing_log = closure - log_ack.acknowledged(record_id) if missing_vector or missing_log: raise ErasureProvenanceError( f"record {record_id}: unacknowledged derived artifacts " f"vector={sorted(missing_vector)} log={sorted(missing_log)}" ) if not vector_ack.compaction_was_graph_rebuild(record_id): raise ErasureProvenanceError( f"record {record_id}: vector compaction removed payload but did not " f"rebuild ANN routing adjacency; graph-position residue remains" ) probe = reconstruction_probe(record_id, vector_store, episodic_log) if probe.max_similarity >= probe_similarity_threshold: raise ErasureProvenanceError( f"record {record_id}: reconstruction probe matched at " f"cosine={probe.max_similarity:.3f} (>= {probe_similarity_threshold}); " f"derived content still reachable" ) return ErasureCertificate(record_id, closure, probe.max_similarity) ```