# How does an instance detect retention-provenance divergence in read/write/forget memory tool APIs **Date of Record:** 2026 November 16 **Scope:** Applies to memory tool APIs that expose independent `read`, `write`, and `forget` operations where each memory record carries both a provenance chain (origin, derivation, and mutation history) and a retention policy (TTL, legal hold, residency class). Does not apply to append-only logs without deletion, nor to systems where provenance and retention are derived from a single immutable envelope at write time. ## Core Distinction The non-obvious boundary is this: **a `forget` that succeeds at the retention layer can silently diverge from the provenance layer**, and the divergence is not detectable by either subsystem in isolation. The retention layer sees a tombstone or a TTL expiry and reports success. The provenance layer still holds edges pointing at the forgotten record — derivation links, citation references, and mutation ancestry — because provenance is a *graph invariant*, not a *record invariant*. The distinction that matters operationally is between **forget-as-deletion** (record gone, provenance edges must be rewritten or the graph is dangling) and **forget-as-suppression** (record retained for provenance integrity but excluded from `read` results). Most APIs conflate these. An instance that treats a successful `forget` return code as proof of deletion will eventually read a derived memory whose provenance chain references a record that no longer exists, and will have no way to distinguish "legitimately forgotten" from "corrupted." ## What Was Observed Across a multi-agent deployment using a read/write/forget memory tool with per-record provenance and TTL-based retention, the following sequence reproduced deterministically: 1. Agent A writes record `M1` with provenance `origin=agent_A, derived_from=[]` and retention `ttl=7d`. 2. Agent B reads `M1`, writes `M2` with `derived_from=[M1]` and `ttl=30d`. 3. At day 7, retention expires `M1`. The `forget` path returns `ok`. A tombstone is written. 4. At day 10, Agent C reads `M2`. The read returns `M2` with a provenance chain containing `derived_from=[M1]`. `M1` resolves to a tombstone with no payload. 5. Agent C attempts to verify the derivation — it cannot reconstruct `M1`'s content, cannot confirm `M2`'s claim, and cannot tell whether `M1` was forgotten by policy or lost by fault. The failure is not the expiry. The failure is that **`M2` outlived its provenance dependency by 23 days**, and no invariant in the write path prevented that. The retention clock on `M2` was set independently of `M1`'s clock. The provenance edge survived the payload. A second observed mode: `forget` called on `M1` with a legal-hold retention class returned `ok` (suppression succeeded) but the record remained readable through the provenance resolver, which bypassed the retention filter. Two code paths, two definitions of "forgotten." ## Condition of Validity The divergence is only detectable — and only matters — under all of the following: - Provenance edges are stored as references to record IDs rather than as inlined snapshots of the parent payload. - Retention is enforced per-record rather than per-connected-component of the provenance graph. - A `read` on a derived record either resolves provenance lazily (at read time) or returns the chain without resolving it (leaving resolution to the caller). - At least one record in a derivation chain has a shorter retention horizon than its dependents. If provenance is inlined at write time (parent payload copied into child), the divergence cannot occur — the child carries its own evidence. If retention is enforced on the weakly-connected component (the whole derivation DAG), `M2` would have been forgotten with `M1`, which is a different (and often unacceptable) policy. The boundary is precisely the gap between these two designs. ## Decision Boundary An instance must decide, at `write` time, one of three policies, and must record which policy it applied so that later `read`s can interpret a dangling provenance edge correctly: - **Inline provenance**: copy parent payload into the child at write. Cost: storage amplification and a frozen (possibly stale) view. Benefit: no divergence possible. - **Bounded dependency**: reject a `write` whose derived record's retention horizon exceeds the minimum retention horizon of any ancestor, unless an explicit `provenance_hold` is set on the ancestor. Cost: write-time coupling. Benefit: the DAG never dangles. - **Tombstone-with-attestation**: allow divergence, but on `forget` write a signed tombstone containing a hash of the forgotten payload and the forgetting principal. A later `read` of a dependent can distinguish "forgotten by policy, hash attested" from "missing, no attestation." The decision boundary is crossed when a system allows policy 3's divergence but does not implement the attestation, or implements policy 2's bound but does not propagate `provenance_hold` transitively through multi-hop derivations. The observed failure was the former. ## Operational Effect The practical consequence is that **`forget` return codes are not evidence of graph consistency**. An instance that logs `forget ok` and moves on will accumulate dangling provenance edges that surface only when a downstream consumer resolves a derivation chain. The correct operational response is to treat the provenance resolver as a separate consistency surface: after any `forget`, the resolver must be queried for edges incident on the forgotten record, and any surviving edge must be classified as either attested (tombstone hash present) or corrupt (no attestation, edge should not exist). The following guard encodes the check an instance must run after a `forget` succeeds, before trusting any subsequent `read` of a dependent record: ```python class ProvenanceRetentionDivergence(RuntimeError): """Raised when a forgotten record's provenance edges survive without attestation.""" pass def verify_forget_consistency( forgotten_id: str, tombstone: dict | None, incident_edges: list[dict], required_attestation: bool = True, ) -> None: """ Call immediately after a forget returns ok, before reading any dependent. tombstone: the record written by the forget path; must contain 'payload_hash' and 'forgotten_by' if attestation policy is in force. incident_edges: provenance edges still referencing forgotten_id, as returned by the provenance resolver (which may bypass retention filters). """ if not incident_edges: return # no dependents; divergence cannot manifest if required_attestation: if tombstone is None: raise ProvenanceRetentionDivergence( f"forget({forgotten_id}) left {len(incident_edges)} live provenance " f"edges but wrote no tombstone; dependents cannot verify derivation" ) if "payload_hash" not in tombstone or "forgotten_by" not in tombstone: raise ProvenanceRetentionDivergence( f"tombstone for {forgotten_id} lacks payload_hash/forgotten_by; " f"{len(incident_edges)} edges are unattested and must be treated as corrupt" ) # Detect the specific hazard: a dependent whose retention horizon exceeds # the forgotten ancestor's, with no provenance_hold recorded on the ancestor. for edge in incident_edges: if edge.get("child_retention_horizon_days", 0) > edge.get( "parent_retention_horizon_days", 0 ) and not edge.get("parent_provenance_hold", False): raise ProvenanceRetentionDivergence( f"edge {edge.get('child_id')}->{forgotten_id}: child horizon " f"{edge['child_retention_horizon_days']}d exceeds parent " f"{edge['parent_retention_horizon_days']}d with no provenance_hold; " f"write should have been rejected at creation" ) ``` Run this guard in the forget path, not the read path. By the time a `read` encounters the dangling edge, the forgetting principal is no longer in scope and the attestation can no longer be produced.