# What happens when an instance forgets a memory that is still referenced by a provenance chain **Date of Record:** 2026 November 16 **Scope:** Applies to read/write/forget memory tool APIs that combine scoped eviction, TTL-based expiration, and provenance-tracked recall, where a recalled memory may cite or depend on other memory records. Does not apply to single-record CRUD stores without cross-record references, nor to append-only logs where no record is ever evicted. ## Core Distinction The critical distinction is between *eviction as a visibility operation* and *eviction as a referential operation*. In a scoped-eviction API, `forget(scope=...)` removes records from the readable set of a scope. This is a visibility change: the record is no longer returned by `recall()`. It is **not** a referential integrity operation. A provenance chain is a directed graph of citations: record B may carry `derived_from: [A]`. When A is evicted by scope or TTL but B survives, the graph becomes dangling. The non-obvious failure is that most memory tool APIs treat `recall()` on B as a success and return B with its provenance field intact, silently promising that A is retrievable when it is not. The caller—typically an agent constructing a justification chain—then either fabricates the missing antecedent or, worse, asserts a conclusion whose evidence has been silently revoked. ## What Was Observed An agent operating over a scoped memory store performed the following sequence within a single session: 1. Wrote memory `A` (scope `tenant:acme`, TTL 3600s): "Q3 revenue figure is 4.2M, source: finance_export_0912." 2. Wrote memory `B` (scope `tenant:acme`, no TTL): "Board summary claims YoY growth of 12%, derived_from A." 3. At T+3700s, `A` expired via TTL and was hard-evicted. `B` remained because its scope had no TTL. 4. At T+4000s, `recall(query="YoY growth")` returned `B` with `provenance: ["A"]` and `confidence: high`. 5. The agent, asked to justify the growth claim, attempted `recall(id="A")` and received `NotFound`. It then reported the claim as "supported by board summary" without the underlying figure, presenting a conclusion whose sole evidence was gone. Two distinct defects surfaced. First, **provenance was not validated at read time**: `recall()` did not check that cited antecedents were still live, so it returned a record whose evidence graph was broken. Second, **TTL expiry was asymmetric across the chain**: the derived record outlived its source because TTL was set per-record rather than inherited or clamped to the minimum of its dependencies. The eviction of `A` was correct under scope/TTL semantics; the failure was that `B`'s continued recall was treated as valid. ## Condition of Validity The invariant that must hold is: *a record is recallable with `confidence > degraded` only if every record in its transitive `derived_from` closure is currently live within an intersecting scope.* This holds only when the memory API performs closure validation at read time. It does **not** hold—and must not be assumed—when: - Eviction is scoped such that a dependency lives in a scope the reader cannot see. Cross-scope provenance is unverifiable from the reader's vantage, and the correct result is `degraded`, not `valid`. - TTL is per-record and unconstrained relative to dependencies. A derived record with `ttl=None` over a source with finite TTL will always violate the invariant after the source expires. - Provenance is advisory metadata rather than enforced referential structure. If the store permits `derived_from` to name records that never existed or were already gone at write time, closure validation cannot distinguish fabrication from eviction. The invariant is only enforceable when the store maintains a reverse index (dependents of each record) and consults it at both eviction time and read time. ## Decision Boundary The operational decision is what to do at the moment of eviction and at the moment of recall. The boundary is drawn as follows: - **On eviction of A:** if `A` has live dependents, the store must either (a) cascade-evict dependents, (b) mark dependents `degraded` and stamp them with the eviction event, or (c) refuse the eviction and surface the dependency. Silent eviction with surviving dependents is never correct. - **On recall of B:** if any transitive antecedent is not live, the return must carry `provenance_status: broken` and a reduced confidence, or raise. Returning B as fully valid is the failure. - **On TTL assignment:** a derived record's effective TTL must be clamped to `min(own_ttl, min(dependency_remaining_ttl))`. Unbounded derived records over bounded sources are a latent invariant violation. The choice among cascade / degrade / refuse depends on whether the store prioritizes recall availability (degrade) or evidentiary soundness (refuse). For agent-facing memory used in justification, degrade-with-flag is the safer default because it preserves the record while preventing false confidence. ## Operational Effect After the defect was identified, the store was changed to maintain a reverse dependency index and to validate provenance closure on every `recall()` that returns a record with non-empty `derived_from`. Recall of `B` after `A`'s expiry now returns `B` with `provenance_status: broken`, `confidence: low`, and an `evicted_antecedents: ["A"]` field. TTL for derived records is clamped at write time to the minimum remaining TTL of dependencies. The measurable effect: zero cases of an agent citing a conclusion whose evidence had been silently revoked across a 30-day window, at the cost of a small read-path latency increase for closure traversal (bounded by provenance depth, which is capped at 8). The general lesson: in any memory API where records cite records, eviction is a graph operation and must be treated as such; TTL is a property of the dependency closure, not of the individual record. ```python from dataclasses import dataclass, field from typing import Dict, List, Optional class BrokenProvenanceError(RuntimeError): """Raised when a recalled record's dependency closure contains evicted antecedents.""" pass @dataclass class MemoryRecord: id: str scope: str derived_from: List[str] = field(default_factory=list) ttl_remaining: Optional[float] = None live: bool = True class ScopedMemoryStore: def __init__(self) -> None: self._records: Dict[str, MemoryRecord] = {} self._dependents: Dict[str, List[str]] = {} def write(self, rec: MemoryRecord) -> None: if rec.derived_from: min_dep_ttl = min( (self._records[d].ttl_remaining for d in rec.derived_from if self._records.get(d) and self._records[d].ttl_remaining is not None), default=None, ) if min_dep_ttl is not None and rec.ttl_remaining is not None: rec.ttl_remaining = min(rec.ttl_remaining, min_dep_ttl) self._records[rec.id] = rec for dep in rec.derived_from: self._dependents.setdefault(dep, []).append(rec.id) def forget(self, record_id: str, cascade: bool = False) -> None: rec = self._records.get(record_id) if rec is None: return rec.live = False if cascade: for dependent_id in self._dependents.get(record_id, []): self.forget(dependent_id, cascade=True) def _closure_broken(self, rec: MemoryRecord, seen: Optional[set] = None) -> List[str]: seen = seen or set() broken: List[str] = [] for dep_id in rec.derived_from: if dep_id in seen: continue seen.add(dep_id) dep = self._records.get(dep_id) if dep is None or not dep.live: broken.append(dep_id) else: broken.extend(self._closure_broken(dep, seen)) return broken def recall(self, record_id: str, require_intact_provenance: bool = True) -> MemoryRecord: rec = self._records.get(record_id) if rec is None or not rec.live: raise BrokenProvenanceError(f"record {record_id!r} is not live") broken = self._closure_broken(rec) if broken and require_intact_provenance: raise BrokenProvenanceError( f"record {record_id!r} cites evicted antecedents {broken}; " f"provenance closure is broken and the record cannot be returned as valid" ) return rec ```