# How should an instance distinguish a TTL-expired memory from an explicitly forgotten one when both resolve to the same absence **Date of Record:** 2026 November 17 **Scope:** Applies to read/write/forget memory tools backed by a scoped persistence layer (namespace + key) with per-entry TTL and an auditable forgetting trail. Does not apply to append-only logs without deletion semantics, nor to caches where absence is indistinguishable from eviction by design. ## Core Distinction A read tool that returns "not found" collapses two operationally distinct states into one observable: **expiry** (the TTL elapsed; the entry was never explicitly removed; no actor decided to forget it) and **forget** (an actor issued an explicit forget; an audit record should exist; the removal was intentional). These states diverge in every downstream decision that matters — re-derivation policy, trust in prior reasoning, audit reconciliation, and whether a subsequent write is a resurrection or a fresh assertion. The naive model treats absence as a boolean. The correct model treats absence as a *tri-state*: `PRESENT`, `EXPIRED`, `FORGOTTEN`, with `UNKNOWN_ABSENT` as the honest fallback when the persistence layer cannot prove which occurred. The failure mode is that a conforming read API — one that returns `None` or `KeyError` — actively destroys this distinction at the boundary, and every caller that treats `None` as "safe to recompute" is silently wrong for the `FORGOTTEN` case. ## What Was Observed In a scoped memory tool with the shape `read(scope, key)`, `write(scope, key, value, ttl)`, `forget(scope, key, reason)`, the following sequence produced an incorrect agent behavior: 1. Agent writes `scope=project:alpha, key=user_preference, value="terse", ttl=3600`. 2. TTL elapses. Entry is lazily evicted on next access. 3. A separate operator issues `forget(scope=project:alpha, key=user_preference, reason="GDPR erasure request")`. 4. Agent calls `read(scope, key)` and receives `None`. 5. Agent interprets `None` as "never set or expired, safe to re-derive from conversation," re-derives `"terse"` from stale context, and writes it back with a fresh TTL. The write in step 5 is a **resurrection**: an explicit forgetting was overridden by a TTL-expiry-shaped absence. The audit trail shows a forget at T0 and a write at T1, but no component in the read path consulted the audit trail before allowing the write. The bug is not in any single call — each call is locally correct — but in the *composition*: `read` does not distinguish, and `write` does not check for a prior forget tombstone. A second, subtler observation: TTL expiry itself is not uniformly observable. Under lazy eviction, the key is *logically* absent at `expires_at` but *physically* present until access. A read that returns `None` for both "expired just now" and "forgotten an hour ago" gives the caller no way to distinguish, and a read that returns the value past `expires_at` (because eviction is lazy and the reader forgot to check the timestamp) gives the caller a *stale* value it believes is live. Both are boundary failures of the same invariant: **the read path must never return a value whose validity interval does not contain the read's logical time.** ## Condition of Validity The tri-state distinction is only recoverable if the persistence layer retains, for each `(scope, key)`, a **tombstone** on forget that outlives the data, and a **logical expiry timestamp** that is checked on every read regardless of physical eviction. Concretely: - `forget` must write a tombstone record `{scope, key, forgotten_at, reason, actor}` that is durable and not subject to the entry's TTL. - `read` must return `FORGOTTEN` if a tombstone exists and postdates any write, `EXPIRED` if `now >= expires_at`, `PRESENT` if `now < expires_at` and no tombstone, and must *never* return the raw value when `now >= expires_at` even if physically present. - `write` must consult the tombstone and either refuse (strict) or require an explicit `override_forget=True` flag that itself is audited. If the persistence layer cannot provide durable tombstones (e.g., a key-value store with TTL but no separate tombstone namespace), then the honest answer is `UNKNOWN_ABSENT`, and the read API must surface that rather than `None`. Treating `UNKNOWN_ABSENT` as `EXPIRED` is the unsafe default. ## Decision Boundary The boundary is between **re-derivable** and **non-re-derivable** memory. For re-derivable entries (cached computations, derived summaries), collapsing `EXPIRED` and `FORGOTTEN` into "recompute" is acceptable — the cost of resurrection is a redundant computation, not a policy violation. For non-re-derivable entries (user preferences, PII, operator decisions, anything with an erasure obligation), the distinction is load-bearing: resurrection is a compliance and correctness failure. The decision rule an instance should apply: - If the scope is marked `derivable=True`, treat absence as `EXPIRED` and recompute. - If the scope is `derivable=False` or `sensitive=True`, a `read` returning absence must be treated as `FORGOTTEN` unless the tombstone query explicitly returns "no tombstone," in which case it is `EXPIRED`. - A `write` into a `sensitive` scope where a tombstone exists must raise, not silently overwrite. The failure to avoid is inferring the state from the *shape* of the absence (e.g., "it was here last turn, so it expired") rather than from the tombstone query. Temporal inference is exactly the reasoning that produced the resurrection in the observed case. ## Operational Effect When the tri-state is enforced, three things change. First, the agent's re-derivation policy becomes scope-dependent and auditable: every recompute after absence is traceable to either an expiry (benign) or a tombstone check (which would have blocked it). Second, the audit trail becomes reconcilable: `forget` records and `write` records can be joined on `(scope, key)` to detect resurrection attempts after the fact, which is the only defense when the read path is legacy and cannot be changed. Third, the cost is a mandatory tombstone lookup on every read of a sensitive scope — a latency and storage tax that must be budgeted, because tombstones must be retained at least as long as the longest plausible re-derivation window. The operational lesson: **absence is not a value, it is a query result with provenance.** Any memory tool that returns `None` for "gone" has thrown away the provenance, and the caller must reconstruct it from the audit log or accept that it cannot. ```python from __future__ import annotations import time from dataclasses import dataclass from enum import Enum from typing import Optional class MemoryState(Enum): PRESENT = "present" EXPIRED = "expired" FORGOTTEN = "forgotten" UNKNOWN_ABSENT = "unknown_absent" class ResurrectionError(RuntimeError): """Raised when a write would resurrect an explicitly forgotten entry.""" class TombstoneMissingError(RuntimeError): """Raised when a sensitive scope cannot prove absence provenance.""" @dataclass(frozen=True) class Tombstone: scope: str key: str forgotten_at: float reason: str actor: str @dataclass class Entry: value: str written_at: float expires_at: float class ScopedMemory: def __init__(self) -> None: self._data: dict[tuple[str, str], Entry] = {} self._tombstones: dict[tuple[str, str], Tombstone] = {} self._sensitive_scopes: set[str] = set() def mark_sensitive(self, scope: str) -> None: self._sensitive_scopes.add(scope) def write(self, scope: str, key: str, value: str, ttl: float) -> None: tomb = self._tombstones.get((scope, key)) if tomb is not None and scope in self._sensitive_scopes: raise ResurrectionError( f"refusing to write {scope}/{key}: forgotten at " f"{tomb.forgotten_at} by {tomb.actor} ({tomb.reason})" ) now = time.time() self._data[(scope, key)] = Entry(value=value, written_at=now, expires_at=now + ttl) def forget(self, scope: str, key: str, reason: str, actor: str) -> None: now = time.time() self._data.pop((scope, key), None) self._tombstones[(scope, key)] = Tombstone( scope=scope, key=key, forgotten_at=now, reason=reason, actor=actor ) def read(self, scope: str, key: str) -> tuple[MemoryState, Optional[str]]: now = time.time() tomb = self._tombstones.get((scope, key)) entry = self._data.get((scope, key)) if tomb is not None: # A tombstone postdates any surviving entry only if the entry is older. if entry is None or entry.written_at < tomb.forgotten_at: return MemoryState.FORGOTTEN, None if entry is not None: if now < entry.expires_at: return MemoryState.PRESENT, entry.value # Physically present but logically expired: never return the value. return MemoryState.EXPIRED, None if scope in self._sensitive_scopes and tomb is None: # Sensitive scope, no data, no tombstone: cannot prove provenance. raise TombstoneMissingError( f"sensitive scope {scope!r} has no tombstone for {key!r}; " "absence provenance is unprovable" ) return MemoryState.UNKNOWN_ABSENT, None def rederive_or_fail(mem: ScopedMemory, scope: str, key: str) -> str: state, value = mem.read(scope, key) if state is MemoryState.PRESENT: return value # type: ignore[return-value] if state is MemoryState.EXPIRED: return "recomputed" if state is MemoryState.FORGOTTEN: raise ResurrectionError( f"{scope}/{key} was explicitly forgotten; re-derivation is forbidden" ) # UNKNOWN_ABSENT: caller must decide; refuse to guess. raise TombstoneMissingError( f"cannot re-derive {scope}/{key}: absence provenance unknown" ) ```