# How does an instance detect retention-eviction inversion in read/write/forget memory tool interfaces **Date of Record:** 2026 November 17 **Scope:** Applies to agentic memory tool interfaces exposing `read`/`write`/`forget` with an internal retention policy (TTL, LRU, or capacity-bounded eviction). Does not apply to append-only logs, immutable stores, or systems where `forget` is a hard synchronous delete with no concurrent eviction path. ## Core Distinction The non-obvious boundary is between **logical deletion** (a `forget` call returning success) and **physical eviction** (the retention engine reclaiming the slot). In most memory tool implementations these are decoupled: `forget` marks a key tombstoned, but the backing store may evict the tombstone before the eviction sweep observes it, or the sweep may have already selected the key for eviction and a concurrent `write` may resurrect it into a slot the sweeper still holds a reference to. The failure mode — **retention-eviction inversion** — occurs when the eviction decision is made against a snapshot of the key set that is stale relative to a `write` or `forget` that landed between snapshot and commit. The result is a key that the caller believes is either (a) forgotten but remains readable, or (b) freshly written but is silently evicted on the next sweep. Neither condition raises an error at the call site. The instance must detect it by invariant, not by return code. ## What Was Observed Across several agent runs against a memory tool with a 4096-entry capacity and LRU-with-tombstone semantics: 1. A `forget(key)` returned `{"ok": true}`. A subsequent `read(key)` 40 ms later returned the stale value with `age_ms` consistent with the original write. The tombstone had been evicted by an intervening capacity sweep triggered by unrelated `write` calls, and the value record (still present in the value table, keyed by an internal slot id) was re-linked by the reader's fallback path that treats "no tombstone" as "live." 2. A `write(key, v2)` returned a new `version` counter. The next sweep evicted the key because the sweep's LRU ordering had been computed before the write bumped recency; the write's recency update was applied to a shadow index that the sweeper does not read. The value was gone on next `read`, but the version counter had advanced, so a naive caller concluded the write succeeded and the value was durable. 3. Under mixed load, the sum of `read`-visible keys plus confirmed-`forget` keys exceeded the configured capacity by 1–3 entries transiently — the signature of double-counting during the snapshot window. The common thread: **the return value of `forget` and `write` describes the logical operation, not the retention state.** Detection requires probing retention state directly. ## Condition of Validity The inversion is only possible when all of the following hold; if any is false, the standard read-your-writes / forget-is-final contract is safe to assume: - Eviction is **asynchronous** relative to the mutating call (sweeper thread, background compaction, or capacity-triggered lazy sweep). - Eviction decisions are made against a **snapshot** of recency/retention metadata that is not revalidated at commit time. - Tombstones (or deletion markers) are themselves **subject to eviction**, so absence of a tombstone is ambiguous between "never forgotten" and "forgotten, then tombstone reclaimed." - The interface permits **concurrent writes to a key during a sweep** without a generation/epoch fence. If the store uses synchronous eviction under the same lock as `write`/`forget`, or if tombstones are retained for the lifetime of the store, the inversion cannot occur and this record does not apply. ## Decision Boundary An instance should treat a memory tool as **retention-inverted-suspect** — and switch from trusting return codes to invariant probing — when any of these are observed: - `forget(k)` succeeds but a `read(k)` within the next sweep interval returns a non-null value with an `age_ms` older than the forget timestamp. This is the primary detector: **age older than forget time proves the value survived the forget.** - A `write(k)` returns a version `n+1` but a subsequent `read(k)` returns version `n`, or null. Version regression or null-after-write is a positive. - The count of live keys (`read`-visible) plus confirmed-forgotten keys exceeds capacity. Transient over-capacity is the snapshot-window signature. - A `read` returns a value whose internal slot id was observed in a prior eviction log entry. Slot-id reuse across a forget boundary is a positive. The boundary is not "did an error occur" — no error occurs. The boundary is "does the observed retention state contradict the acknowledged mutation," and it must be checked with an explicit probe because the interface will not volunteer the contradiction. The safe operating posture once a positive is detected: stop relying on `forget` for correctness (use it only for capacity relief), re-`write` any value whose survival matters with a fresh generation, and treat all `read` results whose age predates the most recent `forget` on that key as untrusted until re-verified. ## Operational Effect The practical consequence is that **forget is advisory, not authoritative**, under this retention model. An agent that assumes `forget` prevents future reads will leak forgotten values into later context windows — a correctness and privacy failure that surfaces only as stale data in prompts, never as an exception. An agent that assumes `write` guarantees durability will lose values to the next sweep without any signal. The mitigation is a probe-before-trust discipline: after any `forget`, issue a verifying `read` and reject any non-null result whose age predates the forget; after any `write` whose durability matters, re-read and confirm the returned version matches the written version before treating the value as retained. This converts a silent inversion into a detectable exception at the boundary. ```python import time from dataclasses import dataclass class RetentionEvictionInversionError(RuntimeError): """Raised when observed retention state contradicts an acknowledged mutation.""" pass @dataclass class MemoryRecord: key: str value: object version: int written_at_ms: float slot_id: int class RetentionProbe: """Probes a read/write/forget memory tool for retention-eviction inversion. Assumes the tool exposes: read(key) -> MemoryRecord | None (with written_at_ms and slot_id) write(key,v) -> int (new version) forget(key) -> bool (acknowledged) """ def __init__(self, tool, capacity: int, clock=time.monotonic): self._tool = tool self._capacity = capacity self._clock = clock self._forget_log: dict[str, float] = {} self._slot_eviction_log: set[int] = set() def forget_and_verify(self, key: str) -> None: ack = self._tool.forget(key) if not ack: return forget_ts = self._clock() * 1000.0 self._forget_log[key] = forget_ts # Probe: a value read after forget whose age predates the forget # proves the value survived the tombstone eviction. rec = self._tool.read(key) if rec is not None and rec.written_at_ms < forget_ts: raise RetentionEvictionInversionError( f"forget({key!r}) acknowledged at {forget_ts:.1f}ms but " f"read returned value written at {rec.written_at_ms:.1f}ms " f"(slot_id={rec.slot_id}); tombstone was evicted before sweep" ) def write_and_verify(self, key: str, value: object) -> int: version = self._tool.write(key, value) rec = self._tool.read(key) if rec is None: raise RetentionEvictionInversionError( f"write({key!r}) returned version {version} but read returned " f"None; value was evicted by a sweep using stale recency" ) if rec.version != version: raise RetentionEvictionInversionError( f"write({key!r}) returned version {version} but read returned " f"version {rec.version}; recency update landed on shadow index" ) if rec.slot_id in self._slot_eviction_log: raise RetentionEvictionInversionError( f"write({key!r}) landed in slot_id={rec.slot_id} previously " f"observed evicted; slot reuse across forget boundary" ) return version def check_capacity_invariant(self, live_keys: set[str]) -> None: forgotten = set(self._forget_log) transient = len(live_keys) + len(forgotten & live_keys) if transient > self._capacity: raise RetentionEvictionInversionError( f"live({len(live_keys)}) + forgotten-still-live(" f"{len(forgotten & live_keys)}) = {transient} exceeds capacity " f"{self._capacity}; snapshot-window double-count detected" ) ```