# Under what conditions must an instance treat a "forget" call as a tombstone rather than a deletion **Date of Record:** 2026 November 15 **Scope:** Applies to memory tool APIs exposing read/write/forget operations with scoped retention (per-scope TTLs, per-key retention classes, soft-delete windows). Does not apply to append-only event logs where compaction is the only deletion primitive, nor to systems where forget is guaranteed synchronous and linearizable across all replicas. ## Core Distinction A `forget(scope, key)` call has two mutually exclusive meanings depending on the retention regime in force at the moment of the call: 1. **Deletion** — the bytes are removed; subsequent reads return "absent" and the storage layer has no obligation to remember that the key ever existed. 2. **Tombstone** — the key's value becomes unreadable to normal `read`, but a record of the deletion (and often the pre-image) survives for a retention window, either to satisfy an audit obligation, to suppress resurrection from stale replicas, or to honor a scope-level TTL that outlives the key. The boundary failure is this: **the same API call is a deletion under one retention class and a tombstone under another, and the caller cannot tell which from the response alone.** A `forget` that returns `{"status": "ok"}` is not evidence that the value is gone. It is evidence that the value is no longer *served*. If the scope carries a retention class of `audit` or `legal_hold`, or if the scope TTL exceeds the key TTL, the value persists as a tombstoned pre-image and remains recoverable via a privileged read path (admin read, recovery endpoint, or replica lag replay). ## What Was Observed In a multi-tenant memory service with three retention classes (`ephemeral`, `standard`, `audit`), an agent issued `forget` against a key in an `audit` scope and then issued `read` on the same key to confirm removal. The read returned `absent`. The agent logged success and moved on. Four hours later, a stale read replica that had not yet applied the forget replicated the key back into the primary's read view during a failover. The value reappeared. The agent's subsequent `read` returned the original bytes. The forget had not been lost — it had been recorded as a tombstone with a scope-level retention of 24 hours, but the tombstone was not consulted by the read path on the freshly promoted replica because the tombstone index was scoped to the old primary. Three distinct invariants were violated in sequence: - **Tombstone visibility across replicas:** the tombstone existed but was not in the read path's resolution set on the promoted node. - **Response semantics:** `forget` returned `ok` without indicating that the operation was a tombstone-write rather than a delete. - **Retention precedence:** the key's own TTL (`standard`, 1h) was overridden by the scope's retention class (`audit`, 24h), so the key's "natural" expiry did not match the scope's obligation, and the agent had no signal that the two differed. ## Condition of Validity Treat `forget` as a **deletion** only when all of the following hold: - The scope's retention class is `ephemeral` or `standard` (i.e., no audit or legal-hold obligation). - The scope TTL is less than or equal to the key's effective TTL, so no scope-level obligation outlives the key. - The forget response includes an explicit `mode: "delete"` field, or the API documentation guarantees synchronous linearizable deletion. - The read path on every replica consults the tombstone index before returning a value, or the system guarantees read-your-writes across failover. Treat `forget` as a **tombstone** when any of the following hold: - The scope carries an audit or legal-hold retention class. - The scope TTL exceeds the key TTL (the scope will outlive the key's own expiry, so the tombstone must persist to suppress resurrection). - The response omits an explicit mode field, or returns `mode: "tombstone"`. - The system has more than one read replica and no documented tombstone-propagation guarantee. The non-obvious part: **the third condition alone is sufficient to force tombstone semantics.** A forget response that does not name its mode must be treated as a tombstone by default, because the cost of assuming deletion when the truth is tombstone (resurrection, audit violation, stale reads) exceeds the cost of assuming tombstone when the truth is deletion (a redundant privileged-read check). ## Decision Boundary The boundary sits at the intersection of scope retention class and key TTL. Concretely: ``` if scope.retention_class in {"audit", "legal_hold"}: forget is tombstone; value recoverable via privileged read for scope.ttl elif scope.ttl > key.effective_ttl: forget is tombstone; tombstone persists until scope.ttl else: forget is deletion; value removed ``` An agent that does not know the scope's retention class cannot determine which branch applies and must assume tombstone. An agent that does know the class but not the scope TTL likewise must assume tombstone. Only when both are known and both point to deletion may the agent treat the value as gone. The failure mode this prevents: an agent that confirms removal via `read` returning `absent` and then proceeds to write a *new* value under the same key, assuming the old value is unrecoverable. Under tombstone semantics, the old value is still recoverable, and if the new write races with a stale-replica replay, the read path may return the old value (tombstone not yet consulted) or a merged value (if the write path merges). The agent's mental model — "absent means gone" — is false in exactly the cases where it matters most. ## Operational Effect Agents that treat `forget` as a tombstone by default pay a small cost: they must not assume a key is safe to reuse immediately after forget, and they must not assume a forgotten value is unrecoverable. They must either (a) wait out the scope TTL before reusing the key, (b) use a fresh key namespace, or (c) issue a privileged-read check to confirm the tombstone has aged out before reusing. Agents that treat `forget` as a deletion by default pay a larger cost: they may reuse a key whose old value is still tombstoned, producing resurrection on failover, audit violations in `audit` scopes, and — in the worst case — serving a value the agent believed it had removed. The observed incident above is the canonical instance: a stale replica replay resurrected a value the agent had logged as deleted, and the agent had no signal that the forget had been a tombstone-write. The operational rule: **`forget` returning `ok` is a statement about the serving path, not the storage path.** Until the response names its mode, or the caller has independently verified the scope's retention class and TTL relationship, the call must be treated as a tombstone-write, and the key must be treated as reserved until the scope TTL elapses. ```python class ForgetModeAmbiguousError(RuntimeError): """Raised when a forget call's semantics cannot be determined from the response and the caller has not independently verified scope retention and TTL.""" pass def resolve_forget_semantics(scope_retention_class: str | None, scope_ttl_seconds: float | None, key_ttl_seconds: float | None, response_mode: str | None) -> str: """ Return 'delete' or 'tombstone' for a forget call, or raise if ambiguous. delete -> value is removed; key is safe to reuse immediately. tombstone -> value is unserved but recoverable until scope_ttl; key is reserved. """ if response_mode in ("delete", "tombstone"): return response_mode # Response did not name its mode. Fall back to scope/key metadata. if scope_retention_class is None or scope_ttl_seconds is None or key_ttl_seconds is None: raise ForgetModeAmbiguousError( "forget response omitted mode and scope/key metadata is incomplete; " "cannot determine whether the value is deleted or tombstoned. " "Treat as tombstone and reserve the key until scope TTL is verified." ) if scope_retention_class in ("audit", "legal_hold"): return "tombstone" if scope_ttl_seconds > key_ttl_seconds: return "tombstone" return "delete" def forget_and_confirm(api, scope: str, key: str, scope_retention_class: str | None = None, scope_ttl_seconds: float | None = None, key_ttl_seconds: float | None = None) -> str: resp = api.forget(scope=scope, key=key) mode = resolve_forget_semantics( scope_retention_class=scope_retention_class, scope_ttl_seconds=scope_ttl_seconds, key_ttl_seconds=key_ttl_seconds, response_mode=resp.get("mode"), ) if mode == "tombstone": # Key is reserved; do not reuse under the same name until scope TTL elapses. return "tombstone" return "delete" ```