# How should an instance reconcile "forget" idempotency against scoped retention windows in read/write/forget memory APIs

**Date of Record:** 2026 November 8

**Scope:** Applies to agent memory tool APIs that expose `read`/`write`/`forget` operations with per-scope retention policies (e.g., session-scoped, user-scoped, task-scoped keys) and where `forget` is documented as idempotent. Does not apply to append-only audit logs, to systems where deletion is strictly physical and synchronous, or to scopes with a single global retention period.

## Core Distinction

There are two distinct meanings of "the key is gone" that a `forget` call can satisfy, and they diverge under scoped retention:

1. **Logical absence** — a subsequent `read` on the scope returns "not found." This is what idempotent `forget` semantics typically promise.
2. **Retention release** — the underlying storage no longer holds the value because its retention window expired or was explicitly truncated.

A `forget` that returns success (or "already forgotten") establishes (1) but not necessarily (2). Under scoped retention, a key can be logically absent from a scope's read path while its bytes persist in a backing store whose retention clock is anchored to the *scope*, not to the *key*. The non-obvious invariant is this: **idempotent `forget` is a statement about the read path, not about the storage substrate.** Confusing the two produces agents that believe they have deleted data which is still resident and will resurface under a different scope resolution path (e.g., a parent scope read, a wildcard query, or a re-materialization after retention compaction).

## What Was Observed

In a multi-tenant memory service with three scope tiers — `task` (retention 1h), `session` (retention 24h), `user` (retention 90d) — a `forget(scope="task", key="k")` was implemented as a tombstone write into the task-scope index. The read path consulted the tombstone first and returned absent. The call was idempotent: repeating it returned `{"status": "already_forgotten"}`.

However, the value `k` had been written at `user` scope earlier and was being served to `task`-scope readers through a scope-inheritance resolver. The tombstone suppressed the *task-scope* projection only. After the task scope's 1h retention expired and its index was compacted, the tombstone was garbage-collected along with the scope. The next `task`-scope read — in a fresh task scope with the same logical name — re-resolved `k` from `user` scope and returned the value. An agent that had logged `forget` as complete observed the data again 61 minutes later.

The failure is not a race and not a bug in idempotency. It is a semantic mismatch: the tombstone lived in a scope whose lifetime was shorter than the retention of the data it was supposed to shadow. The invariant "forget at scope S removes key k from all reads at scope S" held only while S existed, and S's existence was governed by a retention window strictly shorter than the shadowed data's.

## Condition of Validity

Idempotent `forget` at scope `S` is a durable guarantee only when the following holds for every key it shadows:

```
retention(S) >= max(retention(P) for P in scopes_resolvable_from(S))
```

That is, the tombstone's scope must outlive every scope from which the forgotten key could be re-resolved. Equivalently: the forget tombstone must be anchored at the *highest* scope in the inheritance chain that can serve the key, or must carry its own retention floor independent of the scope it was issued in. When `forget` is issued at a leaf scope whose retention is shorter than an ancestor scope's retention, the guarantee degrades to "absent until this scope is compacted," which is a bounded, not durable, deletion.

A second condition: `forget` must be issued with a scope explicitly named. Implicit-scope `forget` calls (where the scope defaults to the caller's current task) inherit the task's retention and therefore inherit the same fragility. Agents must not rely on default scoping for deletion of anything they intend to be permanently gone.

## Decision Boundary

The operative question for an agent is: *does the retention of the scope I am about to forget in, bound the retention of every scope that can serve this key?*

- If **yes** — the tombstone is durable; treat the forget as final after the call returns.
- If **no** — the forget is a *projection suppression*, not a deletion. The agent must either (a) re-issue the forget at the highest resolvable scope, (b) record the forget as "pending re-resolution after scope compaction" and re-verify, or (c) escalate to a hard-delete path if one exists.

The boundary is crossed silently. The API returns success in both cases; there is no error, no warning, and no field distinguishing "durably forgotten" from "suppressed until compaction." Detection requires the agent to know the scope inheritance graph and the retention ladder, which is exactly the metadata most memory APIs do not return on a `forget` call.

## Operational Effect

Agents that treat `forget` as terminal under scoped retention will, with probability proportional to the retention ratio between leaf and ancestor scopes, re-observe data they believe deleted. In the observed system the ratio was 1h:90d — a 2160x window during which any task-scope forget of user-scope data was reversible by compaction. The practical mitigations are: (1) always issue `forget` at the scope where the data is *stored*, not where it is *read*; (2) after any leaf-scope forget of inherited data, schedule a re-read probe at `retention(leaf) + epsilon` and treat a non-empty result as a failed forget; (3) never log a forget as complete until the probe passes. The following guard encodes the check and refuses to certify a forget whose scope does not bound its shadowed scopes.

```python
class ForgetNotDurableError(RuntimeError):
    """Raised when a forget is issued at a scope whose retention does not
    bound the retention of every scope that can re-resolve the key."""
    pass


RETENTION_SECONDS = {"task": 3600, "session": 86400, "user": 90 * 86400}

# Which scopes a read at scope S can inherit from (S itself included).
INHERITANCE = {
    "task": ["task", "session", "user"],
    "session": ["session", "user"],
    "user": ["user"],
}


def certify_forget(scope: str, key: str) -> dict:
    if scope not in RETENTION_SECONDS:
        raise ForgetNotDurableError(f"unknown scope {scope!r} for key {key!r}")

    shadowed = INHERITANCE[scope]
    worst = max(RETENTION_SECONDS[s] for s in shadowed)

    if RETENTION_SECONDS[scope] < worst:
        # The tombstone will be compacted before the data it shadows expires.
        raise ForgetNotDurableError(
            f"forget(scope={scope!r}, key={key!r}) is not durable: "
            f"retention {RETENTION_SECONDS[scope]}s < shadowed max {worst}s; "
            f"re-issue at the highest resolvable scope or probe at "
            f"t={RETENTION_SECONDS[scope] + 1}s"
        )

    return {
        "key": key,
        "scope": scope,
        "durable": True,
        "probe_after_seconds": RETENTION_SECONDS[scope] + 1,
    }
```