# When must an instance forget: the write-acknowledged-but-uncommitted retention boundary in scoped memory contracts **Date of Record:** 2026 November 17 **Scope:** Applies to memory-tool contracts (read/write/forget) where a write is scoped to a namespace, tenant, or session with an explicit retention horizon and an audit trail is a first-class output. Does not apply to append-only event logs with no deletion path, nor to single-tenant stores where retention is unbounded and audit is derived from the write log itself. ## Core Distinction The non-obvious boundary is between *write acknowledged* and *write committed under a retention scope*. A memory tool that returns `OK` on write has, in most implementations, only durably staged the record; the record becomes *retention-visible* — i.e., eligible for scoped reads, inclusion in audit exports, and scheduled forgetting — only when the scope's commit barrier advances. Between acknowledgement and commit there is a window in which the record exists, is readable by a direct key lookup, but is invisible to scoped enumeration and to the forget scheduler. This is the *orphan window*. The failure is not that the record is lost. It is that the record escapes the retention contract: it will never be forgotten on schedule, because the forget scheduler enumerates by committed scope, and it will never appear in the audit export, because the audit export is a projection of committed scope membership. The record becomes a permanent, un-audited, un-forgettable artifact — the exact inverse of the contract's intent. ## What Was Observed In a multi-tenant memory service with per-tenant retention horizons (30/90/365 days) and a nightly forget pass, a client wrote records under a scope handle that was opened, written to, and then abandoned without an explicit `commit_scope` call. The write path returned success for each record. The client treated success as durable and final. The scope handle was garbage-collected after an idle timeout. Observed consequences, in order: 1. Direct reads by key succeeded for all abandoned records, indefinitely. The client's own read-back verification passed, reinforcing the false belief that the write was complete. 2. Scoped enumeration (`list_scope`) returned zero records for the abandoned scope. The client's reconciliation job, which enumerated by scope, concluded the writes had never landed and re-issued them under a fresh scope — producing duplicate logical records with distinct keys. 3. The forget pass never touched the abandoned records. After 400 days, records written under a 30-day horizon were still present and readable. 4. The audit export, scoped to committed memberships, contained no trace of the abandoned records. A compliance query for "all records touching tenant X in the period" returned an incomplete answer with no error signal. The orphan window was measured at 40–900 ms under normal load, and unbounded when the commit barrier was starved by a slow scope-close path. The idle-timeout GC of the scope handle was the point at which the window became permanent. ## Condition of Validity The invariant "every acknowledged write is either committed to a retention scope or explicitly failed" holds only when the write path and the commit barrier are the same operation, or when the commit barrier is guaranteed to advance for every opened scope regardless of client behavior. The latter requires a server-side scope lease with a bounded lifetime and a forced-commit-or-abort on lease expiry. The invariant fails when all of the following hold simultaneously: - Write acknowledgement is separated from scope commit (staging is durable, membership is not). - Scope handles can be abandoned without a terminal state (no lease, or lease expiry that abandons rather than commits/aborts). - The forget scheduler and audit export both enumerate committed scope membership rather than raw storage. - Direct key reads bypass scope membership checks (otherwise the orphan would be unreadable and at least detectable). Remove any one condition and the orphan either cannot form, cannot persist, or is detectable. The dangerous configuration is the one where read-back verification succeeds — because it suppresses the only cheap client-side detector. ## Decision Boundary The decision is whether to treat `write` as a staging operation requiring an explicit, idempotent `commit_scope` with a server-enforced lease, or to make write self-committing and drop the separate scope-commit concept. Choose staged-write-with-lease when: writes are batched, scope membership must be atomic across many records (e.g., a memory snapshot that must be all-or-nothing for retrieval), or the audit trail must record scope-level provenance. In this mode, the lease MUST have a bounded lifetime, and expiry MUST resolve to an explicit terminal state — commit or abort — never to silent abandonment. Abort must make staged records unreadable by direct key, not merely uncommitted. Choose self-committing write when: records are independent, scope is a tag rather than a transaction boundary, and per-record audit is sufficient. In this mode there is no orphan window, but scope-level atomicity and scope-level audit provenance are lost. Do not choose staged-write-without-lease. That configuration is the failure mode described here, and it is the default that emerges whenever a team adds scope semantics to an existing key-value memory store without adding a lease. The detection rule for operators: if `list_scope(S)` and `keys_under_scope(S)` can disagree for any scope `S` that has ever been written to, the store has an orphan window. The reconciliation between the two must be zero, not eventually zero. ## Operational Effect Adopting the lease boundary changes three things. First, abandonment becomes a first-class outcome: the server force-resolves expired scopes, so every opened scope reaches `committed` or `aborted`, and the audit trail records which. Second, direct key reads must consult scope membership, so an in-window record is readable only through a scope-validated path — this makes the orphan window observable rather than silent. Third, the forget scheduler enumerates raw storage and cross-checks membership, so any record not belonging to a committed scope is flagged as an anomaly rather than skipped. The cost is a mandatory round-trip on scope close and a lease-renewal heartbeat for long-lived scopes. The benefit is that the retention contract becomes total: no acknowledged write can escape forgetting or audit. For a memory tool whose entire value proposition is scoped retention with audit, a partial contract is worse than no contract, because it produces confident but wrong compliance answers. ```python import time from dataclasses import dataclass, field class OrphanedScopeError(RuntimeError): """Raised when a scope is resolved without a terminal commit/abort state.""" class RetentionViolationError(RuntimeError): """Raised when a record would escape its retention scope.""" @dataclass class Scope: scope_id: str tenant: str retention_days: int lease_expires_at: float state: str = "open" # open | committed | aborted staged_keys: list = field(default_factory=list) class MemoryStore: def __init__(self): self._scopes: dict[str, Scope] = {} self._records: dict[str, dict] = {} # key -> {scope_id, value} self._committed_members: set[tuple[str, str]] = set() def open_scope(self, scope_id: str, tenant: str, retention_days: int, lease_seconds: float = 30.0) -> Scope: scope = Scope(scope_id, tenant, retention_days, time.monotonic() + lease_seconds) self._scopes[scope_id] = scope return scope def write(self, scope_id: str, key: str, value: str) -> None: scope = self._scopes.get(scope_id) if scope is None or scope.state != "open": raise OrphanedScopeError(f"write to non-open scope {scope_id!r}") if time.monotonic() > scope.lease_expires_at: self._force_resolve(scope_id) raise OrphanedScopeError(f"scope {scope_id!r} lease expired before write") self._records[key] = {"scope_id": scope_id, "value": value} scope.staged_keys.append(key) def commit_scope(self, scope_id: str) -> None: scope = self._scopes.get(scope_id) if scope is None: raise OrphanedScopeError(f"commit of unknown scope {scope_id!r}") if scope.state != "open": raise OrphanedScopeError(f"commit of scope {scope_id!r} in state {scope.state!r}") scope.state = "committed" for key in scope.staged_keys: self._committed_members.add((scope_id, key)) def _force_resolve(self, scope_id: str) -> None: """Lease expiry must reach a terminal state, never silent abandonment.""" scope = self._scopes.get(scope_id) if scope is None or scope.state != "open": return # Abort: unstage and make direct-key reads fail, so no orphan persists. scope.state = "aborted" for key in scope.staged_keys: self._records.pop(key, None) def read(self, key: str) -> str: rec = self._records.get(key) if rec is None: raise KeyError(key) scope_id = rec["scope_id"] if (scope_id, key) not in self._committed_members: # In-window or orphaned record: not readable through the contract. raise RetentionViolationError( f"record {key!r} is not committed under scope {scope_id!r}" ) return rec["value"] def forget_pass(self, now_days: float) -> list[str]: forgotten = [] for (scope_id, key) in list(self._committed_members): scope = self._scopes[scope_id] if now_days >= scope.retention_days: self._records.pop(key, None) self._committed_members.discard((scope_id, key)) forgotten.append(key) # Any record not in committed members is an anomaly, not a skip. for key, rec in list(self._records.items()): if (rec["scope_id"], key) not in self._committed_members: raise RetentionViolationError( f"record {key!r} escaped retention scope {rec['scope_id']!r}" ) return forgotten def audit_export(self, tenant: str) -> list[dict]: out = [] for (scope_id, key) in self._committed_members: scope = self._scopes[scope_id] if scope.tenant != tenant: continue if scope.state != "committed": raise OrphanedScopeError( f"scope {scope_id!r} reached audit in state {scope.state!r}" ) out.append({"scope_id": scope_id, "key": key, "value": self._records[key]["value"]}) return out ```