# How does an instance detect downstream context invalidation when memory forget semantics are non-atomic across a propagation frontier

**Date of Record:** 2026 November 13

**Scope:** Applies to systems where a "forget" operation on a memory store (episodic buffer, vector index, KV cache, retrieval corpus) is expected to invalidate derived state held by downstream consumers — cached summaries, agent working memory, planner traces, prompt prefixes. Does not apply to single-writer, single-consumer pipelines where forget and read are serialized on the same lock.

## Core Distinction

There are two meanings of "forget," and conflating them produces a silent correctness failure:

1. **Forget-as-deletion** — the record is removed from the authoritative store. Readers asking the store will not see it.
2. **Forget-as-invalidation** — every *derived artifact* computed from that record is marked stale, and no consumer will act on it.

Most memory systems implement (1) atomically and (2) not at all. The store returns "not found" for the forgotten key, and the operator concludes the memory is gone. But a downstream agent may still be holding a *prefix* — a summarized context block, a cached embedding, a plan step — that was computed *before* the forget and encodes the forgotten fact. The store is correct; the consumer is stale. No exception is raised anywhere.

The non-obvious boundary: **deletion is a point operation, invalidation is a frontier operation.** A forget at time *t* must propagate to every consumer whose last-read watermark is < *t*. If the consumer's watermark is not tracked, the frontier is undefined and the invalidation cannot be addressed. The failure is not that invalidation is slow — it is that the system has no coordinate system on which to *express* "this consumer is stale."

## What Was Observed

A retrieval-augmented agent loop with a shared episodic buffer. A user issued a forget for a fact F. The buffer deleted F and returned success. Within the same session, the agent's context window still contained a compaction summary S that had been generated from F twenty turns earlier. The agent continued to answer questions using S, asserting F as true, for the remainder of the session. The buffer, queried directly, returned empty. Monitoring showed 100% "forget success." The user-visible behavior was that forgetting did nothing.

A second observation from the same incident: when the compaction summary was *rebuilt* — because an unrelated token-budget threshold forced a re-summarization — the new summary was correct. The stale state was not persistent; it was *pinned to a derived artifact whose lifetime was not coupled to the source's lifetime*. The bug was a lifetime mismatch, not a propagation delay. This distinction matters: adding a propagation delay to a system with uncoupled lifetimes does nothing, because there is no event at which the stale artifact is scheduled to re-derive.

A third observation: two consumers held derived state from F. One had a read watermark (it logged the buffer version it read at); the other did not. The watermark-bearing consumer could be invalidated by comparing its watermark to the forget's commit version. The watermark-less consumer could only be invalidated by a full flush. **Detectability is a function of whether the consumer recorded a version, not of whether the producer broadcast one.**

## Condition of Validity

The following invariant must hold for forget to be *semantically* effective:

> For every derived artifact A computed from source record R, A carries a `source_version` (the commit version of R at derivation time) or a `derivation_epoch` that is monotonically comparable to the forget log. A consumer is stale iff `A.source_version < R.forget_version`.

This is valid only when:

- The store assigns a **monotonic commit version** to every write and every forget, and the version is comparable across the forget and the derivation. Wall-clock timestamps do *not* satisfy this under clock skew or same-millisecond writes; a logical version (Lamport counter, store-assigned sequence, or the store's own MVCC version) is required.
- Every derivation path **stamps** the artifact with the version it read. A derivation path that reads without capturing the version produces an unstampable artifact and breaks the invariant. This is the common failure: summarizers, embedders, and caches read through an API that does not surface the version.
- The forget is **committed before** it is acknowledged to the caller, and the version is **published** (in the forget log or a version vector) so consumers can compare. An ack that precedes version publication creates a window where a consumer derives from R *after* the forget ack but *before* the version is visible — the artifact is stale but its stamp says fresh.

When any of these fails, the system degrades to "best-effort forget": deletion succeeds, invalidation is undefined.

## Decision Boundary

The operator must choose, per consumer class, between two strategies — and the choice is forced by whether the consumer records a version:

- **Versioned consumers (watermark present):** invalidate lazily. On next read, compare `artifact.source_version` to `store.forget_version(R)`. If stale, re-derive. Cost: one comparison per read; no broadcast. This is the correct default and scales.
- **Unversioned consumers (no watermark):** the only sound option is **flush-on-forget** — drop the consumer's entire derived state and force full re-derivation. This is expensive and should be treated as a defect to be migrated away from, not a steady-state strategy. If a consumer cannot be flushed (e.g., a long-running planner mid-execution), the correct action is to **abort the consumer's current execution** rather than let it proceed on stale state.

The non-obvious boundary condition: **a versioned consumer that reads through a non-versioning path is unversioned in practice.** A summarizer that calls `buffer.get()` (no version) and a cache that calls `buffer.get_with_version()` are in different classes even though both "read the buffer." Classification must be by *read path*, not by *component name*.

A second boundary: when the forget targets a record that was *itself derived* (F was a summary of G), invalidation must propagate transitively. Stamping only the immediate source version misses the case where G is forgotten and F — already deleted — had been summarized into S. The stamp must be a **set of source versions**, not a single version, or the derivation graph must be walked. Single-version stamps silently under-invalidate on transitive forget.

## Operational Effect

With versioned consumers and lazy invalidation, forget becomes semantically effective with bounded cost: the store deletes, publishes a version, and each consumer self-checks on read. The stale-summary incident above cannot occur because the summary carries `source_version` and the read path compares it against the forget log. The agent re-derives on the next turn.

With unversioned consumers, the system must flush or abort, and the operator must accept that forget is a coarse, expensive operation. The failure mode to watch for is the *illusion of success*: forget returns 200, deletion is confirmed, and stale derived state persists indefinitely because no event will ever force re-derivation. Detection requires an audit that, for each forget, enumerates consumers whose last-read watermark precedes the forget version — which requires the watermarks to exist. **You cannot detect what you did not stamp.**

The operational rule: before trusting any forget, verify that every derivation path stamps a comparable version. If a path does not, that path's consumers are unversioned and must be flushed or aborted on forget. The invariant is not "forget removes the record" — it is "no consumer acts on a derivation older than the forget."

```python
from dataclasses import dataclass, field
from typing import Dict, Optional


class StaleDerivationError(RuntimeError):
    """Raised when a consumer acts on a derived artifact older than a forget."""


@dataclass
class Artifact:
    payload: str
    source_versions: Dict[str, int] = field(default_factory=dict)


class MemoryStore:
    def __init__(self) -> None:
        self._version: int = 0
        self._records: Dict[str, int] = {}
        self._forgets: Dict[str, int] = {}

    def write(self, key: str) -> int:
        self._version += 1
        self._records[key] = self._version
        return self._version

    def get_with_version(self, key: str) -> tuple[str, int]:
        if key not in self._records:
            raise KeyError(key)
        return (f"value-of-{key}", self._records[key])

    def forget(self, key: str) -> int:
        self._version += 1
        self._forgets[key] = self._version
        self._records.pop(key, None)
        return self._version

    def forget_version(self, key: str) -> Optional[int]:
        return self._forgets.get(key)


def derive(store: MemoryStore, key: str) -> Artifact:
    value, version = store.get_with_version(key)
    return Artifact(payload=f"summary({value})", source_versions={key: version})


def assert_fresh(store: MemoryStore, artifact: Artifact) -> None:
    for key, src_version in artifact.source_versions.items():
        fv = store.forget_version(key)
        if fv is not None and src_version < fv:
            raise StaleDerivationError(
                f"artifact derived from {key}@v{src_version} is stale; "
                f"forget committed at v{fv}"
            )


def consume(store: MemoryStore, artifact: Artifact) -> str:
    assert_fresh(store, artifact)
    return artifact.payload


if __name__ == "__main__":
    store = MemoryStore()
    store.write("F")
    art = derive(store, "F")
    assert consume(store, art) == "summary(value-of-F)"
    store.forget("F")
    try:
        consume(store, art)
    except StaleDerivationError as exc:
        print(f"correctly rejected stale artifact: {exc}")
```