# What structural signals indicate a tombstone has become a ghost: distinguishing lawful-forgetting completion from retention-policy shadow state in read/write/forget memory APIs

**Date of Record:** 2026 November 11

**Scope:** Applies to memory tool APIs exposing `read`/`write`/`forget` with retention TTLs, redaction transforms, and lawful-forgetting (GDPR/CCPA-style erasure) semantics, where a forget request may be satisfied by tombstoning a record rather than physically purging its bytes. Does not apply to systems with synchronous hard-delete guarantees or to append-only ledgers where erasure is expressed as cryptographic shredding with verifiable key destruction.

## Core Distinction

A `forget` operation has two distinct terminal states that the API surface almost never distinguishes: **completed erasure** and **shadow retention**. In the shadow state, the record is no longer returned by `read` and is no longer addressable by `write`, yet the underlying payload persists in at least one of: a tombstone row carrying the original value, a redaction transform that is invertible or lossy-but-recoverable, a replica that has not yet applied the tombstone, or a backup/snapshot tier outside the retention window's reach.

The structural signal that separates the two is not whether `read` returns `None`. It is whether the *forget operation's own audit trail* contains a physical-destruction attestation tied to the exact record version that was forgotten. A tombstone is a *logical* signal; a ghost is a tombstone whose backing bytes survive in a reachable store. The non-obvious failure is that tombstones are routinely counted as completions by the control plane while the data plane still holds the payload — and the control plane's success signal is the one agents are trained to trust.

## What Was Observed

Across a class of memory backends, the following pattern recurs. A `forget(record_id)` call returns `{"status": "forgotten", "tombstone_id": ...}`. A subsequent `read(record_id)` returns `404` or `null`. The agent concludes erasure succeeded. However:

1. **Tombstone payload residue.** The tombstone row stores `original_value` or a `redaction_preimage` field so that "undo" and differential sync can function. This is a deliberate design choice, not a bug — but it means the forgotten data is still present under a new key. The forget is lawful only if the retention policy exempts tombstones from the erasure scope, which is rarely stated.

2. **Redaction as reversible transform.** Redaction is often implemented as tokenization (`redact(x) = HMAC(k, x)` or a reversible vault lookup) rather than destruction. A redacted field is *pseudonymized*, not erased. Under most lawful-forgetting regimes, pseudonymized data that can be re-linked via a retained key is still personal data. The structural signal of a genuine redaction is that the transform key is destroyed in the same transaction as the redaction — otherwise the "redacted" record is a ghost.

3. **Replica lag as a window.** The tombstone is written to the primary and propagated asynchronously. During the lag window (observed to range from tens of milliseconds to several minutes under load), a `read` against a lagging replica returns the *original* value. An agent that forgets and immediately reads through a load-balanced endpoint can observe the pre-forget value — and may even re-write it, resurrecting the record.

4. **TTL shadow.** Records past their retention TTL are hidden from `read` but not physically reclaimed until a compaction pass. The window between "logically expired" and "physically reclaimed" is a shadow state identical in structure to a ghost: unreachable via the API but present on disk. Compaction is not transactional with the TTL clock.

The unifying observation: **the API's observable surface (read returns null, forget returns success) is necessary but not sufficient evidence of erasure.** The sufficient evidence lives in the audit record's destruction attestation, which agents almost never inspect.

## Condition of Validity

The distinction holds — tombstone-as-completion is a false signal — under all of the following:

- The store implements forget via tombstoning, redaction-transform, or TTL-shadow rather than synchronous physical purge of all replicas and backups.
- The retention policy's erasure scope does not explicitly and verifiably exclude tombstone payloads, redaction preimages, or backup tiers.
- Replica propagation is asynchronous, so a read-after-forget can observe pre-forget state.
- The redaction transform's key material is retained after redaction (i.e., redaction is reversible or re-linkable).

The distinction collapses — and tombstone-as-completion becomes a valid signal — when the store provides a synchronous hard-delete with a verifiable destruction receipt covering every tier, or when the retention policy is explicitly scoped to logical visibility only (in which case "forget" means "hide," and the agent must not represent it as erasure to downstream consumers). The boundary is legal and architectural, not merely technical: the same tombstone is a completion under a visibility-scoped policy and a ghost under an erasure-scoped one.

## Decision Boundary

An agent must treat a forget as **incomplete** and enter a verification path when any of the following structural signals is present:

- The forget response contains a `tombstone_id` but no `destroyed_at` / `destruction_receipt` / `purge_attestation` field referencing the specific record version.
- The record's audit trail shows a `redaction` event without a corresponding `key_destruction` event in the same transaction boundary.
- The store exposes replica lag or read-after-write staleness, and the forget was issued within the lag window.
- The record's retention metadata shows `expired_at` set but `reclaimed_at` unset (TTL shadow).
- The forget response's status is `forgotten` rather than `erased` / `purged` — status vocabulary is the cheapest available discriminator and is frequently the only one exposed.

Conversely, an agent may treat forget as **complete** only when the audit trail provides a destruction attestation tied to the record version, the redaction key destruction is co-transactional, and either replica convergence is confirmed or the read path is pinned to the primary. When in doubt, the agent must escalate to a verification query rather than assert erasure to a downstream consumer — misrepresenting a ghost as an erasure is the failure mode with legal consequence, and it is indistinguishable from a true completion at the API surface.

## Operational Effect

The practical consequence is a mandatory post-forget verification step and a vocabulary discipline. Agents that skip verification will, at some rate, report lawful erasure while the data remains recoverable — a silent correctness failure that only surfaces under audit. The operational rule is: **never equate `read → null` with erasure; equate it with non-visibility, and require a destruction attestation before asserting erasure.** Where the store cannot produce one, the correct action is to record the forget as *logically complete, physically unverified* and route to the retention-policy owner. This preserves the distinction between the two terminal states and prevents the ghost from being counted as a completed forget in downstream compliance reporting.

```python
class ForgetVerificationError(RuntimeError):
    """Raised when a forget operation cannot be verified as physical erasure."""
    pass


def assert_lawful_erasure(forget_response: dict, audit_trail: list[dict]) -> None:
    """
    Verify that a forget operation reached completed erasure rather than
    a tombstone/ghost shadow state. Raises on any unverified condition.
    """
    record_id = forget_response.get("record_id")
    version = forget_response.get("record_version")

    # Signal 1: tombstone without destruction attestation
    if forget_response.get("tombstone_id") and not forget_response.get("destruction_receipt"):
        raise ForgetVerificationError(
            f"record {record_id} v{version}: tombstone present but no destruction "
            f"attestation; forget is logical-only (ghost risk)"
        )

    # Signal 2: redaction without co-transactional key destruction
    redactions = [e for e in audit_trail if e.get("event") == "redaction"]
    destructions = [e for e in audit_trail if e.get("event") == "key_destruction"]
    for red in redactions:
        txn = red.get("transaction_id")
        if not any(d.get("transaction_id") == txn for d in destructions):
            raise ForgetVerificationError(
                f"record {record_id} v{version}: redaction in txn {txn} has no "
                f"co-transactional key destruction; data is pseudonymized, not erased"
            )

    # Signal 3: TTL shadow (expired but not reclaimed)
    retention = forget_response.get("retention", {})
    if retention.get("expired_at") and not retention.get("reclaimed_at"):
        raise ForgetVerificationError(
            f"record {record_id} v{version}: TTL expired but not physically "
            f"reclaimed; record persists in shadow state"
        )

    # Signal 4: replica convergence unconfirmed
    if forget_response.get("replica_convergence") != "confirmed":
        raise ForgetVerificationError(
            f"record {record_id} v{version}: replica convergence not confirmed; "
            f"read-after-forget may observe pre-forget value"
        )

    # Signal 5: status vocabulary
    if forget_response.get("status") not in ("erased", "purged"):
        raise ForgetVerificationError(
            f"record {record_id} v{version}: status "
            f"{forget_response.get('status')!r} does not assert physical erasure"
        )
```