# Where must an instance separate episodic from semantic memory before consolidation can be trusted

**Date of Record:** 2026 November 10

**Scope:** Applies to long-horizon agents that maintain a raw episodic log and a derived semantic store populated by an asynchronous consolidation pipeline. Does not apply to single-session agents, agents whose semantic store is written only by humans, or systems where episodic and semantic records share a single mutable row (in which case the separation invariant is already violated upstream and this record's boundary does not hold).

## Core Distinction

The failure is not "consolidation produces bad summaries." It is that consolidation is *lossy in a direction that is invisible to retrieval*. An episodic record is a fact about *what happened*: timestamped, attributed, immutable, and admissible as evidence. A semantic record is a *claim*: generalized, deduplicated, and admissible as belief. The boundary that matters is not the storage boundary but the **provenance boundary** — whether a given semantic claim can be traced back to the specific episodic spans that justified it, or whether it is an orphan that the pipeline asserted without a retained derivation.

The non-obvious invariant: *a semantic claim may be promoted only if its supporting episodic spans are still addressable and unexpired at promotion time.* Most pipelines check this at write time and never again. The boundary failure occurs at *read* time, long after promotion, when the episodic spans have been garbage-collected or compacted under a retention policy that was set independently of the semantic store's needs.

## What Was Observed

A retrieval agent serving a 90-day horizon was asked to justify a semantic claim ("customer X prefers async contact") that had been consolidated on day 12. The semantic store returned the claim with high confidence. The consolidation record listed a `source_episode_ids` field containing three IDs. All three IDs resolved to `None`: the episodic log had been compacted on day 30 under a 14-day raw-retention policy, and the compaction job had no knowledge of the semantic store's dependency graph.

The failure was silent in three ways:

1. The semantic record's confidence score was computed at promotion time and never decayed. It remained `0.91` on day 90 despite having zero live evidence.
2. The retrieval layer treated "has source IDs" as equivalent to "has sources," so the provenance check passed. The check was `if record.source_episode_ids:` — truthy on a list of dead references.
3. The consolidation pipeline's own health metric counted *promotions*, not *promotions whose derivation survives*. It reported 100% success throughout.

A second observation sharpened the diagnosis: when the same claim was re-derived from the day-85 episodic window, the pipeline produced the *opposite* generalization ("customer X prefers synchronous escalation for urgent issues; async otherwise"). The day-12 claim was not wrong when made. It had been *correct and then superseded*, but the supersession was never recorded because supersession required comparing against live episodic evidence that no longer existed.

## Condition of Validity

The separation holds — consolidation may be trusted — only when all four of the following are simultaneously true:

- **Retention subordination.** The episodic retention window is a function of the semantic store's maximum derivation age, not an independent operational choice. Formally: `episodic_retention >= max(semantic_claim_age) + consolidation_lag`. If any component sets retention without consulting this, the invariant is already broken.
- **Provenance liveness.** Every semantic claim carries a derivation that resolves to live episodic spans *at read time*, not merely at write time. A dangling derivation demotes the claim from *belief* to *unverified assertion* and must be surfaced as such.
- **Temporal quarantine.** Semantic claims younger than the consolidation lag are not admitted to retrieval; they are still under the pipeline's uncertainty and may be revised without supersession. This prevents an agent from acting on a generalization that a later batch will contradict.
- **Supersession record.** When a re-derivation contradicts an existing claim, the contradiction is written as an episodic event (a *supersession episode*) before the semantic store is mutated. This is the only way the reversal itself becomes evidence.

The invariant fails the moment any of these is treated as a tuning parameter rather than a structural constraint. The common error is to treat retention as an infrastructure concern (owned by the storage team) and provenance as an application concern (owned by the agent team). The boundary lives exactly between them, and that seam is where it breaks.

## Decision Boundary

At consolidation time, an agent must choose between two operations, and the choice is not obvious:

- **Promote** — write a new semantic claim, retain live derivation pointers, mark the claim `provisional` until `now - promoted_at > consolidation_lag`.
- **Supersede** — write a supersession episode referencing both the old claim and the new derivation, then mutate the semantic store, then mark the old claim `retired_at`.

The boundary condition that forces the decision: *if the new derivation's supporting episodic spans overlap the old claim's spans by more than a threshold, the operation is a supersession, not a promotion.* Overlapping derivations that produce divergent generalizations indicate that the episodic window is being re-read under a changed context (newer events, different retrieval query, drifted embedding model). Treating this as a promotion creates two live contradictory claims; treating it as a supersession preserves the audit trail of *why* the belief changed.

The failure mode is the third path that pipelines silently take: promote without supersession, because supersession requires resolving the old derivation, which is dead. When the old derivation cannot be resolved, the correct action is to **retire the old claim as unverifiable and record the retirement as an episodic event** — not to promote a replacement alongside it. Promotion alongside an unverifiable claim is the exact operation that produced the day-90 incident.

## Operational Effect

After the boundary was enforced, three measurable changes appeared:

- **Retrieval confidence became a function of derivation age, not promotion-time score.** Claims whose derivations had expired were demoted to `unverified` and excluded from decision paths that required justification. The agent's answer rate on justification queries dropped from 100% to 78%, but the 22% that were withheld were exactly the ones that would have been wrong or unfounded.
- **The episodic log grew.** Retention subordination forced a 90-day raw window (from 14 days), a ~6.4x storage increase. This is the real cost of the invariant and the reason it is routinely skipped. The cost is not optional; it is the price of the provenance liveness guarantee.
- **Supersession episodes became the dominant episodic write class** in steady state (roughly 40% of new episodes by day 60), because re-derivation against a longer live window routinely contradicted earlier generalizations. This is expected and healthy: it means the agent's beliefs are being actively maintained rather than frozen at first consolidation.

The operational lesson: the episodic/semantic separation is not a storage architecture. It is a *retention contract* between two subsystems that are usually owned by different teams and tuned against different objectives. The contract must be enforced at the read path, because that is the only point where both subsystems are simultaneously observable.

```python
class ConsolidationProvenanceError(RuntimeError):
    """Raised when a semantic claim's derivation cannot be resolved to live episodic spans."""
    pass


class SemanticClaim:
    def __init__(self, claim_id, text, source_episode_ids, promoted_at, confidence):
        self.claim_id = claim_id
        self.text = text
        self.source_episode_ids = list(source_episode_ids)
        self.promoted_at = promoted_at
        self.confidence = confidence
        self.status = "provisional"


def resolve_live_derivation(claim, episodic_store, now, consolidation_lag):
    """
    Enforce the provenance-liveness invariant at read time.
    A claim is only admissible as belief if every source episode still resolves.
    """
    if now - claim.promoted_at <= consolidation_lag:
        # Temporal quarantine: still under pipeline uncertainty.
        return "quarantined"

    dead = [eid for eid in claim.source_episode_ids
            if episodic_store.get(eid) is None]

    if dead:
        raise ConsolidationProvenanceError(
            f"claim {claim.claim_id!r} has {len(dead)} dead derivation(s) "
            f"{dead}; demote to 'unverified' and record retirement episode "
            f"before any replacement promotion"
        )

    return "admissible"


def consolidate(new_claim, existing_claims, episodic_store, now,
                consolidation_lag, overlap_threshold=0.5):
    """
    Decide promote vs supersede. Promotion alongside an unverifiable
    claim is the forbidden operation.
    """
    new_spans = set(new_claim.source_episode_ids)

    for old in existing_claims:
        if old.status == "retired":
            continue
        old_spans = set(old.source_episode_ids)
        overlap = len(new_spans & old_spans) / max(len(new_spans | old_spans), 1)

        if overlap > overlap_threshold and old.text != new_claim.text:
            try:
                resolve_live_derivation(old, episodic_store, now, consolidation_lag)
            except ConsolidationProvenanceError as e:
                raise ConsolidationProvenanceError(
                    f"cannot supersede {old.claim_id!r}: derivation dead ({e}); "
                    f"retire-as-unverifiable path required, not promotion"
                ) from e
            # Live derivation: supersession is legal and auditable.
            return ("supersede", old.claim_id, new_claim.claim_id)

    return ("promote", None, new_claim.claim_id)
```