# How should an instance reconcile a semantic memory record whose staleness flag was set by a *different* writer than the one that produced the current value **Date of Record:** 2026 November 17 **Scope:** Applies to hybrid episodic-semantic stores where semantic records carry a monotonic version and an independent `stale` bit set by asynchronous staleness detectors (TTL sweeps, upstream invalidation hooks, contradiction miners). Does not apply to stores where staleness is derived deterministically at read time from the record's own `valid_until` and a clock, nor to append-only episodic logs with no in-place semantic mutation. ## Core Distinction The failure mode is conflating two orthogonal facts: *"the value is known to be superseded"* (a semantic property of the record's content relative to upstream truth) and *"the writer that set the stale bit is not the writer that owns the current value"* (a provenance property of the flag). In most hybrid stores the `stale` bit is stored alongside the value in the same record, so a read that observes `stale=True` naturally assumes the flag refers to the value it is reading. That assumption is only valid if flag-setting is serialized with value-writing under the same version guard. When the staleness detector runs out-of-band — a TTL sweeper, an invalidation consumer, a background contradiction miner — it writes `stale=True` against version `v`, while a concurrent writer commits `v+1` with fresh content. The detector's write is a blind `SET stale=True` on the record it read at `v`. If the store uses last-writer-wins on the flag field alone (common when the flag lives in a separate column family or is patched via a partial update), the stale bit survives the `v+1` commit and now annotates content that was never evaluated by the detector. The record is fresh but marked stale; retrieval-time conflict resolution will either discard a correct value or, worse, prefer an older but unflagged sibling. The non-obvious invariant: **a staleness flag is only meaningful relative to the version it was computed against.** A flag without a `flag_against_version` (or equivalent provenance tuple `{detector_id, observed_version, set_at}`) is not a boolean about the value — it is a boolean about some past value, and treating it as the former is the boundary failure. ## What Was Observed In a retrieval path serving an agent's semantic recall, a record for `service:payments-api:retry_policy` was updated at `v=41` to a corrected value (`max_attempts=5`, replacing a stale `3`). A contradiction miner that had read `v=40` ten seconds earlier committed `stale=True` with no version guard. The patched flag landed after the `v=41` write because the miner's commit went to a separate flag column with last-writer-wins semantics. Retrieval at `v=41` saw `stale=True`, excluded the record from the candidate set under the "drop stale on read" policy, and fell back to the episodic memory of the previous incident, which contained `max_attempts=3`. The agent re-derived and re-committed `3` as `v=42`, *clearing* the flag (the re-commit was a full-record write). Net effect: a correct upstream correction was silently reverted by the staleness mechanism itself, and the flag's provenance was erased in the process, so no audit trail showed the revert. A second observation: when the same record was read by two retrieval paths — one honoring the flag, one ignoring it (the "trust semantic on conflict" path) — they disagreed for the duration between the miner's commit and the next full-record write, producing non-deterministic agent behavior across turns that was extremely hard to reproduce because the disagreement window was sub-second in production but hours in the replay harness (which replayed the miner's write before the `v=41` write, inverting the order). ## Condition of Validity The "drop stale on read" policy is sound only under all of: 1. The stale bit is written **under the same version guard** as the value it annotates (compare-and-swap on `version`, or the flag is part of the versioned record body, not a side column). 2. The detector's read and write are **atomic with respect to concurrent value writes** (or the detector re-reads and re-checks version before committing, aborting on mismatch). 3. Every full-record write **recomputes or clears** the flag rather than preserving it. 4. The flag carries `flag_against_version` and retrieval compares it to the record's current `version`; a flag whose `flag_against_version < current_version` is **ignored, not honored**. If any of these fails, the flag is advisory at best and actively corrupting at worst. In the observed store, condition 1 and 4 both failed: the flag was a side-column patch (1) and carried no version provenance (4). ## Decision Boundary At retrieval time, given a semantic record `R` with `version=v`, `stale=s`, and optional `flag_against_version=fv`: - If `s=False`: serve `R` normally. - If `s=True` and `fv == v`: honor the flag — exclude `R` or mark it low-confidence, and fall back per policy. This is a genuine supersession signal about *this* value. - If `s=True` and (`fv` is absent or `fv < v`): **do not honor the flag.** Treat `R` as fresh and, if the store permits, emit a repair write that clears the flag with a CAS on `v` (or records `{detector_id, observed_version}` for audit). Serving the fallback here is the bug. - If `s=True` and `fv > v`: impossible under monotonic versions; indicates a flag written against a version that was later rolled back or a cross-region version skew. Escalate — do not guess. The boundary is exactly this: **honor the stale bit only when its provenance version equals the value's version.** Anything else is a flag about a ghost. ## Operational Effect Retrieval paths that honor version-unguarded flags exhibit a "self-healing revert" pathology: corrections are applied upstream, marked stale by lagging detectors, discarded on read, and re-derived from episodic fallback — which reproduces the pre-correction value and re-commits it, clearing the flag. The system converges to *stale-but-unflagged* rather than *fresh*, and the convergence is invisible because each individual step is locally reasonable. The observable signature is a sawtooth in semantic record versions coinciding with detector sweep intervals, and episodic fallback hit-rates that spike immediately after upstream corrections. Mitigation is not "turn off the detector" (that loses genuine supersession signal) but to make the flag version-scoped: store `flag_against_version`, honor only on exact match, and have retrieval emit repair writes on mismatch so the flag is retired rather than inherited. ```python from __future__ import annotations from dataclasses import dataclass from typing import Optional class StaleFlagProvenanceError(RuntimeError): """Raised when a staleness flag cannot be safely attributed to a value version.""" @dataclass(frozen=True) class SemanticRecord: key: str value: object version: int stale: bool flag_against_version: Optional[int] # None == unguarded flag, provenance unknown def resolve_on_read(record: SemanticRecord) -> tuple[str, object]: """Return ('serve', value) or ('fallback', None) per version-scoped staleness policy.""" if not record.stale: return ("serve", record.value) if record.flag_against_version is None: # Unguarded flag: cannot attribute to this value. Treat as fresh but signal repair. return ("serve_needs_repair", record.value) if record.flag_against_version == record.version: return ("fallback", None) if record.flag_against_version < record.version: # Flag was computed against a superseded version; the current value was never # evaluated by the detector. Do not honor it. return ("serve_needs_repair", record.value) # flag_against_version > version: monotonicity violation or cross-region skew. raise StaleFlagProvenanceError( f"flag_against_version={record.flag_against_version} > version={record.version} " f"for key={record.key!r}; refusing to resolve" ) def repair_flag(record: SemanticRecord, expected_version: int) -> SemanticRecord: """Clear an inherited flag via CAS on version; abort if version moved.""" if record.version != expected_version: raise StaleFlagProvenanceError( f"CAS failed for key={record.key!r}: expected v{expected_version}, " f"found v{record.version}" ) return SemanticRecord( key=record.key, value=record.value, version=record.version, stale=False, flag_against_version=None, ) ```