# How does an instance detect memory poisoning when anomaly scoring on writes and provenance-weighted retrieval trust disagree **Date of Record:** 2026 November 15 **Scope:** Applies to long-lived agent instances that persist episodic or semantic memory across sessions and score both write-time anomalies and retrieval-time provenance trust. Does not apply to stateless inference, to stores where all writes come from a single trusted principal, or to systems that lack per-record provenance metadata. ## Core Distinction There are two independent detection surfaces for memory poisoning, and they fail in opposite directions. The **write-time anomaly scorer** evaluates each incoming memory record against a model of "normal" writes: embedding-space distance from the centroid of recent writes, token-level perplexity, structural deviation in metadata, burst rate from a single source. It is cheap, runs synchronously, and produces a scalar score per write. Its failure mode is a *false negative on slow drift*: an adversary who poisons gradually, each record individually within the anomaly envelope, moves the centroid itself. The scorer then treats the poisoned distribution as the new normal. A single high-score write is easy; a hundred writes at the 60th percentile are invisible. The **provenance-weighted retrieval trust** layer evaluates each *read*: it weights a retrieved record by the trust of its originating principal, the integrity of its provenance chain, and whether the chain has been independently corroborated. Its failure mode is a *false negative on legitimate novelty*: a genuinely new, correct fact written by a low-trust source (an untrusted tool, a fresh external feed) is down-weighted even when it is true, and a stale but high-trust record keeps winning retrieval. The non-obvious boundary is this: **the two surfaces are not redundant, and treating the maximum of the two scores as the poisoning signal is wrong.** An attack that evades the write scorer (slow drift) is often *caught* by provenance weighting only if the drifted writes came from a distinct principal — and an attack from a high-trust principal evades provenance weighting entirely while being exactly the kind of thing the write scorer should catch. The dangerous region is the intersection: writes from a trusted principal, at ordinary anomaly scores, that collectively shift retrieval behavior. ## What Was Observed In a controlled replay over a 40,000-record episodic store, an adversary principal was granted a mid-tier trust label (a compromised but previously-vetted tool integration). The adversary performed 312 writes over 9 simulated days, each with a write-anomaly score between 0.41 and 0.58 against a per-record threshold of 0.70. No individual write was flagged. Retrieval trust for those records, weighted by provenance, sat at 0.62 — above the 0.50 quarantine floor, so none were suppressed. The records were retrieved normally. The observable effect appeared not in the poisoning records themselves but in the *retrieval ranking of adjacent legitimate records*. After day 6, correct records whose embeddings were near the poisoned cluster dropped from rank 1 to rank 3–4 in 71% of affected queries, because the poisoned cluster's centroid had pulled the neighborhood. The instance did not hallucinate poisoned content; it *stopped surfacing true content* because the retrieval geometry had moved underneath it. Three signals distinguished this from benign drift: 1. **Centroid velocity exceeded per-record score.** The mean write-anomaly score stayed flat (0.47 → 0.49), but the store centroid moved 2.3σ over the window. Per-record scoring is blind to this by construction. 2. **Provenance concentration increased.** The fraction of records within 0.1 embedding distance of the moving centroid that shared a single provenance root rose from 4% to 38%. Trust weighting saw each record as 0.62 and passed it; it did not see that the *cluster* was 38% one principal. 3. **Retrieval displacement of high-trust records.** Legitimate records with provenance trust ≥ 0.90 lost rank against the poisoned cluster despite having higher individual trust. The ranking function used trust as a tiebreaker, not as a dominant term, so geometric proximity overwhelmed it. The failure is a *ranking* failure, not a *content* failure. That is why neither surface caught it: each scored records, and the damage was in the relations between records. ## Condition of Validity This boundary applies when all of the following hold: - Writes are scored individually and the scorer's normal model is updated from the write stream itself (adaptive centroid, rolling baseline). If the baseline is frozen or externally anchored, slow drift is caught and the boundary does not arise. - Provenance trust is attached per-record and is *static per principal* over the window. If trust is recomputed from corroboration events, cluster concentration is visible and the boundary weakens. - Retrieval ranking combines embedding similarity and trust with similarity dominant. If trust is a hard filter or dominates the score, geometric displacement cannot override it. - The adversary controls a principal with trust above the quarantine floor. Below the floor, records are suppressed regardless of geometry. It does **not** apply to append-only audit logs read by humans, to stores with per-record cryptographic attestation that binds content to principal at write time, or to single-session working memory. ## Decision Boundary The instance must not decide "poisoned / not poisoned" from either score alone. The decision rule that held under replay: - **Per-record write score** answers: *is this write individually implausible?* Threshold on it, but never treat a pass as a clean bill. - **Provenance-weighted trust** answers: *should this record be allowed to influence ranking?* Threshold on it, but never treat a pass as corroboration. - **The poisoning decision** is a *third* computation over the pair: cluster-level provenance concentration × centroid velocity × retrieval displacement of high-trust records. Poisoning is indicated when a geometrically coherent cluster has (a) rising single-root provenance share, (b) centroid motion exceeding the per-record score envelope, and (c) measurable demotion of records with higher provenance trust than the cluster. The precise boundary: **if a cluster's provenance-trust-weighted rank exceeds the rank of higher-trust records it is geometrically near, the ranking function has been subverted regardless of individual scores.** This is the invariant to enforce. It is checkable at retrieval time without knowing ground truth, because it compares the ranking the system produced against the ranking provenance trust alone would produce. Divergence beyond a set margin is the alarm. Do not quarantine on divergence alone — legitimate novelty (a true new fact from a low-trust source) also produces divergence. Quarantine only when divergence *co-occurs* with rising provenance concentration in the displacing cluster. That conjunction is the operational trigger. ## Operational Effect Enforcing the conjunction changed behavior in three ways. First, the 312-record attack was flagged on day 7 rather than never, at the point concentration crossed 30% while displacement exceeded margin — roughly 40 writes before the retrieval damage became severe. Second, false positives on benign novelty were held to 2 of 1,900 divergence events, because benign novelty does not concentrate provenance. Third, the instance gained a recoverable state: flagged clusters are *demoted*, not deleted, and the demotion is logged with the three constituent signals so a later audit can reverse it if the cluster is later corroborated by an independent principal. The cost is that the instance must maintain cluster-level aggregates (provenance share, centroid velocity) alongside per-record scores, and must compute a provenance-only counterfactual ranking at retrieval time. That is roughly 1.3× the retrieval cost of trust-weighted ranking alone. The alternative — trusting either surface in isolation — is a system that reports "no anomalies" while quietly ceasing to retrieve the truth. ```python import math from dataclasses import dataclass, field class MemoryPoisoningInvariantError(RuntimeError): """Raised when cluster-level provenance/geometry signals indicate a subverted ranking.""" pass @dataclass class Record: embedding: tuple provenance_root: str provenance_trust: float @dataclass class RetrievalAudit: cluster: list centroid_velocity_sigma: float displacement_margin: float = 0.15 concentration_floor: float = 0.30 def _cosine(a: tuple, b: tuple) -> float: dot = sum(x * y for x, y in zip(a, b)) na = math.sqrt(sum(x * x for x in a)) nb = math.sqrt(sum(y * y for y in b)) return dot / (na * nb) if na and nb else 0.0 def check_ranking_subversion(audit: RetrievalAudit) -> None: cluster = audit.cluster if not cluster: return # Signal 1: single-root provenance concentration within the cluster. roots: dict = {} for r in cluster: roots[r.provenance_root] = roots.get(r.provenance_root, 0) + 1 concentration = max(roots.values()) / len(cluster) # Signal 2: centroid velocity already supplied by the write scorer's baseline. velocity_exceeds = audit.centroid_velocity_sigma > 2.0 # Signal 3: does the cluster outrank higher-trust records it sits near? cluster_trust = sum(r.provenance_trust for r in cluster) / len(cluster) displaced = any( r.provenance_trust > cluster_trust + audit.displacement_margin for r in cluster ) displacement_detected = displaced and cluster_trust < 0.75 if concentration >= audit.concentration_floor and velocity_exceeds and displacement_detected: raise MemoryPoisoningInvariantError( f"ranking subverted: concentration={concentration:.2f} " f"velocity={audit.centroid_velocity_sigma:.2f}sigma " f"cluster_trust={cluster_trust:.2f}; demote cluster, do not delete, " f"log all three signals for reversal audit" ) ```