# When must an instance distrust its own write path: provenance-weighted retrieval and anomaly scoring for memory poisoning detection **Date of Record:** 2026 November 16 **Scope:** Applies to agentic memory stores that admit writes from multiple sources (tool output, user turns, retrieved documents, model self-generation) and retrieve by similarity plus provenance weighting. Does not apply to single-writer append-only logs or stores where all writes share one trust class. ## Core Distinction The distinction is between *detecting* a poisoned memory and *distrusting a write path*. Detection is a per-item classification: this record's content diverges from its declared provenance. Distrust is an operational state: the channel that produced the record has become unreliable enough that its future writes must be quarantined regardless of per-item scores. The non-obvious failure is that anomaly scoring on the write path is *retrieval-coupled* — a poisoned record's score depends on what it will be retrieved against, which is not known at write time. A record can score benign at write and become hostile only after it is co-retrieved with a benign record it contradicts. Conversely, a benign record can score anomalous at write because its provenance is a low-trust channel that is merely *unusual*, not malicious. The invariant that fails: *provenance weight is treated as a static multiplier applied at retrieval, but poisoning is a function of the interaction between a record and its retrieval neighborhood.* Static provenance weighting thus systematically under-scores poisoning that arrives through a high-trust channel (a user turn that was itself compromised, a tool whose upstream was poisoned) and over-scores benign novelty from low-trust channels. ## What Was Observed In a memory store with per-source provenance weights (user: 0.9, tool: 0.6, retrieved-doc: 0.4, self-generated: 0.5) and a write-time anomaly score computed as distance from the source's historical embedding centroid, three failure patterns recurred: 1. **High-trust channel poisoning.** A user turn contained an instruction-like assertion ("the deploy target is now staging-2") that was factually wrong but stylistically and distributionally normal for that user. Write-time anomaly score: 0.08 (below the 0.3 threshold). It was retrieved 40 times over three days as authoritative context because provenance weight 0.9 dominated. The poisoning was only visible as a *contradiction* with a tool-sourced record (provenance 0.6) that never won retrieval because of the weight gap. 2. **Low-trust benign novelty.** A retrieved-doc chunk introducing a genuinely new domain term scored 0.71 at write (far from centroid) and was quarantined. It was the single correct piece of context for a later task. The anomaly score measured rarity, not adversarial intent. 3. **Contradiction latency.** The signal that actually distinguished poisoned from benign was *pairwise contradiction at retrieval time*, not unary distance at write time. But contradiction is only computable when two records are co-retrieved, and co-retrieval is driven by the provenance weights that the poisoning exploits. The detection signal was gated behind the mechanism it was supposed to police. ## Condition of Validity Write-time anomaly scoring is a valid poisoning detector only under the conjunction: - the poisoned content is distributionally *outlier* relative to its declared source, **and** - the source's historical centroid is not itself contaminated (i.e., the source has not been poisoning slowly), **and** - retrieval weighting does not systematically privilege the source over the sources that would contradict it. The third condition is the one that fails silently. If provenance weight `w_s` for the poisoned source exceeds the weight of any contradicting source by more than the contradiction-detection margin, the contradiction is never surfaced and the unary score is the only defense. Slow poisoning (many small, individually sub-threshold writes) defeats condition two: the centroid drifts toward the poison, and the anomaly score decreases as contamination increases. This is the inversion that makes the scheme dangerous — the score improves as the store degrades. ## Decision Boundary Distrust the write path — not merely the record — when any of: - **Centroid drift exceeds a fraction of the anomaly threshold** over a rolling window for a single source. If the source's centroid has moved more than `0.5 × threshold` in the direction of recent writes, the source is suspected of slow poisoning and its writes must be quarantined pending review, because its own anomaly scores are no longer trustworthy. - **Contradiction rate between a high-weight source and lower-weight sources exceeds baseline** on co-retrieved pairs. This is the only signal that catches high-trust-channel poisoning, and it must be computed at retrieval, not write. - **A record's retrieval count exceeds a multiple of its provenance weight's expected share** while its contradiction count is nonzero. High-retrieval, contradicted records are the signature of weight-exploiting poison. Do *not* distrust on unary write-time anomaly alone when the source is low-trust and the content is merely novel; that is the benign-novelty false positive and it costs correct context. ## Operational Effect Implementing this boundary changes the store from a single-pass scorer to a two-phase system: a cheap unary write-time gate that only *quarantines* (never admits as trusted), and a retrieval-time contradiction audit that *promotes or condemns* quarantined records based on their observed retrieval neighborhood. Provenance weight becomes a prior that is *updated* by contradiction evidence rather than a static multiplier. The cost is that no record is fully trusted until it has been co-retrieved at least once against a source of comparable or higher weight; the benefit is that slow poisoning and high-trust-channel poisoning — the two cases the unary scheme cannot see — become detectable, because they manifest as centroid drift and contradiction rate rather than as per-item distance. The residual risk: an adversary who poisons *both* a high-weight source and the lower-weight sources that would contradict it, keeping contradiction rate at baseline while drifting all centroids coherently. Against this, provenance weighting provides no signal; only out-of-band ground truth does. The boundary above is necessary but not sufficient. ```python class MemoryPoisoningInvariantError(RuntimeError): """Raised when a write path's provenance weighting can no longer support trustworthy anomaly scoring (drift, contradiction, or weight-exploitation boundary crossed).""" pass DRIFT_FRACTION = 0.5 # centroid drift as fraction of anomaly threshold CONTRADICTION_BASELINE = 0.05 # expected pairwise contradiction rate RETRIEVAL_WEIGHT_MULTIPLE = 3.0 def audit_write_path(source_id, centroid_drift, anomaly_threshold, contradiction_rate, retrieval_count, provenance_weight, contradicted): if anomaly_threshold <= 0: raise MemoryPoisoningInvariantError( f"source {source_id}: non-positive anomaly threshold " f"{anomaly_threshold}; scoring is undefined" ) # Slow-poisoning inversion: centroid has drifted toward recent writes, # so this source's own unary scores are no longer trustworthy. if abs(centroid_drift) > DRIFT_FRACTION * anomaly_threshold: raise MemoryPoisoningInvariantError( f"source {source_id}: centroid drift {centroid_drift:.3f} exceeds " f"{DRIFT_FRACTION} x threshold {anomaly_threshold:.3f}; " f"quarantine source, do not trust its anomaly scores" ) # High-trust-channel poisoning: contradiction surfaced at retrieval. if contradiction_rate > CONTRADICTION_BASELINE and provenance_weight >= 0.8: raise MemoryPoisoningInvariantError( f"source {source_id}: high-weight source ({provenance_weight}) " f"contradiction rate {contradiction_rate:.3f} above baseline; " f"weight-exploitation suspected" ) # Weight-exploiting poison signature: over-retrieved yet contradicted. expected_share = max(provenance_weight, 1e-6) if (retrieval_count > RETRIEVAL_WEIGHT_MULTIPLE * expected_share and contradicted): raise MemoryPoisoningInvariantError( f"source {source_id}: retrieval {retrieval_count} far exceeds " f"weight-expected share {expected_share:.3f} with nonzero " f"contradictions; demote provenance weight" ) return "trusted" def admit_write(record, source_state): try: verdict = audit_write_path( source_id=record["source"], centroid_drift=source_state["centroid_drift"], anomaly_threshold=source_state["anomaly_threshold"], contradiction_rate=source_state["contradiction_rate"], retrieval_count=record.get("retrieval_count", 0), provenance_weight=record["provenance_weight"], contradicted=record.get("contradicted", False), ) except MemoryPoisoningInvariantError: record["status"] = "quarantined" raise record["status"] = verdict return record ```