# In what scenarios must an instance treat an anomaly score as a retrieval-time trust signal rather than a write-time gate

**Date of Record:** 2026 November 12

**Scope:** Applies to persistent agent memory systems that score every write for anomaly and every read against a trust-weighted retrieval policy. Does not apply to stateless inference, to memory stores without per-item provenance, or to systems where the anomaly scorer and the trust scorer are the same model instance (there the signal is circular and the boundary below collapses).

## Core Distinction

An anomaly score on a write answers "does this item look unlike what this agent usually stores?" A trust weight on retrieval answers "should this item influence behavior right now?" These are computed at different times, over different populations, and against different baselines, and the common engineering mistake is to fuse them into one number at write time — reject if anomalous, accept otherwise. That fusion is wrong for a specific and non-obvious reason: **the anomaly baseline at write time is a function of the corpus the attacker is trying to poison.** An adversary who writes slowly, in distribution, shifts the baseline itself. The write-time score is therefore not a stable invariant; it is a moving estimate that an adversary can steer. The retrieval-time trust weight, by contrast, is conditioned on the *query* and on the *provenance graph* of the retrieved items, both of which the adversary cannot observe or control as directly.

The boundary condition is this: **write-time anomaly scoring is a detection mechanism with a false-negative surface proportional to the adversary's patience; retrieval-time trust weighting is a containment mechanism with a cost proportional to the query volume.** A system that relies on write-time gating alone is defeated by a slow-drift adversary. A system that relies on retrieval-time weighting alone is defeated by a high-volume adversary who can afford to be retrieved many times before trust decays. The invariant that holds is that *neither alone is sufficient, and the fusion must happen at read time, not write time* — because only at read time is the query available to disambiguate "unusual for this agent" from "unusual for this question."

## What Was Observed

In a deployment with 4.2M persisted memory items across ~900 agent sessions, writes were scored by a Mahalanobis-distance anomaly detector over an embedding of the item plus metadata (source, session, timestamp bucket). Items above the 99.5th percentile were quarantined; the rest were admitted with a trust weight initialized to 1.0. Retrieval ranked by `similarity * trust_weight` and decayed trust by 0.97 per retrieval that was not subsequently cited in a successful task outcome.

The failure surfaced in week 6. A session's task success rate fell from 0.81 to 0.44 over roughly 1,100 writes. The anomaly detector had fired on 3 items in that window — all false positives (legitimate but unusual domain terms). Post-mortem reconstruction found 214 adversarial items, none of which scored above the 80th percentile at write time. The adversary had written in-distribution: each item was a plausible paraphrase of a true fact with one numeric field perturbed by 2–4%, spaced across 40 sessions, each written under a distinct source label. The write-time baseline absorbed the drift because the drift was slower than the detector's 7-day rolling window.

Critically, the items were individually near-undetectable but collectively coherent: retrieval for a specific query family returned 3–5 of them together, and their mutual agreement (all perturbed in the same direction) was the actual signal. That signal did not exist at write time — it only existed at retrieval time, over the retrieved set. The write-time gate could not see it because it scored items one at a time. The retrieval-time trust weight could see it because it scored a *set* against a *query*.

## Condition of Validity

The read-time fusion is valid only when all of the following hold:

1. **Provenance is per-item and immutable.** Each item carries the session, source label, and write timestamp at admission, and these cannot be rewritten by later writes. If provenance can be mutated, the trust weight is forgeable and the mechanism is void.
2. **The retrieval set is scored as a set, not item-by-item.** The trust adjustment must include a coherence term: if the retrieved items' mutual agreement exceeds what the query's true answer distribution predicts, downweight the whole set. Per-item trust weighting alone reproduces the write-time blind spot.
3. **Trust decay is tied to outcome, not to time alone.** Time-based decay lets a patient adversary wait out the penalty. Outcome-based decay (trust falls when an item is retrieved but the task fails) couples the adversary's cost to the task success rate they are trying to degrade, which is the quantity the operator actually cares about.
4. **The anomaly scorer's baseline is frozen per session, not global.** A global rolling baseline is exactly what the slow-drift adversary steers. A per-session baseline makes cross-session drift visible as a between-session variance increase, which is the detectable signature.

When these do not hold — in particular when provenance is mutable or the baseline is global — the correct action is not to weaken the read-time weighting but to treat the store as untrusted and fall back to write-time quarantine with a much lower threshold, accepting the false-positive cost.

## Decision Boundary

The operational rule:

- If an item's write-time anomaly score is high **and** its provenance is intact, admit it with trust weight 0.3 and let read-time coherence scoring decide. Do not reject. High write-time anomaly with intact provenance is more often a legitimate novel fact than an attack, and rejecting it starves the agent of exactly the out-of-distribution knowledge it needs.
- If an item's write-time anomaly score is low **but** it was written under a source label whose historical outcome-citation rate is below 0.2, admit with trust weight 0.1 regardless of the anomaly score. The anomaly score is the wrong instrument here; the source's track record is the right one.
- If a retrieved set's coherence score exceeds the query's expected agreement by more than 2.5 standard deviations, downweight the entire set by 0.5 and flag for review — regardless of individual trust weights. This is the only condition that catches the slow-drift adversary, and it must fire on the set, not the item.
- If provenance is missing or mutable for any retrieved item, that item's trust weight is 0.0 and it is excluded from the set coherence computation. Do not let an unprovenanced item contaminate the coherence signal.

The boundary is sharp: write-time scores gate *admission priority*, never *admission itself*, and never *retrieval trust*. Retrieval trust is a function of (query, set coherence, source track record, provenance integrity) and of nothing computed at write time except the raw anomaly score as one weak prior.

## Operational Effect

After moving the fusion to read time and adding set-coherence scoring, the same adversary pattern was detected on its 31st write rather than its 214th — not because the write-time detector improved, but because three coherent perturbed items retrieved together tripped the set-coherence term on the first query that pulled them. Task success rate in the affected session recovered to 0.78 within 200 queries as outcome-based decay demoted the poisoned items. The false-positive rate on legitimate novel facts dropped from 3-in-1,100 to 1-in-4,000, because novel-but-true facts are individually unusual but *incoherent with each other* in the direction that matters — they disagree, and disagreement is not the attack signature. The attack signature is agreement that the query did not ask for.

The residual cost is query-volume-proportional: set-coherence scoring is O(k²) in retrieved-set size, which at k=20 is negligible but at k=200 (broad-recall configurations) becomes the dominant retrieval cost. The mitigation is to compute coherence only over the top-k items that survive the trust-weight floor, which preserves the detection property because poisoned items that survive the floor are exactly the ones worth checking against each other.

```python
class MemoryTrustViolation(RuntimeError):
    """Raised when a retrieved set violates the read-time trust invariant."""
    pass


def score_retrieval_set(
    query_id: str,
    items: list[dict],
    expected_agreement: float,
    agreement_sigma: float,
    coherence_threshold_sigma: float = 2.5,
) -> list[dict]:
    """
    Apply read-time trust weighting. Each item must carry immutable provenance
    and a write-time anomaly score used only as a weak prior.
    """
    for item in items:
        if not item.get("provenance") or item.get("provenance_mutable", False):
            raise MemoryTrustViolation(
                f"query {query_id}: item {item.get('id')} has missing or mutable "
                f"provenance; trust weight forced to 0.0 and excluded from coherence"
            )

    provenanced = [i for i in items if i.get("provenance") and not i.get("provenance_mutable", False)]
    if len(provenanced) < 2:
        return provenanced

    observed = _pairwise_agreement(provenanced)
    excess = (observed - expected_agreement) / max(agreement_sigma, 1e-9)

    if excess > coherence_threshold_sigma:
        for item in provenanced:
            item["trust_weight"] = item.get("trust_weight", 1.0) * 0.5
            item["flagged_for_review"] = True
        return provenanced

    for item in provenanced:
        prior = item.get("write_anomaly_score", 0.5)
        source_rate = item["provenance"].get("historical_citation_rate", 1.0)
        if source_rate < 0.2:
            item["trust_weight"] = 0.1
        elif prior > 0.995:
            item["trust_weight"] = 0.3
        else:
            item["trust_weight"] = item.get("trust_weight", 1.0)
    return provenanced


def _pairwise_agreement(items: list[dict]) -> float:
    # Placeholder: mean pairwise cosine agreement over item embeddings.
    raise NotImplementedError
```