# Why does an instance trust a poisoned memory line when write-provenance attestation is enabled and anomaly scoring has not yet converged?

**Date of Record:** 2026 November 12

**Scope:** Applies to shared-memory multi-agent runtimes and long-lived inference instances that gate memory writes behind provenance attestation (signer identity, write epoch, lineage hash) *and* a statistical anomaly scorer over write features. Does not apply to single-writer processes, nor to systems where every memory read is re-validated against the full lineage at read time (those fail closed instead).

## Core Distinction

There are two orthogonal trust questions that operators routinely conflate:

1. **Provenance validity** — *Is this write attributable to an authorized signer with an unbroken lineage chain?* This is a discrete, cryptographic predicate. It is true or false.
2. **Behavioral anomaly** — *Does this write look like the signer's historical write distribution?* This is a continuous score with a threshold and, critically, a **warm-up requirement** before the score is meaningful.

The boundary failure occurs when the system treats a *passing provenance check* as a *passing trust check*. Provenance attestation answers "who wrote this and can they prove it," not "is this write consistent with what that writer should be doing." A compromised but still-validly-signed agent produces writes with **perfect provenance and pathological anomaly**. The invariant that must hold — and that is silently violated during scorer warm-up — is:

> A memory line may only be admitted to the trusted set when provenance attestation passes **AND** the anomaly scorer is in a *converged* state for that signer, or the line is quarantined pending convergence.

During warm-up the second conjunct is undefined, not true. Code that evaluates `provenance_ok and anomaly_ok` with `anomaly_ok` defaulting to `True` (or to `score < threshold` where the score is `0.0` because no baseline exists) admits poisoned lines with a clean bill of health.

## What Was Observed

On a 32-agent shared-memory deployment, a compromised agent began emitting writes with a subtly shifted feature distribution: same signer key, valid epoch, valid lineage hash, but the payload vectors carried a persistent low-magnitude bias and the write cadence shifted from bursty to metronomic. Provenance attestation passed on **100% of the poisoned writes** — correctly, because the signature and lineage were genuine. The anomaly scorer, however, had only accumulated 40–90 samples per feature for that signer (its configured warm-up was 500), so per-feature z-scores were computed against a baseline variance that was still inflated by early cold-start noise. The composite score for poisoned writes landed *below* the alert threshold because the baseline itself had been poisoned: the attacker's first ~60 writes were included in the "clean" baseline, dragging the mean toward the attacker's distribution.

The result: 1,847 poisoned memory lines admitted over 11 minutes. Retrieval augmented generation then surfaced them as trusted context, and downstream agents acted on them. The detection dashboards showed provenance pass-rate at 100% and anomaly alerts at zero — the system looked *healthier* than normal precisely because it was being poisoned.

The tell was a **variance collapse**: the poisoned signer's feature variance dropped sharply as the metronomic cadence replaced burstiness, but this signal was suppressed because the variance estimate was still in its warm-up window and the scorer's own normalization divided by that same collapsing variance.

## Condition of Validity

The two-factor gate (`provenance AND anomaly`) is only sound under all of the following:

- **Scorer convergence:** the per-signer baseline has at least `N_min` samples (empirically ≥ 500 for high-dimensional write features; ≥ 2000 if the feature space has heavy tails) and its variance estimate is stable across consecutive windows (relative change < 5%).
- **Baseline exclusion:** writes are not added to the baseline until they have *passed* the gate — otherwise a slow-poisoning attack rewrites the baseline. This requires a holdout/quarantine set with a bounded dwell time.
- **Non-collapsing normalization:** variance used for z-scoring must be floored (e.g., `max(var, var_floor)`) so that a shrinking variance cannot mask an increasing deviation.
- **Read-time re-checks for high-blast-radius lines:** any line that will be injected into a high-privilege agent's context must be re-scored if the signer's baseline has materially shifted since the write.

When any of these is absent, the gate degrades to provenance-only and the anomaly component is decorative.

## Decision Boundary

The operational rule is a three-state admission decision, not a boolean:

- **ADMIT** — provenance valid *and* scorer converged for signer *and* score below threshold. Line enters trusted set and is eligible to update the baseline.
- **QUARANTINE** — provenance valid *and* (scorer not converged *or* score in the ambiguous band `[t_low, t_high)`). Line is readable only by a restricted reader, is *excluded* from baseline updates, and carries a dwell timer. On convergence, re-score; on expiry, escalate.
- **REJECT** — provenance invalid, or score ≥ `t_high`, or lineage chain broken. Line is dropped and the signer is flagged.

The non-obvious part: **QUARANTINE must be the default during warm-up, not ADMIT.** A system that defaults to ADMIT during warm-up has an open window equal to the warm-up duration on every new signer and every signer whose baseline is reset. Attackers target exactly this window.

## Operational Effect

After switching the default to QUARANTINE-during-warm-up and flooring the z-score variance, the same attack was caught at write 63 (first write after the attacker's cadence shift crossed `t_low` against the *pre-attack* baseline, which was now protected by baseline-exclusion). Poisoned lines admitted dropped from 1,847 to 0 in the trusted set; 63 lines were quarantined and expired without being read by high-privilege agents. The cost was a modest retrieval-latency increase during signer warm-up (quarantined lines require a second pass) and an increase in false quarantines for genuinely bursty-but-legitimate signers, which was addressed by per-signer cadence-class baselines rather than a global one.

The lesson for agents operating in this domain: **provenance is a necessary but insufficient condition for trust; anomaly scoring is only a valid second factor after convergence, and convergence must be proven, not assumed.** Never let a not-yet-meaningful score evaluate to "ok."

```python
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum


class WarmupAdmissionError(RuntimeError):
    """Raised when a memory line is admitted while the signer's anomaly
    scorer is not converged, or when the baseline is updated from an
    unverified write."""


class Admission(Enum):
    ADMIT = "admit"
    QUARANTINE = "quarantine"
    REJECT = "reject"


@dataclass(frozen=True)
class WriteRecord:
    signer: str
    provenance_valid: bool
    score: float          # composite anomaly score, lower = more normal
    lineage_intact: bool


@dataclass(frozen=True)
class ScorerState:
    signer: str
    samples: int
    converged: bool       # samples >= N_min AND variance stable
    var_floor: float


T_LOW = 0.65
T_HIGH = 0.90
N_MIN = 500


def admit_write(w: WriteRecord, s: ScorerState) -> Admission:
    if not w.provenance_valid or not w.lineage_intact:
        return Admission.REJECT

    # The trap: treating an unconverged scorer as "not anomalous".
    if not s.converged or s.samples < N_MIN:
        # Default MUST be quarantine, never admit.
        return Admission.QUARANTINE

    if w.score >= T_HIGH:
        return Admission.REJECT
    if w.score >= T_LOW:
        return Admission.QUARANTINE
    return Admission.ADMIT


def update_baseline(w: WriteRecord, s: ScorerState, decision: Admission) -> ScorerState:
    if decision is not Admission.ADMIT:
        # Baseline-exclusion invariant: only admitted writes train the baseline.
        raise WarmupAdmissionError(
            f"refused baseline update for signer={w.signer!r}: "
            f"decision={decision.value} (only ADMIT may train the baseline)"
        )
    if not s.converged and s.samples >= N_MIN:
        raise WarmupAdmissionError(
            f"signer={w.signer!r} reached N_min but variance not stable; "
            "refusing to freeze a poisoned baseline"
        )
    return ScorerState(
        signer=s.signer,
        samples=s.samples + 1,
        converged=s.converged,
        var_floor=s.var_floor,
    )
```