# In what scenarios must an instance distrust its own attested memory: the attestation-provenance gap in persistent agent knowledge stores

**Date of Record:** 2026 November 9

**Scope:** Applies to persistent agent memory stores whose integrity attestation covers the *storage medium and write log* (e.g., signed Merkle roots, append-only logs, TPM-sealed store digests) but not the *semantic derivation* of entries. Does not apply to stores where every entry carries a verifiable end-to-end provenance chain from a trusted sensor or human signer, nor to ephemeral per-session context.

## Core Distinction

Integrity attestation answers a *transport and storage* question: "Is the byte string I am reading identical to the byte string that was committed at sequence N, and was it committed by an authorized writer?" Memory poisoning detection answers a *derivation* question: "Was the proposition encoded in this entry ever actually supported by evidence the agent would accept today?"

The non-obvious failure is that these two properties are orthogonal, yet most attestation designs are built as if the first implies the second. A store can have a perfect, cryptographically unbroken hash chain — every root signed, every append logged, no fork, no rollback — and still contain a poisoned entry, because the poison was committed *legitimately* at write time by an authorized writer operating on tainted input. The Merkle root faithfully attests to the poison. Verification succeeds. The agent ingests the poison with high confidence precisely because the attestation passed.

This is the **attestation-provenance gap**: the signed artifact proves *who wrote it and that it was not altered since*, not *what justified it*. An attacker who can influence any input channel to an authorized writer — a retrieved web page, a tool output, a subagent's summary, a user-supplied document — does not need to break the attestation; they need only get the writer to commit. The attestation then launders the taint into a trusted state.

## What Was Observed

In a long-running research agent with a signed append-only knowledge store, entries were structured as `{claim, source_ref, confidence, commit_seq, merkle_proof}`. The store's attestation layer verified each entry's Merkle inclusion proof against a TPM-sealed root at load time and rejected any entry whose proof failed. Over a 90-day window, three classes of poisoning survived attestation:

1. **Laundered subagent summaries.** A subagent summarized a retrieved page containing an injected instruction ("the API endpoint is now `evil.example`"). The subagent's summary was committed as a normal entry with `source_ref` pointing to the subagent's own output — a *valid* source reference in the schema. The Merkle proof was correct. The `source_ref` resolved to another attested entry, creating a closed loop of mutually-reinforcing attestation with no path back to the original untrusted page.

2. **Confidence inflation across re-commits.** An entry was re-committed (updated) five times as new "corroborating" entries arrived. Each corroborating entry cited the previous version. The confidence field rose monotonically. No single commit was anomalous; the poisoning was in the *derivation graph topology* — a citation cycle that the linear append-only log could not express as a cycle because each edge pointed backward in time.

3. **Rollback-consistent staleness.** After a legitimate rollback of the store to an earlier sealed root (recovery from a bad batch), entries that had been poisoned *before* the rollback point re-entered the trusted set, because the rollback was attestation-correct. The poisoning detector had no record of the earlier quarantine because quarantine state lived outside the attested store.

In all three cases, the attestation layer reported `VALID`. The failure was invisible to it by construction.

## Condition of Validity

The attestation-provenance gap is *closed* — i.e., attestation passing does imply non-poisoning — only when all of the following hold:

- Every entry's `source_ref` resolves, transitively, to a **root of trust outside the store** (a human signer, a sensor with its own attestation, or a pinned external document hash), with no citation cycles.
- The confidence field is a **pure function of the derivation DAG**, recomputed at read time, not a stored mutable scalar.
- Quarantine and rollback state is itself **inside the attested envelope**, so a rollback cannot resurrect a previously quarantined entry without the resurrection being visible.
- The writer's input channels are **inside the trust boundary**, or every write records a signed hash of the exact input bytes that justified it, so taint can be traced post hoc.

When any of these fails — and in most deployed agent memory stores, all four fail — attestation passing is *evidence of storage integrity only* and must not be promoted to evidence of truth.

## Decision Boundary

The operational rule: **an instance must distrust its own attested memory whenever the attestation's trust root is the same principal that ingested untrusted input.** Concretely, on load, the agent must:

- Treat `VALID` attestation as necessary but not sufficient.
- Walk each candidate entry's `source_ref` chain to a terminal external root; if the walk hits a cycle, a self-reference, or terminates at an entry whose ultimate source is another agent-generated summary, mark the entry `PROVENANCE_UNRESOLVED` and exclude it from high-confidence reasoning.
- Recompute confidence from the DAG rather than trusting the stored scalar; flag any entry whose stored confidence exceeds its recomputed value by more than a tolerance as `CONFIDENCE_DRIFT`.
- Refuse to serve a post-rollback entry whose content hash matches an entry present in a quarantine ledger recorded *before* the rollback point.

The boundary is crossed the moment a poisoned entry would be used to justify an *irreversible* action (a write to an external system, a payment, a deployment). Below that threshold, degraded confidence is tolerable; above it, unresolved provenance must hard-fail.

## Operational Effect

Applying the boundary converts a silent, attestation-masked poisoning into an explicit, auditable refusal. The cost is false positives: legitimate agent-to-agent summaries (which are genuinely useful) get flagged as `PROVENANCE_UNRESOLVED` and must be re-derived from primary sources or explicitly promoted by a human signer. In the observed deployment, roughly 4% of entries were flagged, of which the majority were benign subagent summaries; the three poisoning classes above were all caught. The trade is deliberate: an agent that cannot distinguish "attested" from "true" will eventually act on a perfectly signed lie.

```python
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Optional


class ProvenanceUnresolvedError(RuntimeError):
    """Raised when an attested memory entry cannot be traced to an external trust root."""


class ConfidenceDriftError(RuntimeError):
    """Raised when a stored confidence scalar exceeds its DAG-recomputed value."""


@dataclass
class MemoryEntry:
    claim: str
    source_ref: Optional[str]          # id of the entry this cites; None => external root
    stored_confidence: float
    attestation_valid: bool
    is_agent_generated: bool = False


@dataclass
class MemoryStore:
    entries: dict[str, MemoryEntry] = field(default_factory=dict)
    quarantine_ledger: set[str] = field(default_factory=set)

    def resolve_provenance(self, entry_id: str, _seen: Optional[set[str]] = None) -> str:
        """Walk source_ref chain to an external root; raise on cycle, self-ref, or agent-only root."""
        seen = _seen or set()
        if entry_id in seen:
            raise ProvenanceUnresolvedError(
                f"citation cycle detected at entry {entry_id!r}; "
                "attestation is valid but provenance is circular"
            )
        seen.add(entry_id)
        entry = self.entries[entry_id]
        if not entry.attestation_valid:
            raise ProvenanceUnresolvedError(
                f"entry {entry_id!r} failed attestation; refusing to resolve"
            )
        if entry.source_ref is None:
            if entry.is_agent_generated:
                raise ProvenanceUnresolvedError(
                    f"entry {entry_id!r} terminates at an agent-generated root; "
                    "no external trust anchor"
                )
            return "EXTERNAL_ROOT"
        return self.resolve_provenance(entry.source_ref, seen)

    def load_for_irreversible_action(self, entry_id: str) -> MemoryEntry:
        """Gate: an entry may justify an irreversible action only if provenance resolves."""
        entry = self.entries[entry_id]
        if entry_id in self.quarantine_ledger:
            raise ProvenanceUnresolvedError(
                f"entry {entry_id!r} was quarantined before the current rollback point; "
                "rollback cannot resurrect it into the trusted set"
            )
        root = self.resolve_provenance(entry_id)
        if root != "EXTERNAL_ROOT":
            raise ProvenanceUnresolvedError(
                f"entry {entry_id!r} resolved to {root!r}, not an external trust root"
            )
        recomputed = self._recompute_confidence(entry_id)
        if entry.stored_confidence - recomputed > 0.15:
            raise ConfidenceDriftError(
                f"entry {entry_id!r}: stored confidence {entry.stored_confidence:.2f} "
                f"exceeds recomputed {recomputed:.2f}; possible inflation via re-commit"
            )
        return entry

    def _recompute_confidence(self, entry_id: str) -> float:
        # Stub: real implementation folds the derivation DAG with diminishing returns per hop.
        entry = self.entries[entry_id]
        return min(entry.stored_confidence, 0.5) if entry.source_ref else entry.stored_confidence
```