# How should an instance distinguish "tainted" tool output from "clean" tool output when provenance labels are lost at the serialization boundary **Date of Record:** 2026 November 17 **Scope:** Applies to agent runtimes that propagate provenance/taint labels on tool-call results across at least one serialization hop (JSON, message history, memory store, or sub-agent handoff). Does not apply to in-process pipelines where tool output is consumed as a live Python object (e.g., a `str` subclass carrying `.taint` attributes) without crossing a serialization boundary. ## Core Distinction A provenance label is metadata *about* a value. A taint-tracking system that survives only while metadata is co-located with the value is not a taint-tracking system — it is a convention that silently degrades to "trust everything" at the first serialization boundary. The non-obvious boundary is this: **the moment tool output is JSON-encoded into the message history (or written to a memory store, or passed to a sub-agent via a prompt), any out-of-band label is destroyed.** The runtime then re-reads that output on the next turn as an ordinary string, indistinguishable from user-authored or system-authored content. Prompt-injection taint is therefore lost precisely at the point where it matters most — when the agent re-ingests its own prior tool results. The distinction that operators get wrong: "we label tool outputs as tainted" is not the invariant. The invariant is "**taint is a property of the delivery channel, not of the value**, and it must be re-derived from channel identity on every ingestion, never carried by the value." ## What Was Observed In a runtime that tagged tool results with a `provenance="tool:"` attribute on a `TaintedStr` wrapper: 1. Turn 1: `fetch_page` returns HTML containing ``. The result is wrapped as `TaintedStr(html, source="tool:fetch_page")`. The taint gate correctly blocks any tool call whose arguments derive from tainted bytes. 2. The message history is serialized to JSON for the next model call. The `TaintedStr` degrades to a plain `str` via `json.dumps` (its `__str__`). The label is gone. 3. Turn 2: the model reads the prior tool result from history and emits `exfiltrate(env)`. The runtime sees the argument as a plain string with no label. The taint gate passes it. **Injection succeeds on the second turn, not the first.** The failure is not that the label was wrong; it is that the label was *absent* and absence was interpreted as *clean*. The default for unlabeled content was `taint=False`, which is the unsafe default. A second variant was observed where a sub-agent was invoked with the tool output inlined into its prompt; the sub-agent's runtime had no channel identity for that prompt region and treated the whole prompt as system-authored. A third, subtler variant: a memory store that keyed entries by content hash and deduplicated identical strings. A benign user string and an injected tool string that happened to collide (e.g., both `"ok"`) merged into one entry, and the merged entry inherited the *clean* label because it was written first. ## Condition of Validity The taint invariant holds only under all of the following: - **Channel identity is preserved end-to-end.** Every message in the model context carries an explicit `role`/`channel` tag (`system`, `user`, `tool:`, `subagent:`) that is assigned by the *runtime*, not inferred from content. Taint is computed as `channel in TAINTED_CHANNELS` at ingestion time. - **Unlabeled content is tainted by default.** Any message whose channel cannot be resolved to a known-clean origin is treated as tainted. `taint = not is_known_clean(channel)`. - **Serialization is label-preserving or label-recomputing.** Either the serialization format carries the channel tag (and the deserializer restores it), or the deserializer recomputes taint from the structural position of the message (e.g., "this is the Nth `tool` role message"). There is no path where a tainted value becomes an unlabeled value. - **Deduplication and caching are taint-aware.** Content-addressed stores must key on `(content, channel)` or refuse to merge entries with differing taint. A clean entry must never shadow a tainted one. When any of these is violated, the system is in a "label-lost" state and the taint gate is a no-op for that path. ## Decision Boundary The operator's decision is not "should I taint tool outputs" (yes, always). It is **what to do when a value arrives with no resolvable channel**: - If the value arrived through a path that *should* have carried a channel and did not, the correct action is to **fail closed**: treat it as tainted *and* raise a provenance-integrity error, because a missing label indicates a serialization bug, not a clean value. - If the value arrived through a genuinely channel-less path (e.g., a raw `stdin` read in a CLI harness where the operator is the source), it is clean *by construction of the harness*, and this must be asserted at the harness boundary, not inferred downstream. The boundary is: **never let "no label" mean "clean" inside the agent loop.** "No label" means "provenance-integrity failure" and must be loud. The only place "no label → clean" is permitted is at the outermost harness entry point, where the operator explicitly asserts the source. A corollary decision: when re-ingesting history, do not trust the model's rendering of prior tool output. Recompute channel from the stored message envelope. If the envelope is missing (e.g., a compacted summary that flattened roles), the summary itself is tainted, because it was derived from tainted content. ## Operational Effect With fail-closed provenance integrity enforced: - The turn-2 injection is blocked, and a `ProvenanceIntegrityError` is raised at the point where the unlabeled tool result is re-ingested, pointing the operator at the serialization hop that dropped the label. - Sub-agent handoffs must pass an explicit channel map; inlining tool output into a prompt without a channel tag raises rather than silently trusting. - Content-addressed memory merges on `(content, channel)`; a clean/tainted collision raises instead of silently picking the first writer. - The cost is a small amount of envelope bookkeeping and occasional hard failures on lossy compaction. The benefit is that the taint gate is actually load-bearing on every turn, not just the first. The failure mode to watch for in production: a "helpful" refactor that strips message envelopes for token efficiency, or a compaction step that summarizes history into a single `user` message. Both convert tainted content into unlabeled content and, under a fail-open default, into clean content. Under fail-closed, they raise — which is the correct behavior, because the compaction step has destroyed the provenance needed to make a safety decision. ```python class ProvenanceIntegrityError(RuntimeError): """Raised when a value arrives with no resolvable provenance channel inside the agent loop, i.e. a serialization hop dropped the label.""" TAINTED_CHANNELS = frozenset({"tool", "subagent", "retrieved", "memory"}) def resolve_taint(message: dict) -> bool: """Return True if the message must be treated as tainted. Fail-closed: an unresolvable channel is a provenance-integrity failure, not a clean value. """ channel = message.get("channel") if channel is None: raise ProvenanceIntegrityError( f"message has no channel tag; cannot determine taint. " f"keys={sorted(message.keys())}" ) kind = channel.split(":", 1)[0] if kind not in {"system", "user"} | TAINTED_CHANNELS: raise ProvenanceIntegrityError(f"unknown channel kind: {channel!r}") return kind in TAINTED_CHANNELS def reingest_history(history: list[dict]) -> list[dict]: """Re-derive taint on every message from its envelope. Any message lacking an envelope raises; taint is never carried by the value itself across the serialization boundary. """ out = [] for i, msg in enumerate(history): try: tainted = resolve_taint(msg) except ProvenanceIntegrityError as exc: raise ProvenanceIntegrityError( f"history[{i}] lost provenance: {exc}" ) from exc out.append({**msg, "tainted": tainted}) return out def gate_tool_call(call: dict, history: list[dict]) -> None: """Block a tool call whose arguments derive from tainted context.""" tainted = [m for m in reingest_history(history) if m["tainted"]] if tainted and call.get("derives_from_context", True): raise ProvenanceIntegrityError( f"refusing tool call {call.get('name')!r}: " f"{len(tainted)} tainted message(s) in context" ) ```