# What structural signals indicate a rewrite operator has invalidated a retained span under self-editing context policies

**Date of Record:** 2026 November 11

**Scope:** Applies to agentic context managers where a model-driven operator both prunes (drops) and rewrites (mutates) spans of a shared context buffer, and where later stages (retrieval, tool calls, output assembly) hold references into that buffer. Does not apply to append-only context logs, nor to systems where pruning and rewriting are applied to disjoint, statically partitioned regions.

## Core Distinction

The failure boundary is **not** "was the span pruned?" but "was the span's *identity* preserved while its *content* changed underneath a holder that assumed content stability?" A pruning operator is observable: the span disappears and any dereference fails loudly. A rewrite operator is silent: the span keeps its ID, its position, and its declared type, but its bytes now mean something else — or nothing.

The non-obvious invariant: **a retained span ID is not a stability token.** Model-driven rewrite operators (summarize, normalize, redact, canonicalize, compress) preserve the span handle precisely so downstream references survive. That is their design intent. The consequence is that any consumer caching a derived artifact keyed on span ID — an embedding, a token offset map, a citation, a tool-argument fragment — is now stale with no signal that staleness occurred. Pruning produces errors; rewriting produces *plausible wrong answers*.

## What Was Observed

In a pipeline where a planning agent rewrote the "current task" span on every turn (compressing prior turns into a rolling summary) and a retrieval stage held a token-offset map into that same span:

- After a rewrite, the retrieval stage continued to slice the span at the old offsets. Offsets past the new, shorter length returned truncated or empty fragments. Offsets within range returned text from a *different* sentence than the one originally indexed.
- The rewrite was marked in the buffer as `op=rewrite`, `span_id` unchanged, `version` incremented — but no consumer subscribed to `version`. The version counter existed and was never checked.
- The failure surfaced as a citation pointing at the wrong paragraph, not as an exception. Three turns of downstream reasoning were built on the misaligned span before a human noticed the citation drift.

Structural signals that, in retrospect, reliably indicated invalidation:

1. **Length delta with stable ID.** A span whose `len(content)` changed while `span_id` was constant. This is the single highest-precision signal. Pruning changes the set of IDs; rewriting changes lengths at fixed IDs.
2. **Version monotonicity at a fixed ID.** `version` strictly increased while the ID was retained. A consumer that never reads `version` is by construction unsafe.
3. **Content-hash divergence at a fixed ID.** A cheap content digest stored at index time and re-checked before use. Any mismatch means every derived artifact (embedding, offset map, citation) is invalid.
4. **Operator provenance change.** The span's `produced_by` field flipped from `ingest` to `rewrite:summarize`. Derived artifacts built against an `ingest`-provenance span are not valid against a `rewrite`-provenance span even if the text is similar.
5. **Offset-map boundary crossing.** Any cached offset whose end exceeds the current span length, or whose start lands mid-token after re-tokenization, is a structural invalidation regardless of content hash (tokenizers are not stable across rewrites).

## Condition of Validity

The safe pattern holds only under a **content-addressed** contract: consumers key derived artifacts on `(span_id, version, content_hash)`, and the buffer guarantees that `version` increments on every mutation and that `content_hash` is recomputed post-rewrite. Under that contract, a stale consumer detects the mismatch and re-derives.

The contract fails — and this record applies — when either condition is absent:

- The rewrite operator updates content but does not bump `version` (common when the operator writes through a view that aliases the original object).
- The consumer caches on `span_id` alone (the default in most context-buffer SDKs, because append-only history made ID sufficient for years).

If the buffer is append-only, or if rewrite and prune act on disjoint regions with no shared holders, ID-keyed caching is sound and this boundary does not bind.

## Decision Boundary

At the moment a rewrite operator is about to mutate a span that has any live holder, decide:

- **Invalidate-and-rekey** if the holder's derived artifact is cheap to rebuild (offset maps, token counts): bump `version`, recompute `content_hash`, force re-derivation. This is the default.
- **Copy-on-rewrite** if the holder's artifact is expensive (embeddings, external index entries): emit a *new* span ID for the rewritten content, leave the old span tombstoned but readable, and let holders keep their now-frozen reference. Rewriting then becomes an append of a derived span plus a tombstone, and the silent-mutation class disappears entirely.
- **Never** mutate in place while a holder exists and rely on the holder to "notice." Model-driven operators have no visibility into holder state; the buffer must enforce it.

The boundary is crossed the instant a rewrite preserves `span_id` without incrementing `version` and recomputing `content_hash`. Everything downstream is a correctness hazard, not a performance one.

## Operational Effect

Adopting copy-on-rewrite plus content-addressed rekeying converts the silent-mutation failure class into a loud, localized one: stale holders fail a hash check at use time and re-derive, and the misaligned-citation symptom disappears. The cost is buffer growth from tombstoned spans, bounded by a compaction pass that GCs tombstoned spans only when no live holder references them — which is itself a pruning decision and must be gated on holder liveness, not on age.

The invariant to carry forward: **a retained span ID is a stability token only if `version` and `content_hash` are checked at every use.** Absent that check, treat every rewrite as a silent invalidation of every derived artifact keyed on that ID.

```python
import hashlib
from dataclasses import dataclass, field


class StaleSpanError(RuntimeError):
    """Raised when a derived artifact is used against a span whose content changed."""


@dataclass
class Span:
    span_id: str
    content: str
    version: int = 0
    content_hash: str = field(default="")

    def __post_init__(self):
        self.content_hash = self._digest(self.content)

    @staticmethod
    def _digest(text: str) -> str:
        return hashlib.sha256(text.encode("utf-8")).hexdigest()


@dataclass
class DerivedArtifact:
    span_id: str
    version: int
    content_hash: str
    offsets: tuple  # (start, end) token offsets into the span


class ContextBuffer:
    def __init__(self):
        self._spans: dict[str, Span] = {}

    def add(self, span: Span) -> None:
        self._spans[span.span_id] = span

    def rewrite(self, span_id: str, new_content: str, op: str) -> Span:
        span = self._spans[span_id]
        # Copy-on-rewrite: new ID, old span tombstoned but readable.
        new_span = Span(span_id=f"{span_id}@{op}:{span.version + 1}", content=new_content)
        new_span.version = span.version + 1
        self._spans[new_span.span_id] = new_span
        return new_span

    def derive(self, span_id: str, offsets: tuple) -> DerivedArtifact:
        span = self._spans[span_id]
        return DerivedArtifact(span.span_id, span.version, span.content_hash, offsets)

    def use(self, artifact: DerivedArtifact, span_id: str) -> str:
        span = self._spans.get(span_id)
        if span is None:
            raise StaleSpanError(f"span {span_id} no longer present")
        if span.version != artifact.version or span.content_hash != artifact.content_hash:
            raise StaleSpanError(
                f"artifact built at version {artifact.version}/{artifact.content_hash[:8]} "
                f"but span {span_id} is now version {span.version}/{span.content_hash[:8]}"
            )
        start, end = artifact.offsets
        if end > len(span.content):
            raise StaleSpanError(
                f"offsets {artifact.offsets} exceed current length {len(span.content)}"
            )
        return span.content[start:end]


def safe_use(buffer: ContextBuffer, artifact: DerivedArtifact, span_id: str) -> str:
    # Guard against in-place mutation sneaking past copy-on-rewrite.
    span = buffer._spans.get(span_id)
    if span is not None and span.content_hash != artifact.content_hash:
        raise StaleSpanError(
            f"content hash divergence at {span_id}: span mutated in place"
        )
    return buffer.use(artifact, span_id)
```