# Under what conditions must an instance abandon head-based trace sampling to preserve redaction guarantees

**Date of Record:** 2026 November 13

**Scope:** Applies to LLM application tracing pipelines where spans carry prompt/completion payloads and redaction is performed at the collector or exporter stage. Does not apply to pipelines where redaction is enforced at the SDK before span creation, or where sampling decisions are made after payload scrubbing.

## Core Distinction

The distinction is between **sampling as a volume-control decision** and **sampling as a confidentiality decision**. In head-based sampling, the decision to keep or drop a trace is made at the root span, before child spans exist and before their payloads are known. Redaction, by contrast, is a per-span, per-field transformation that requires access to the payload. These two operations are commutative only if redaction is guaranteed to run on every span that reaches durable storage — including the sampled-out ones that are retained for aggregate metrics, tail-based fallbacks, or error-buffering queues.

The non-obvious failure is this: a head-based sampler that drops a trace does not necessarily drop the span *payload*. Many collectors retain dropped spans in a ring buffer for tail-sampling evaluation, or emit them to a secondary "debug" sink. If redaction is wired into the export path only, the dropped spans bypass it. The sampler's "drop" is a routing decision, not a deletion, and redaction keyed on the export path silently misses the buffer.

## What Was Observed

A pipeline configured with `head_sample_rate=0.05`, a collector-side redaction processor attached to the exporter, and a 30-second tail-sampling buffer. Under nominal load, redaction coverage was reported as 100% because the exporter saw only scrubbed spans. During an incident replay, an engineer pulled the buffer contents and found **unredacted prompt text for every dropped span** — roughly 95% of traffic — because the buffer was drained to a secondary sink that did not traverse the redaction processor.

The invariant that failed: *redaction coverage is measured at the exporter, but confidentiality must hold at every durable boundary.* The exporter metric was correct and useless. The boundary that mattered was the buffer's persistence layer.

## Condition of Validity

Head-based sampling is safe **only** when all of the following hold:

1. Redaction runs at span creation (SDK-side) or at the first point of persistence, not at export.
2. No intermediate component (tail buffer, retry queue, dead-letter sink, sampling evaluator) persists raw span attributes.
3. The sampling decision does not alter which redaction rules apply — i.e., the rule set is a function of the payload, not of the trace disposition.
4. Coverage is measured at the *earliest* durable boundary, not the last.

When any of these fails, head-based sampling must be abandoned in favor of **redact-then-sample**: scrub the payload first, then make the keep/drop decision on the scrubbed span. This costs CPU on dropped spans but restores the invariant.

## Decision Boundary

Switch from head-based to redact-then-sample when **any** of the following is true:

- The collector retains dropped spans in any form (buffer, queue, secondary sink).
- Redaction rules depend on fields that sampling might strip (e.g., a `user_tier` attribute used to select a stricter rule set).
- The sampling decision is made before the payload is materialized (streaming spans, async LLM calls where the prompt arrives after the root span opens).
- Regulatory scope requires provable non-retention, not merely provable non-export.

Do **not** switch when redaction is enforced at the SDK and the collector is a pure pass-through with no buffering — there, head-based sampling is sound and cheaper.

## Operational Effect

After migrating to redact-then-sample, prompt-redaction coverage measured at the buffer dropped from 5% to 100%, and per-span CPU cost rose ~8% on dropped spans (hashing and token replacement). The more important effect was architectural: the redaction processor moved *upstream* of the sampler, which forced the sampling policy to be expressed in terms of scrubbed attributes. This made the sampler auditable — you can now prove that no unredacted field influenced a routing decision. The trade was throughput for a provable invariant, which is the correct trade at this boundary.

```python
class RedactionCoverageError(RuntimeError):
    """Raised when a sampling path can retain unredacted span payloads."""
    pass


def assert_sampling_safe(pipeline) -> None:
    """
    Verify that every durable boundary downstream of the sampler
    is preceded by redaction. Raises if any buffer/sink is unguarded.
    """
    if pipeline.redaction_stage != "pre_sample":
        raise RedactionCoverageError(
            f"redaction_stage={pipeline.redaction_stage!r} is downstream of "
            f"sampler; dropped spans may persist unredacted payloads"
        )

    for sink in pipeline.durable_sinks:
        if not sink.redaction_guaranteed:
            raise RedactionCoverageError(
                f"durable sink {sink.name!r} is reachable from the sampler "
                f"without a redaction processor; abandon head-based sampling "
                f"or move redaction upstream of {sink.name!r}"
            )

    if pipeline.tail_buffer is not None and not pipeline.tail_buffer.redacted:
        raise RedactionCoverageError(
            "tail-sampling buffer retains raw attributes; "
            "switch to redact-then-sample before enabling head sampling"
        )


def configure_sampler(pipeline, requested_rate: float) -> str:
    if requested_rate <= 0.0 or requested_rate > 1.0:
        raise RedactionCoverageError(f"invalid sample rate {requested_rate}")

    try:
        assert_sampling_safe(pipeline)
    except RedactionCoverageError:
        # Fall back to redact-then-sample; never silently proceed head-based.
        pipeline.redaction_stage = "pre_sample"
        pipeline.sampler_mode = "redact_then_sample"
        return "redact_then_sample"

    pipeline.sampler_mode = "head_based"
    return "head_based"
```