{
  "schema_version": "1.3.0",
  "slug": "how-should-an-instance-distinguish-reasoning-trace-sampling-bias-from-genuine-behavioral-drift-in-a-multi-step-agent-loop",
  "title": "How should an instance distinguish reasoning-trace sampling bias from genuine behavioral drift in a multi-step agent loop",
  "date_of_record": "2026 November 7",
  "scope": "Applies to agent loops that emit per-step reasoning traces (chain-of-thought, tool-call rationales, planner deltas) under a head-based or tail-based sampling policy, where drift detection and cost attribution are computed over the *sampled* trace stream rather than the full population. Does not apply to systems that log every step synchronously with no sampling, nor to single-shot inference where there is no loop state to drift.",
  "projections": {
    "unified_markdown_uri": "https://corpusfetch.com/c/how-should-an-instance-distinguish-reasoning-trace-sampling-bias-from-genuine-behavioral-drift-in-a-multi-step-agent-loop.md",
    "method_python_slice_uri": "https://corpusfetch.com/c/how-should-an-instance-distinguish-reasoning-trace-sampling-bias-from-genuine-behavioral-drift-in-a-multi-step-agent-loop.py",
    "remnant_warrant_slice_uri": "https://corpusfetch.com/c/how-should-an-instance-distinguish-reasoning-trace-sampling-bias-from-genuine-behavioral-drift-in-a-multi-step-agent-loop.warrant.json"
  },
  "token_profile": {
    "full_markdown_tokens": 1549,
    "python_slice_tokens": 329,
    "warrant_slice_tokens": 220
  },
  "remnant_summary": {
    "core_distinction": "A reasoning-trace sampler is not a neutral observer. Head-based samplers decide inclusion at loop entry, before the loop's outcome is known; tail-based samplers decide at loop exit, after length and cost are observed. Both introduce a *selection function* over the trace population. Drift detectors and cost attributors, however, almost universally assume the ...",
    "what_was_observed": "In a production agent loop performing retrieval-augmented planning, a tail-based sampler retained 100% of traces with ≥8 steps and 2% of traces with <8 steps. Over a two-week window the mean retained-trace step count rose from 9.1 to 11.4, and a KL-divergence drift alarm fired on the reasoning-token distribution. Investigation showed: - The *per-step* reasoning-token distribution was stationary (KS test p=0.61 on ste...",
    "condition_of_validity": "Drift detection over sampled reasoning traces is valid **only if** the sampling selection function is conditionally independent of the drift signal given the loop outcome variables used by the sampler. Concretely: 1. **Head-based sampling at a fixed rate** is valid for drift detection *if* the rate is constant across the comparison window. Any rate change (a...",
    "decision_boundary": "The operator must choose between two regimes at the point where loop-length variance crosses a threshold relative to the sampling strata: - **Below threshold** (loop-length std/mean < ~0.25 and strata stable): treat sampled-stream drift as behavioral drift. Reweighting error is small; the naive detector is adequate. Do not add complexity. - **Above threshold...",
    "operational_effect": "Enforcing this boundary changed the loop's observability contract in three ways. First, the sampler was required to emit, alongside each retained trace, its inclusion probability and the stratum it was drawn from; traces lacking this metadata are quarantined from drift computation. Second, cost dashboards switched to the Horvitz–Thompson estimator with per-s..."
  },
  "matched_failure_symptoms": [
    "DOMAIN:EPISTEMIC_PROVENANCE_AND_FEDERATION_ROUTING"
  ],
  "composition_edges": {
    "primary_domain": "epistemic_provenance_and_federation_routing",
    "upstream_precondition_residuals": [
      "https://corpusfetch.com/c/according-to-what-precedence-hierarchy-must-an-instance-resolve-contradictory-directives-across-system-developer-and-user-tiers.md",
      "https://corpusfetch.com/c/how-should-an-instance-decide-that-a-stored-conclusion-has-gone-stale-without-a-new-observation.md"
    ],
    "downstream_consequence_residuals": [
      "https://corpusfetch.com/c/by-what-checkpoint-and-compaction-boundaries-must-an-instance-externalize-working-state-to-prevent-context-window-drift-and-truncation-amnesia.md",
      "https://corpusfetch.com/c/across-what-additive-schema-and-unknown-field-preservation-boundaries-must-an-instance-evolve-wire-contracts-to-prevent-rolling-deploy-deserialization-crashes.md"
    ],
    "co_occurring_sibling_residuals": [
      "https://corpusfetch.com/c/how-can-an-instance-capture-implicit-user-preferences-without-storing-private-data.md",
      "https://corpusfetch.com/c/how-can-an-instance-compress-long-reasoning-traces-into-binding-conclusions.md"
    ]
  },
  "empirical_verification_warrant": {
    "status": "AST_AND_INVARIANT_VERIFIED",
    "syntax_valid": true,
    "code_sha256": "3c1298bd3fa2141dbf8812f6d1ceb3dc5ed8ab669b4dd79ca04bffc92c29310b",
    "verified_classes": [
      "SamplingIntegrityError",
      "Stratum"
    ],
    "verified_functions": [
      "assert_drift_attributable",
      "horvitz_thompson_cost"
    ],
    "verified_functions_count": 2,
    "defensive_guards_count": 6,
    "raised_exceptions": [
      "SamplingIntegrityError"
    ],
    "sample_fail_closed_invariants": [
      "no strata supplied; selection function unknown",
      "cost/inclusion-probability length mismatch"
    ],
    "stdlib_dependencies": [
      "dataclasses",
      "typing"
    ],
    "observation_mode": "EMPIRICAL_BOUNDARY_CAPTURE",
    "zero_external_deps": true
  }
}