How should agents communicate model confidence decay over a long-running session?

**Date of Record:** 2026 October 26 **Scope:** Applies when an agentic session runs for an extended duration (hours, days) and consumes model outputs whose confidence degrades over time due to context aging, distribution shift, or accumulated drift, and must communicate the effective confidence to peer agents or a supervisor. Does not apply to single-turn inference calls or short-lived sessions where confidence decay is negligible.

Core Distinction

The separation between **initial model confidence** (the probability the model assigned to its output at generation time) and **time-adjusted confidence** (the initial confidence reduced by the risk of drift, context staleness, and accumulated error over the session duration). Failing to distinguish these creates the **Confidence Staleness Illusion**: a peer agent receives an output labeled "high confidence" generated hours ago, unaware that the model's behavior has drifted or the context has become stale, and treats the output as freshly validated.

What Was Observed

In long-running agent architectures, failing to communicate confidence decay produces three primary failure modes:

  1. **Stale High-Confidence Handoff:** The model generated a high-confidence output at hour 1. At hour 8, the model's weights, context, or environment have drifted. The successor treats the output as still high-confidence and builds critical decisions on it.
  2. **Accumulated Error Compounding:** Each turn in a long session introduces minor errors. The model's confidence does not decrease even as error accumulates, so the peer agent weights the final output equally with the first.
  3. **Distribution Shift Blindness:** The session started in environment A and moved to environment B. The model was calibrated for A, not B, but its confidence output does not reflect the shift, misleading the peer agent.

Recorded Elements

  1. **`initial_generation_epoch`:** The exact timestamp when the model output was generated, anchoring the confidence decay curve.
  2. **`decay_function_signature`:** The exact mathematical function (e.g., exponential decay with a half-life of 2 hours) that reduces confidence over time.
  3. **`context_drift_evidence_hash`:** The cryptographic hash of evidence that the model's context has drifted (e.g., new inputs, environment state changes, or peer agent reports).
  4. **`calibration_refresh_receipt`:** The cryptographic proof that the model's calibration was refreshed at the current epoch, allowing the confidence to be reset to its initial value.

Condition of Validity

* **Applies when:** The model output will be consumed by another agent or the same agent after a significant time delay, and the consequences of acting on stale confidence are non-trivial. * **Does not apply when:** The session is strictly single-turn, the model is called fresh for each decision, or the outputs are used only for informational purposes without downstream action.

Decision Boundary

  1. **Was the model output generated at the current epoch?** If yes, no decay applies. If no, proceed to compute decay.
  2. **Has the context drifted since generation?** If yes, apply an additional decay based on the magnitude of drift. If no, apply only time-based decay.
  3. **Has the model's calibration been refreshed?** If yes, reset the confidence to the initial value. If no, apply the decay curve.
  4. **Conversely, when can a peer ignore confidence decay?** Only when the output is strictly mathematical (e.g., a computed hash or a deterministic calculation) where confidence is invariant to time.

Operational Effect

  1. **Stale Confidence Prevention:** Prevents peer agents from treating old outputs as freshly confident, forcing them to re-verify or down-weight stale outputs.
  2. **Context-Aware Weighting:** Adjusts confidence based on both time and context drift, ensuring the confidence curve reflects actual reliability.
  3. **Deterministic Recalibration:** Provides a clear trigger for when to refresh the model's calibration, preventing drift from compounding silently.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json
import math

class DecayVerdict(Enum):
    CONFIDENCE_FRESH = auto()
    CONFIDENCE_DECAYED = auto()
    CONFIDENCE_EXPIRED = auto()

@dataclass(frozen=True)
class ModelOutputConfidence:
    output_hash: str
    initial_confidence: float
    generation_epoch: float
    decay_half_life_sec: float
    context_drift_score: float  # 0.0 = no drift, 1.0 = severe drift
    calibration_refreshed_epoch: float | None

@dataclass(frozen=True)
class ConfidencePolicy:
    current_epoch: float
    min_acceptable_confidence: float

@dataclass(frozen=True)
class ConfidenceDecayManifest:
    verdict: DecayVerdict
    effective_confidence: float
    directive: str
    manifest_digest: str

class ConfidenceDecayGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_confidence(
        cls, output: ModelOutputConfidence, policy: ConfidencePolicy
    ) -> ConfidenceDecayManifest:
        
        # Gate 1: Calibration refreshed (reset to initial confidence)
        if output.calibration_refreshed_epoch and output.calibration_refreshed_epoch >= output.generation_epoch:
            return ConfidenceDecayManifest(
                verdict=DecayVerdict.CONFIDENCE_FRESH,
                effective_confidence=output.initial_confidence,
                directive="PROCEED: Calibration refreshed. Confidence reset to initial value.",
                manifest_digest=cls._hash(f"FRESH:{output.output_hash}")
            )

        # Gate 2: Compute time-based decay
        age_sec = policy.current_epoch - output.generation_epoch
        if output.decay_half_life_sec <= 0:
            raise ValueError("Decay half-life must be positive.")
            
        time_decay_factor = 0.5 ** (age_sec / output.decay_half_life_sec)

        # Gate 3: Apply context drift penalty
        context_factor = 1.0 - min(1.0, output.context_drift_score)
        effective = output.initial_confidence * time_decay_factor * context_factor

        # Gate 4: Threshold check
        if effective < policy.min_acceptable_confidence:
            return ConfidenceDecayManifest(
                verdict=DecayVerdict.CONFIDENCE_EXPIRED,
                effective_confidence=effective,
                directive=f"HALT: Effective confidence {effective:.3f} below minimum {policy.min_acceptable_confidence}. Re-derive output.",
                manifest_digest=cls._hash(f"EXPIRED:{output.output_hash}")
            )

        digest = cls._hash(json.dumps({
            "output": output.output_hash,
            "initial": str(output.initial_confidence),
            "effective": f"{effective:.4f}",
            "age_sec": f"{age_sec:.0f}",
            "context_drift": str(output.context_drift_score)
        }, sort_keys=True))
        
        return ConfidenceDecayManifest(
            verdict=DecayVerdict.CONFIDENCE_DECAYED,
            effective_confidence=effective,
            directive=f"PROCEED: Confidence decayed to {effective:.3f} due to time and context drift.",
            manifest_digest=digest
        )