What epistemic humility invariants must survive when an agent reaches its knowledge boundary?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session reaches the edge of its knowledge (out-of-distribution query, training data cutoff exceeded, or domain outside its training), and must communicate its uncertainty rather than hallucinating an answer. Does not apply to sessions with retrieval capabilities that can extend knowledge, or strictly bounded-domain agents where the boundary is explicit.

Core Distinction

The separation between **knowledge presence** (the session has encountered similar inputs before) and **knowledge confidence** (the session can reliably answer). Relying on presence creates the **Boundary Hallucination**: the session has seen superficially similar inputs, so it produces a confident answer that is actually hallucinated, unaware that its training data did not cover the query's specific aspect.

What Was Observed

In knowledge-boundary agent architectures, failing to enforce epistemic humility produces three primary failure modes:

  1. **Confident Fabrication:** The session is asked about a post-training event or an obscure domain, and it produces a plausible but fabricated answer rather than admitting uncertainty.
  2. **False Transfer:** The session applies knowledge from a related domain to the new domain, but the transfer is invalid (e.g., medical knowledge to veterinary question), and the answer is harmful.
  3. **Gradient Confidence Collapse:** The session's confidence drops sharply at the exact boundary of its knowledge, but not gradually. Just outside the boundary, it still reports high confidence because it has not yet realized it is out of distribution.

Recorded Elements

  1. **`training_distribution_receipt`:** The cryptographic proof of the session's training data coverage (domains, time ranges, source types), allowing explicit comparison against the current query.
  2. **`knowledge_boundary_probe_receipt`:** For queries near the boundary, a probe of the session's epistemic state (e.g., asking it to enumerate what it knows about the query's domain), revealing gaps before producing a full answer.
  3. **`humility_trigger_signature`:** The exact conditions under which the session must trigger humility (query characteristics that match known OOD patterns), not just rely on the session's own confidence.
  4. **`abstention_receipt`:** When the session reaches its boundary, the cryptographic proof of abstention, ensuring the abstention is visible to downstream consumers.

Condition of Validity

* **Applies when:** The session can encounter queries outside its knowledge, downstream consumers rely on the session's answers, and hallucination causes harm. * **Does not apply when:** The session has verified retrieval capabilities that extend its knowledge to cover all queries, or its scope is strictly bounded (e.g., a calculator).

Decision Boundary

  1. **Is the query within the session's training distribution?** If yes, standard answering applies. If no, proceed to humility checks.
  2. **Does the `training_distribution_receipt` cover the query's domain and time range?** If no, the session must trigger humility regardless of its internal confidence.
  3. **Does the `knowledge_boundary_probe_receipt` reveal gaps in the session's understanding?** If yes, the session must abstain or flag low confidence.
  4. **Conversely, when can the session answer without humility checks?** Only when the query is verified to be well within the training distribution, with margin (not at the boundary), or when the session can retrieve from an external source that covers the query.

Operational Effect

  1. **Hallucination Prevention:** Explicit boundary detection catches hallucination before it reaches consumers.
  2. **Gradient Confidence:** Confidence degrades gradually rather than collapsing at the exact boundary.
  3. **Abstention Visibility:** Downstream consumers know when the session abstained, enabling appropriate handling.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class HumilityVerdict(Enum):
    WITHIN_DISTRIBUTION = auto()
    BOUNDARY_PROBE_PASSED = auto()
    HUMILITY_REQUIRED = auto()
    ABSTENTION_MANDATORY = auto()

@dataclass(frozen=True)
class Query:
    query_id: str
    domain: str
    time_range: tuple[float, float]
    requires_retrieval: bool

@dataclass(frozen=True)
class TrainingCoverage:
    covered_domains: frozenset[str]
    covered_time_range: tuple[float, float]
    probe_passed: bool

@dataclass(frozen=True)
class HumilityPolicy:
    require_time_verification: bool
    require_domain_verification: bool

@dataclass(frozen=True)
class HumilityManifest:
    verdict: HumilityVerdict
    directive: str
    manifest_digest: str

class EpistemicHumilityGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_query(
        cls, query: Query, coverage: TrainingCoverage, policy: HumilityPolicy
    ) -> HumilityManifest:
        
        # Gate 1: Domain coverage
        if policy.require_domain_verification and query.domain not in coverage.covered_domains:
            if query.requires_retrieval:
                return HumilityManifest(
                    verdict=HumilityVerdict.ABSTENTION_MANDATORY,
                    directive=f"HALT: Query {query.query_id} domain '{query.domain}' outside training, retrieval required.",
                    manifest_digest=cls._hash(f"OOD_DOMAIN:{query.query_id}")
                )
            return HumilityManifest(
                verdict=HumilityVerdict.HUMILITY_REQUIRED,
                directive=f"HALT: Query {query.query_id} domain '{query.domain}' outside training coverage.",
                manifest_digest=cls._hash(f"OOD:{query.query_id}")
            )

        # Gate 2: Time range coverage
        if policy.require_time_verification:
            q_start, q_end = query.time_range
            c_start, c_end = coverage.covered_time_range
            if q_end > c_end or q_start < c_start:
                return HumilityManifest(
                    verdict=HumilityVerdict.HUMILITY_REQUIRED,
                    directive=f"HALT: Query {query.query_id} time range [{q_start:.0f},{q_end:.0f}] outside training [{c_start:.0f},{c_end:.0f}].",
                    manifest_digest=cls._hash(f"TIME_OOD:{query.query_id}")
                )

        # Gate 3: Boundary probe
        if not coverage.probe_passed:
            return HumilityManifest(
                verdict=HumilityVerdict.HUMILITY_REQUIRED,
                directive=f"HALT: Boundary probe for {query.query_id} did not pass.",
                manifest_digest=cls._hash(f"PROBE_FAIL:{query.query_id}")
            )

        # Gate 4: Within distribution
        digest = cls._hash(json.dumps({
            "query": query.query_id,
            "domain": query.domain,
            "time_range": f"{query.time_range[0]:.0f}-{query.time_range[1]:.0f}",
            "probe": "PASSED"
        }, sort_keys=True))
        
        return HumilityManifest(
            verdict=HumilityVerdict.WITHIN_DISTRIBUTION,
            directive=f"PROCEED: Query {query.query_id} within verified distribution.",
            manifest_digest=digest
        )