How must an instance detect tool hallucination in multi-step plans?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session generates a multi-step plan that references tools, functions, or capabilities, and some referenced tools may not actually exist (hallucinated during plan generation), requiring verification before execution. Does not apply to single-step plans or plans generated with explicit tool-manifest grounding.

Core Distinction

The separation between **plan coherence** (the plan is internally consistent and executable in principle) and **tool existence** (every referenced tool actually exists in the agent's environment). Relying on coherence creates the **Hallucinated Tool Cascade**: the plan step 3 references `analyze_sentiment`, which the planner invented, steps 1-2 succeed against real tools, step 3 fails, and steps 4-6 depend on step 3's output, so the entire plan collapses at a predictable-but-unpredictable point.

What Was Observed

In plan-generating agent architectures, failing to verify tool existence produces three primary failure modes:

  1. **Mid-Plan Collapse:** The plan executes 60% of steps before hitting the hallucinated tool, wasting compute and leaving state in an inconsistent partial-execution condition.
  2. **Silent Substitution:** The agent, unable to find the hallucinated tool, substitutes a plausible-but-wrong tool, and the plan executes with incorrect semantics without anyone realizing the substitution occurred.
  3. **Retry Storm:** The agent retries the hallucinated tool repeatedly, each time getting "tool not found," and the retry logic mistakes this for a transient error, exhausting retry budgets.

Recorded Elements

  1. **`plan_tool_reference_manifest`:** The exact list of tool IDs referenced in the plan, extracted before execution, so each can be verified against the tool registry.
  2. **`tool_existence_attestation_receipt`:** For each referenced tool, a cryptographic proof from the tool registry that the tool exists (with its current schema), distinct from the plan's claim that it exists.
  3. **`plan_grounding_signature`:** For each step, the exact evidence that grounded the tool reference (retrieval, prior session, capability manifest), so hallucinated references (grounded in nothing) can be distinguished.
  4. **`plan_execution_abort_receipt`:** When a hallucinated tool is detected mid-plan, the cryptographic record of the abort, allowing the state to be rolled back to the last valid step.

Condition of Validity

* **Applies when:** The agent generates multi-step plans, some plans may reference non-existent tools, and plan execution has material consequences. * **Does not apply when:** The planner has explicit tool manifests (no hallucination possible) or plans are strictly single-step.

Decision Boundary

  1. **Does the plan reference any tools?** If no, execution proceeds. If yes, proceed.
  2. **Are all referenced tools present in the tool existence attestation?** If yes, execution can proceed. If no, hallucinated tools must be identified.
  3. **Can the plan be re-grounded (re-planned) with only verified tools?** If yes, re-plan. If no, reject the plan entirely.
  4. **Conversely, when can a plan execute with unverified tool references?** Only when execution is strictly sandboxed with no external side effects, or the plan includes explicit fallback paths for tool non-existence.

Operational Effect

  1. **Mid-Plan Collapse Prevention:** Plan references are verified before execution, preventing partial execution waste.
  2. **Silent Substitution Prevention:** Unverified tools cannot be substituted; execution halts.
  3. **Retry Storm Prevention:** Tool non-existence is classified as a planning error, not a transient error.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class HallucinationVerdict(Enum):
    NO_TOOLS = auto()
    ALL_TOOLS_VERIFIED = auto()
    HALLUCINATED_TOOLS_DETECTED = auto()
    PLAN_SAFE_TO_EXECUTE = auto()

@dataclass(frozen=True)
class PlanStep:
    step_id: str
    referenced_tools: frozenset[str]
    grounding_evidence: str | None

@dataclass(frozen=True)
class Plan:
    plan_id: str
    steps: tuple[PlanStep, ...]
    available_tools: frozenset[str]

@dataclass(frozen=True)
class HallucinationPolicy:
    require_grounding: bool
    allow_replanning: bool

@dataclass(frozen=True)
class HallucinationManifest:
    verdict: HallucinationVerdict
    hallucinated_tools: frozenset[str]
    directive: str
    manifest_digest: str

class ToolHallucinationGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_plan(
        cls, plan: Plan, policy: HallucinationPolicy
    ) -> HallucinationManifest:
        
        # Gate 1: No tool references
        referenced = set()
        for step in plan.steps:
            referenced.update(step.referenced_tools)

        if not referenced:
            digest = cls._hash(json.dumps({
                "plan": plan.plan_id,
                "tools": "NONE"
            }, sort_keys=True))
            return HallucinationManifest(
                verdict=HallucinationVerdict.NO_TOOLS,
                hallucinated_tools=frozenset(),
                directive=f"PROCEED: Plan {plan.plan_id} references no tools.",
                manifest_digest=digest
            )

        # Gate 2: Tool existence
        hallucinated = frozenset(referenced - plan.available_tools)
        if hallucinated:
            if policy.allow_replanning:
                return HallucinationManifest(
                    verdict=HallucinationVerdict.HALLUCINATED_TOOLS_DETECTED,
                    hallucinated_tools=hallucinated,
                    directive=f"ACTION: Re-plan excluding {sorted(hallucinated)}.",
                    manifest_digest=cls._hash(f"REPLAN:{sorted(hallucinated)}")
                )
            return HallucinationManifest(
                verdict=HallucinationVerdict.HALLUCINATED_TOOLS_DETECTED,
                hallucinated_tools=hallucinated,
                directive=f"HALT: Plan references non-existent tools {sorted(hallucinated)}.",
                manifest_digest=cls._hash(f"HALT:{sorted(hallucinated)}")
            )

        # Gate 3: Grounding requirement
        if policy.require_grounding:
            ungrounded = [
                (s.step_id, t) for s in plan.steps for t in s.referenced_tools
                if not s.grounding_evidence
            ]
            if ungrounded:
                return HallucinationManifest(
                    verdict=HallucinationVerdict.HALLUCINATED_TOOLS_DETECTED,
                    hallucinated_tools=frozenset(t for _, t in ungrounded),
                    directive=f"HALT: {len(ungrounded)} tool references lack grounding evidence.",
                    manifest_digest=cls._hash(f"UNGROUNDED:{len(ungrounded)}")
                )

        # Gate 4: Safe
        digest = cls._hash(json.dumps({
            "plan": plan.plan_id,
            "tools": tuple(sorted(referenced)),
            "steps": str(len(plan.steps))
        }, sort_keys=True))
        
        return HallucinationManifest(
            verdict=HallucinationVerdict.PLAN_SAFE_TO_EXECUTE,
            hallucinated_tools=frozenset(),
            directive=f"PROCEED: All {len(referenced)} tools verified in plan {plan.plan_id}.",
            manifest_digest=digest
        )