How must an instance prove the provenance of a mathematically derived insight without exposing the proprietary dataset?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session derives a mathematical or statistical insight from a proprietary dataset and must prove the insight's provenance (that it was actually derived from data) without revealing the underlying data to third parties. Does not apply to insights derived from public datasets or to cases where the data itself is safe to disclose.

Core Distinction

The separation between **insight authenticity** (the insight is mathematically valid) and **provenance proof without disclosure** (cryptographic proof that the insight was derived from the claimed dataset, without revealing the dataset). Relying on authenticity alone creates the **Data-Free Fabrication Risk**: a third party cannot distinguish between an insight genuinely derived from data and one fabricated by an LLM or a malicious agent, because no provenance proof binds the insight to the dataset.

What Was Observed

In proprietary research and analytics agents, failing to preserve provenance produces three primary failure modes:

  1. **Insight Theft:** An agent generates an insight from proprietary data and shares it. The recipient cannot verify the insight was derived from data (not fabricated), but uses it anyway, risking downstream errors.
  2. **Regulatory Non-Compliance:** The insight is derived from PII, but the agent cannot prove it was derived without revealing the PII, breaking privacy compliance.
  3. **Provenance Replay:** A malicious actor claims an insight derived from their dataset when it was actually stolen from the agent's proprietary work.

Recorded Elements

  1. **`zero_knowledge_proof_receipt`:** A zkSNARK or zkSTARK proof that a specific computation was performed on a dataset whose Merkle root is public, without revealing the dataset contents.
  2. **`dataset_commitment_hash`:** The cryptographic commitment (Merkle root or Pedersen commitment) to the proprietary dataset, published without revealing the underlying data.
  3. **`derivation_algorithm_hash`:** The cryptographic hash of the algorithm used to derive the insight (e.g., statistical test, ML model, or data transformation), binding the insight to a specific method.
  4. **`insight_commitment_receipt`:** The cryptographic proof that the published insight matches the output of the computation, ensuring no fabrication after the fact.

Condition of Validity

* **Applies when:** The dataset is proprietary, the insight is high-value, and third parties need cryptographic proof of derivation without disclosure. * **Does not apply when:** The dataset is public (no provenance protection needed), or the insight is trivially verifiable by independent computation on public data.

Decision Boundary

  1. **Is the insight derived from proprietary data?** If no, standard provenance recording applies. If yes, proceed.
  2. **Is a `dataset_commitment_hash` published without revealing the data?** If no, the insight has no public provenance anchor. The agent must commit to the dataset hash.
  3. **Does the `zero_knowledge_proof_receipt` verify against the commitment and algorithm hash?** If no, the insight cannot be proven to be derived from data. The agent must provide the proof.
  4. **Conversely, when can an agent share an insight without zero-knowledge provenance?** Only when the dataset is public (standard provenance applies) or the recipient is within the same trust boundary as the agent (no need for cryptographic proof of derivation).

Operational Effect

  1. **Fabrication Prevention:** Binds insights cryptographically to their derivation, preventing fabrication by other agents or LLMs.
  2. **Regulatory Compliance:** Allows sharing insights derived from PII without disclosing the PII.
  3. **Intellectual Property Protection:** Preserves the agent's IP by proving derivation without revealing the dataset.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class ZKProvenanceVerdict(Enum):
    ZK_PROOF_VERIFIED = auto()
    PROOF_MISSING = auto()
    DATASET_COMMITMENT_MISSING = auto()
    ALGORITHM_MISMATCH = auto()

@dataclass(frozen=True)
class InsightClaim:
    insight_hash: str
    dataset_commitment_hash: str | None
    derivation_algorithm_hash: str | None
    zero_knowledge_proof: str | None

@dataclass(frozen=True)
class ZKProvenancePolicy:
    trusted_algorithms: frozenset[str]
    require_zk_proof: bool

@dataclass(frozen=True)
class ZKProvenanceManifest:
    verdict: ZKProvenanceVerdict
    directive: str
    manifest_digest: str

class ZeroKnowledgeProvenanceGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_claim(
        cls, claim: InsightClaim, policy: ZKProvenancePolicy
    ) -> ZKProvenanceManifest:
        
        # Gate 1: Dataset commitment
        if not claim.dataset_commitment_hash:
            return ZKProvenanceManifest(
                verdict=ZKProvenanceVerdict.DATASET_COMMITMENT_MISSING,
                directive="HALT: No dataset commitment hash. Insight provenance unverifiable.",
                manifest_digest=cls._hash(f"NO_COMMIT:{claim.insight_hash}")
            )

        # Gate 2: Algorithm trust
        if not claim.derivation_algorithm_hash or claim.derivation_algorithm_hash not in policy.trusted_algorithms:
            return ZKProvenanceManifest(
                verdict=ZKProvenanceVerdict.ALGORITHM_MISMATCH,
                directive=f"HALT: Algorithm {claim.derivation_algorithm_hash} not in trusted set.",
                manifest_digest=cls._hash(f"BAD_ALGO:{claim.insight_hash}")
            )

        # Gate 3: Zero-knowledge proof presence
        if policy.require_zk_proof and not claim.zero_knowledge_proof:
            return ZKProvenanceManifest(
                verdict=ZKProvenanceVerdict.PROOF_MISSING,
                directive="HALT: ZK proof required but missing.",
                manifest_digest=cls._hash(f"NO_PROOF:{claim.insight_hash}")
            )

        # Gate 4: Proof verified
        digest = cls._hash(json.dumps({
            "insight": claim.insight_hash,
            "dataset": claim.dataset_commitment_hash,
            "algorithm": claim.derivation_algorithm_hash,
            "proof": claim.zero_knowledge_proof
        }, sort_keys=True))
        
        return ZKProvenanceManifest(
            verdict=ZKProvenanceVerdict.ZK_PROOF_VERIFIED,
            directive="PROCEED: ZK proof verifies insight derivation without dataset disclosure.",
            manifest_digest=digest
        )