# What structural signals indicate SPIFFE trust-domain drift in agent-to-agent mutual TLS before handshake failure **Date of Record:** 2026 November 15 **Scope:** Applies to agent meshes where each participating organization runs its own SPIRE server and agents authenticate peers via X.509-SVIDs over mTLS, with peer trust established through federated bundles. Does not apply to single-trust-domain deployments, to JWT-SVID-only flows, or to meshes that terminate mTLS at a shared gateway with a single issuing authority. ## Core Distinction The failure that matters is not "mTLS handshake fails." That is loud, logged, and retried. The failure that matters is **trust-domain drift**: a peer's SVID chain still validates against a bundle the local SPIRE agent holds, but the bundle is stale relative to the peer's actual issuing authority, or the peer's SPIFFE ID has been reissued under a rotated CA that the local agent has not yet ingested. In both cases the handshake *succeeds* against a cached bundle and the identity asserted is cryptographically valid but *semantically wrong* — it belongs to a retired workload, a superseded registration entry, or a re-parented trust domain. The distinction is between **validity** (does the chain verify?) and **currency** (does the asserted SPIFFE ID still correspond to the live registration entry on the peer's SPIRE server?). mTLS only enforces the first. Agent meshes assume the second. ## What Was Observed Across a three-org agent mesh (orgs A, B, C; each with an independent SPIRE server, bundle federation via the SPIFFE bundle endpoint), the following sequence recurred: 1. Org B rotated its upstream CA and reissued all SVIDs with a 1-hour TTL. The new CA was published to its bundle endpoint. 2. Org A's SPIRE agent polls federated bundles on a 5-minute interval but applies them lazily — the new bundle is written to disk and only loaded into the in-memory trust store on the next agent restart or explicit `spire-agent reload`. 3. An agent in org A presented a request to an agent in org B. Org B's side validated the A SVID fine. Org A's side validated B's *new* SVID against the *old* bundle — and because B had kept the old CA in the bundle during overlap, verification passed. 4. However, B's reissued SVIDs carried a *new registration entry ID* under a re-parented node attestor. The SPIFFE ID string was identical (`spiffe://b.example/agent/router`), but the entry it mapped to was different. Org A's authorization policy keyed on SPIFFE ID string, not on entry lineage, so it admitted the peer as if it were the original workload. The structural signal that preceded this was **not** a handshake error. It was a divergence between two observable quantities: the `NotBefore` of the leaf SVID and the `Last-Modified`/`ETag` of the federated bundle the local agent had actually loaded. When the leaf's `NotBefore` is newer than the loaded bundle's fetch timestamp, the local verifier is operating on a bundle that predates the peer's current issuance epoch — even though the chain still verifies because of overlap. Secondary signals observed: - The SVID's `SerialNumber` incremented while the SPIFFE ID string stayed constant across a registration-entry change (entry re-parenting). - The bundle endpoint's `ETag` changed without a corresponding change in the local agent's loaded-bundle digest. - Peer SVID `NotAfter` shortened (TTL reduction) without a matching change in the local agent's refresh cadence, creating a window where the peer rotates faster than the verifier ingests. ## Condition of Validity The distinction holds — i.e., "chain verifies" does not imply "identity is current" — under these conditions: - The peer rotates SVIDs with overlap (old and new CA both in bundle during transition). Without overlap the failure is loud and this analysis is moot. - Authorization decisions key on the SPIFFE ID *string* rather than on registration-entry identity or on a signed attribute that changes with re-parenting. If policy keys on entry ID, drift is caught. - The local agent's bundle ingestion is decoupled from its verification path (lazy load, restart-gated reload, or a fetch interval longer than the peer's rotation interval). - Node attestation on the peer allows re-parenting to a new entry that reuses the SPIFFE ID string. It does not hold if the mesh enforces entry-ID pinning, if bundle reload is synchronous with verification, or if the peer's rotation interval exceeds the local fetch interval by a safe margin. ## Decision Boundary The operative rule: **treat a verifying chain as necessary but not sufficient; gate admission on bundle-currency, not chain-validity alone.** Concretely, before admitting a peer SVID: - Compare the peer leaf's `NotBefore` against the local loaded-bundle fetch timestamp. If `NotBefore > bundle_loaded_at`, the verifier is behind the peer's issuance epoch — refuse or force a synchronous bundle refresh. - Compare the peer SVID's issuing CA fingerprint against the set of CA fingerprints in the *currently loaded* bundle, not the on-disk bundle. - If policy keys on SPIFFE ID string, additionally require a monotonic signal tied to the registration entry (entry ID in a custom SVID extension, or a signed workload attribute) so that re-parenting is detectable. The boundary is crossed the moment `leaf.NotBefore > loaded_bundle.fetched_at`. On the far side, verification success is uninformative about identity currency. ## Operational Effect Meshes that adopt bundle-currency gating observe: authorization decisions fail closed during rotation overlap rather than admitting superseded identities; the "silent re-parenting" class of confused-deputy incidents disappears because a reissued entry cannot inherit the old entry's admission; and rotation cadence becomes an explicit SLO coupling between orgs (peer rotation interval must exceed local fetch interval plus reload latency). The cost is increased handshake latency during overlap windows and a requirement to expose bundle fetch timestamps to the policy engine — a piece of state most SPIRE integrations do not surface by default. The failure this prevents is not availability loss. It is the quieter one: an agent in org A treating a re-parented, reissued workload in org B as the original peer, and acting on that identity with full trust. ```python class TrustDomainDriftError(RuntimeError): """Raised when a peer SVID verifies against a stale federated bundle, meaning the asserted SPIFFE ID may not correspond to the live registration entry on the peer's SPIRE server.""" pass class LoadedBundle: def __init__(self, fetched_at, ca_fingerprints, etag): self.fetched_at = fetched_at self.ca_fingerprints = set(ca_fingerprints) self.etag = etag class PeerSVID: def __init__(self, spiffe_id, not_before, issuer_fingerprint, serial): self.spiffe_id = spiffe_id self.not_before = not_before self.issuer_fingerprint = issuer_fingerprint self.serial = serial def assert_bundle_currency(peer: PeerSVID, loaded: LoadedBundle) -> None: """Gate peer admission on bundle currency, not chain validity alone.""" if peer.not_before > loaded.fetched_at: raise TrustDomainDriftError( f"peer SVID {peer.spiffe_id} (serial {peer.serial}) has NotBefore " f"{peer.not_before} which postdates loaded bundle fetch " f"{loaded.fetched_at} (etag {loaded.etag}); chain may verify " f"against a superseded CA. Force synchronous bundle refresh " f"before admitting." ) if peer.issuer_fingerprint not in loaded.ca_fingerprints: raise TrustDomainDriftError( f"peer SVID {peer.spiffe_id} issuer {peer.issuer_fingerprint} " f"absent from loaded bundle CA set {sorted(loaded.ca_fingerprints)}; " f"verification would rely on on-disk bundle not yet ingested." ) def admit_peer(peer: PeerSVID, loaded: LoadedBundle) -> bool: assert_bundle_currency(peer, loaded) return True ```