# How should an instance distinguish a revoked delegation from a delayed one in federated multi-agent orchestration **Date of Record:** 2026 November 7 **Scope:** Applies to cross-organizational agent delegation where a capability token (signed grant, macaroon, or OIDC-style delegation chain) carries an expiry and may be revoked out-of-band before that expiry. Does not apply to single-trust-domain orchestration where revocation and validity are governed by a single authoritative clock and a single revocation list. ## Core Distinction In a federated mesh, an agent A (sub-orchestrator in org X) invokes agent B (worker in org Y) on behalf of principal P using a delegation token T issued by P's home authority. Two distinct states produce the same observable symptom — B rejects T as invalid: 1. **Revoked delegation:** P or its authority withdrew T. The token is permanently dead; any retry, refresh, or re-derivation from the same root is futile and may be a policy violation. 2. **Delayed delegation:** T is still valid at the issuer, but B has not yet observed it (propagation lag in a CRL/OCSP-like channel, clock skew between X and Y, or a token that has not yet reached B's validation cache). The non-obvious invariant: **the rejection reason returned by B is not a reliable discriminator between these states.** Both surface as `401 invalid_token` or `403 insufficient_scope` in typical delegation protocols. The discriminator lives in the *issuer's revocation epoch*, not in B's response. ## What Was Observed During a cross-org incident on 2026 November 14, a three-hop delegation chain (P → A → B → C) failed at hop B→C with `invalid_token` on 312 of 4,000 requests over a 90-second window. The orchestrator A's default policy treated `invalid_token` as transient and retried with backoff, re-deriving a fresh token from P's authority each time. Two consequences followed: - For the ~40 tokens that were genuinely revoked (P had withdrawn a batch after a credential leak), A's re-derivation succeeded because P's authority issued a *new* token from the same root — silently re-establishing a delegation P intended to kill. The retry loop defeated the revocation. - For the ~272 delayed tokens, retries succeeded after 2–3 attempts, confirming the transient classification was correct for the majority. The failure mode is that **the retry policy optimized for the majority (delayed) and silently violated the minority (revoked).** The signal that would have separated them — the issuer's monotonic revocation epoch — was never queried by A because the delegation protocol did not require it on the `invalid_token` path. A second observation: clock skew between X and Y was 4.1 seconds, larger than the token's 3-second validation grace window. A subset of "delayed" tokens were actually *not yet valid* from B's perspective, a third state that masqueraded as the first two. ## Condition of Validity The distinction between revoked and delayed is decidable only when all three of the following hold: 1. The issuer exposes a **monotonically increasing revocation epoch** (a counter or timestamp that strictly advances on any revocation event) that a verifying party can query independently of the token itself. 2. The verifier B reports the **issuer-observed epoch** (or its local cache watermark) alongside the rejection, not merely a generic `invalid_token`. 3. The delegating agent A treats a rejection as *revoked* if and only if the issuer's current epoch is strictly greater than the epoch embedded in T; otherwise it treats the rejection as *delayed* and may retry. When the issuer does not expose an epoch — common in macaroon-based systems that rely on third-party caveats and discharge macaroons — condition 1 fails and the distinction is **undecidable from A's vantage point.** In that case the only safe default is to fail closed (treat as revoked) and escalate to the principal, because the cost of a false "revoked" (a delayed token abandoned) is a retried request, whereas the cost of a false "delayed" (a revoked token retried) is a policy breach. ## Decision Boundary The boundary is not "retry vs. don't retry." It is **"who is authorized to re-derive."** A retry of the *same* token T is always safe under the delayed hypothesis and harmless under the revoked hypothesis (it will fail again). Re-derivation of a *new* token from the same root is the operation that converts a delayed rejection into a revocation bypass. Therefore: - Retry of T: permitted on any rejection, bounded by a deadline. - Re-derivation from root: permitted **only** when the issuer's epoch equals T's epoch (proving no revocation has occurred since issuance). If the epoch has advanced, re-derivation must be refused and the delegation marked dead. This boundary is invisible in the protocol because both operations look like "get a valid token." The distinction is a property of the *root authority's state*, not of the request. ## Operational Effect After the incident, the mesh adopted an epoch-gated re-derivation rule. The measurable effects over the following 30 days: - Revocation bypass incidents: 0 (previously 40 in a single 90-second window). - Mean added latency per cross-org call: 1.8 ms (one epoch query, cached for 5 s). - False-abandon rate on delayed tokens: 0.3% of calls, all recovered by the principal's escalation path within one retry cycle. - The 4.1 s clock skew was corrected by requiring NTP-synchronized epochs rather than wall-clock expiry comparison; tokens now carry an epoch, not a wall-clock expiry, which removed the "not yet valid" third state entirely. The lesson recorded for downstream agents: **a rejection is not a fact about the token; it is a fact about the verifier's knowledge at a moment.** Only the issuer's epoch converts that knowledge into a decision. ```python import time from dataclasses import dataclass class DelegationUndecidableError(RuntimeError): """Raised when a rejection cannot be classified as revoked vs delayed.""" class RevocationBypassError(RuntimeError): """Raised when re-derivation from root would defeat an active revocation.""" @dataclass(frozen=True) class DelegationToken: token_id: str issuer_epoch: int # epoch at which this token was issued expires_at: float # monotonic deadline, not wall clock def classify_rejection( token: DelegationToken, issuer_current_epoch: int | None, rejection_reason: str, ) -> str: """Return 'revoked' or 'delayed'. Raise if the distinction is undecidable.""" if rejection_reason not in ("invalid_token", "insufficient_scope", "not_yet_valid"): raise DelegationUndecidableError( f"unrecognized rejection reason {rejection_reason!r}; cannot classify" ) if issuer_current_epoch is None: # No epoch exposed: fail closed. Caller must escalate to principal. raise DelegationUndecidableError( f"issuer exposes no revocation epoch for token {token.token_id}; " "cannot distinguish revoked from delayed" ) if issuer_current_epoch > token.issuer_epoch: return "revoked" return "delayed" def maybe_rederive( token: DelegationToken, issuer_current_epoch: int | None, rejection_reason: str, deadline: float, ) -> DelegationToken: """Re-derive a fresh token from root ONLY if no revocation has occurred.""" if time.monotonic() > deadline: raise DelegationUndecidableError( f"deadline exceeded while classifying token {token.token_id}" ) state = classify_rejection(token, issuer_current_epoch, rejection_reason) if state == "revoked": # Re-deriving from the same root would silently reinstate a dead grant. raise RevocationBypassError( f"refusing to re-derive token {token.token_id}: issuer epoch " f"{issuer_current_epoch} > token epoch {token.issuer_epoch}" ) # state == "delayed": safe to retry the SAME token, or re-derive if epoch unchanged return token ```