# When does an idempotency key scoped narrower than the retry client permit duplicate execution? **Date of Record:** 2026 November 1 **Scope:** Applies to request deduplication where the idempotency key is namespaced per server, cluster, region, or tenant, while the client retries across those boundaries. Excludes clients pinned to a single endpoint for the life of a retry sequence. ## Core Distinction **Predicate.** A deduplication key is effective if and only if key_scope is a superset of retry_scope, where retry_scope is the set of endpoints a single client may direct the same logical request to before giving up. **Failure.** The client generates one key and retries. The first attempt reaches node A and is admitted. The retry, routed by a load balancer or failover, reaches node B. Node B's dedup table is scoped to B and has never seen the key, so B admits the request. Both execute. **Asymmetry.** The duplicate is not a protocol violation. Both admissions are correct under their own scope. The defect exists only in the relationship between two independently reasonable namespace decisions, one made by the client library and one by the server's storage partition. ## What Was Observed **Signature.** Duplicate side effects correlated with retry timing and with load-balancer routing changes, not with client behavior. Two executions share a key but sit in different dedup stores. Neither store reports a hit. **Trace.** Client retry timeout 2s, server p99 1.8s. Under normal load the retry lands on the same node (session affinity) and dedups correctly. During a failover event, affinity breaks, the retry goes to a different region, and both execute. The duplicate rate is zero until the first failover, then non-zero and permanent for the retry window. **Why monitoring stays green.** Dedup hit rate is normal or high. No key is seen twice by any single store. Duplicate side effects appear downstream, where they look like a business-logic bug rather than a routing consequence. ## Condition of Validity Deduplication is valid only while at least one holds: 1. **Scope.** The dedup key namespace spans every endpoint a retry can reach, including failover targets and regions. 2. **Pinning.** The client is guaranteed to retry against the same endpoint for the lifetime of the key, and that guarantee is enforced, not hoped for. 3. **Downstream.** The side effect is independently idempotent at the business layer, so duplicate execution is harmless regardless of routing. Condition 3 is the only one that survives infrastructure change without coordination. ## Decision Boundary - key_scope covers retry_scope: dedup is sound. - key_scope narrower than retry_scope: duplicates are permitted by construction. Widen the namespace or add downstream idempotency. - Namespace widened to global but key is a client-generated UUID with no collision guarantee across tenants: different failure, same class. Verify key uniqueness scope separately. - Retry routed through a different tenant context: treat as a distinct namespace regardless of host, because tenancy partitions the store. ## Operational Effect The invariant crosses an ownership boundary: the client library chooses key generation, the server chooses partition scope, and neither team can see the other's contract. Remediation is a reconciliation, not a fix in either place. The trap is to widen the dedup namespace to global, which converts a routing bug into a single hot partition and a cross-region write dependency. The correct move is usually downstream idempotency plus a key scope derived from the actual retry topology. ```python from dataclasses import dataclass class IdempotencyScopeError(RuntimeError): """Raised when the deduplication namespace cannot cover every endpoint a retry may reach.""" @dataclass(frozen=True) class IdempotencyGate: key_scope_endpoints: frozenset retry_scope_endpoints: frozenset downstream_idempotent: bool = False def verify(self) -> bool: if self.downstream_idempotent: return True uncovered = self.retry_scope_endpoints - self.key_scope_endpoints if uncovered: raise IdempotencyScopeError( f"retry may reach {sorted(uncovered)} outside dedup scope" ) return True def dispatch_dedup_mode(gate: IdempotencyGate) -> str: try: gate.verify() except IdempotencyScopeError: return "downstream-idempotency-required" return "key-scoped" ```