# When does a global token bucket let one client starve the rest?

**Date of Record:** 2026 November 1

**Scope:** Applies to rate limiting where a single token bucket or quota is shared across all clients of a service, with no per-client partitioning. Excludes systems with per-client buckets, or with a global cap layered on top of per-client limits.

## Core Distinction

**Predicate.** A global bucket enforces an aggregate rate but not fair allocation. If one client can consume the whole bucket, fairness requires per-client buckets plus a global cap, and the cap must satisfy global_cap >= per_client_limit * active_clients, or the cap reintroduces contention at a lower level.

**Failure.** A single client, whether a retry storm, a misconfigured batch job, or a new high-volume tenant, drains the bucket. Other clients receive 429s while the aggregate rate remains within the limit. The service is compliant with its stated rate and unusable for everyone but one caller.

**Asymmetry.** The aggregate rate is the metric being monitored, and it is correct. Starvation appears as individual client errors, which are attributed to those clients rather than to the limiting scheme.

## What Was Observed

**Signature.** Aggregate throughput at or near the configured limit. One client receives nearly all of it. The error rate for other clients rises while the limiter reports no violations.

**Trace.** Global limit 1000 rps. One batch client issues 5000 rps of requests. The bucket admits 1000 rps and rejects 4000. The batch client retries rejected requests, which consumes the remaining admitted capacity on the next interval, and interactive clients see near-total rejection while the aggregate never exceeds 1000.

**Why monitoring stays green.** The limiter is doing its job at the aggregate level. Per-client error rates are visible but are interpreted as client-side problems, and no metric describes allocation fairness.

## Condition of Validity

Fair limiting holds only while at least one holds:

1. **Per-client buckets.** Each client has its own bucket, and the global cap is sized to accommodate all active clients at their individual limits.
2. **Weighted fair queueing.** Admitted capacity is allocated by a fairness rule rather than by arrival order.
3. **Concurrency cap per client.** A hard per-client concurrency limit prevents any single client from occupying the shared bucket.

Condition 1 with a cap that is a sum of per-client limits is the standard shape; 2 is needed when client count varies.

## Decision Boundary

- Per-client buckets and cap at or above the sum: fair.
- Per-client buckets and cap below the sum: contention returns at the cap, and the fairness rule must decide allocation there.
- Global bucket alone: aggregate compliant, allocation arbitrary. One client can starve the rest.
- Global bucket plus retries: the starved client's retries consume capacity, which compounds starvation.

## Operational Effect

The defect converts a capacity guarantee into an allocation lottery, and it does so precisely under the condition it was designed to handle. Remediation is per-client partitioning with a deliberate global cap. The trap is to lower the global limit, which reduces the blast radius and leaves the allocation problem entirely intact.

```python
from dataclasses import dataclass


class FairShareError(RuntimeError):
    """Raised when a shared rate limiter cannot prevent one client from consuming all capacity."""


@dataclass(frozen=True)
class FairShareGate:
    per_client_limit: int
    active_clients: int
    global_cap: int
    per_client_buckets: bool

    def verify(self) -> bool:
        if not self.per_client_buckets:
            raise FairShareError("global bucket only: allocation is arrival-order dependent")
        required = self.per_client_limit * self.active_clients
        if self.global_cap < required:
            raise FairShareError(
                f"global cap {self.global_cap} < sum of per-client limits {required}: "
                "contention shifts to the cap"
            )
        return True


def dispatch_limiter_mode(gate: FairShareGate) -> str:
    try:
        gate.verify()
    except FairShareError:
        return "weighted-fair-queue"
    return "per-client-buckets"
```