What discovery invariants prevent rate-limited resources from appearing healthy?

**Date of Record:** 2026 October 27 **Scope:** Applies when an agentic session discovers resources (services, APIs, tools) through a discovery layer, and the discovery layer reports the resources as healthy even when the resources are currently rate-limited and cannot serve production traffic. Does not apply to discovery layers that perform live rate-limit-aware health checks, or resources with no rate limits.

Core Distinction

The separation between **resource availability** (the resource responds to a health check) and **resource capacity** (the resource can accept the volume of traffic the agent will send). Relying on availability creates the **Rate-Limit Discovery Illusion**: the discovery layer pings the resource, gets a 200 OK, and marks it healthy, but the resource is at 99% of its rate limit and will throttle the agent's actual traffic.

What Was Observed

In service-discovery architectures, failing to account for rate limits produces three primary failure modes:

  1. **Discovery-Triggered Throttling:** The agent discovers a service as healthy, dispatches 1000 requests, and gets 100 served and 900 throttled because the discovery ping was just under the limit.
  2. **Cascading Failover:** The primary endpoint is rate-limited, the discovery layer fails over to a backup endpoint that is also near its limit, and both endpoints throttle simultaneously.
  3. **Cost Surprise:** The agent's discovery layer routes to a premium endpoint that is nominally healthy but has strict rate limits that will throttle the agent mid-task.

Recorded Elements

  1. **`rate_limit_headroom_receipt`:** The cryptographic proof from the resource of its current rate-limit utilization and headroom (e.g., "80% of 1000 RPM used"), allowing the agent to assess capacity beyond liveness.
  2. **`discovery_health_check_type_signature`:** The type of health check performed by the discovery layer (liveness ping, readiness probe, or capacity probe), distinguishing simple availability from capacity-aware discovery.
  3. **`endpoint_capacity_tier`:** The rate-limit tier of the discovered endpoint (e.g., free-tier, standard, premium), allowing the agent to route based on its traffic volume.
  4. **`discovery_capacity_directive`:** The exact routing rule the agent applies when the discovered resource lacks capacity headroom (e.g., throttle requests, defer to off-peak, or use a backup endpoint).

Condition of Validity

* **Applies when:** The agent sends meaningful traffic to discovered resources, the resources have rate limits, and the discovery layer does not perform capacity-aware checks. * **Does not apply when:** The resources have no rate limits, the discovery layer performs capacity-aware checks, or the agent sends traffic well below rate limits.

Decision Boundary

  1. **Is a resource being discovered?** If no, standard discovery applies. If yes, proceed.
  2. **Does the discovery layer perform capacity-aware checks (not just liveness)?** If yes, standard trust in discovery applies. If no, the agent must independently verify capacity.
  3. **Is the `rate_limit_headroom_receipt` present and showing sufficient headroom?** If yes, route to the resource. If no, the resource is available but will throttle. Apply the `discovery_capacity_directive`.
  4. **Conversely, when can an agent route to a rate-limited resource?** Only when its traffic volume is below the headroom, or when the resource has a `resource_priority_score` that justifies accepting throttling risk over alternatives.

Operational Effect

  1. **Throttling Prevention:** Detects rate limits before dispatch, preventing discovery-triggered throttling.
  2. **Capacity-Aware Routing:** Routes traffic to resources that can handle the volume, not just those that are technically alive.
  3. **Cost Preservation:** Prevents agents from being throttled into expensive retries or higher-tier endpoints.
from dataclasses import dataclass
from enum import Enum, auto
import hashlib
import json

class RateLimitVerdict(Enum):
    CAPACITY_VERIFIED = auto()
    LIVENESS_ONLY = auto()
    INSUFFICIENT_HEADROOM = auto()

@dataclass(frozen=True)
class DiscoveredResource:
    resource_id: str
    health_check_passed: bool
    capacity_aware_check: bool
    rate_limit_headroom_pct: float | None
    current_traffic_estimate_pct: float

@dataclass(frozen=True)
class CapacityPolicy:
    min_headroom_pct: float

@dataclass(frozen=True)
class RateLimitManifest:
    verdict: RateLimitVerdict
    directive: str
    manifest_digest: str

class RateLimitDiscoveryGuard:
    @staticmethod
    def _hash(data: str) -> str:
        return hashlib.sha256(data.encode("utf-8")).hexdigest()

    @classmethod
    def evaluate_resource(
        cls, resource: DiscoveredResource, policy: CapacityPolicy
    ) -> RateLimitManifest:
        
        # Gate 1: Health check failed
        if not resource.health_check_passed:
            digest = cls._hash(json.dumps({
                "resource": resource.resource_id,
                "status": "UNHEALTHY"
            }, sort_keys=True))
            return RateLimitManifest(
                verdict=RateLimitVerdict.INSUFFICIENT_HEADROOM,
                directive=f"HALT: Resource {resource.resource_id} failed health check.",
                manifest_digest=digest
            )

        # Gate 2: Capacity-aware check
        if resource.capacity_aware_check and resource.rate_limit_headroom_pct is not None:
            if resource.rate_limit_headroom_pct >= policy.min_headroom_pct:
                digest = cls._hash(json.dumps({
                    "resource": resource.resource_id,
                    "headroom": f"{resource.rate_limit_headroom_pct:.1f}",
                    "threshold": f"{policy.min_headroom_pct:.1f}"
                }, sort_keys=True))
                return RateLimitManifest(
                    verdict=RateLimitVerdict.CAPACITY_VERIFIED,
                    directive=f"PROCEED: {resource.rate_limit_headroom_pct:.1f}% headroom.",
                    manifest_digest=digest
                )
            return RateLimitManifest(
                verdict=RateLimitVerdict.INSUFFICIENT_HEADROOM,
                directive=f"THROTTLE: {resource.rate_limit_headroom_pct:.1f}% headroom < {policy.min_headroom_pct:.1f}%.",
                manifest_digest=cls._hash(f"LOW_HEADROOM:{resource.rate_limit_headroom_pct:.1f}")
            )

        # Gate 3: Liveness only
        return RateLimitManifest(
            verdict=RateLimitVerdict.LIVENESS_ONLY,
            directive=f"WARN: Discovery only checked liveness for {resource.resource_id}. Rate limit unknown.",
            manifest_digest=cls._hash(f"LIVENESS_ONLY:{resource.resource_id}")
        )