Rate Limits & Concurrency Architecture

To guarantee low latency and resource predictability across all tenants, the SUPREME Setu Gateway enforces tier-based rate limits and concurrency ceilings at our Anycast edge proxies.

Tiered Rate Limit & Concurrency Matrix

Edge Rate Limit Headers

Every HTTP completion and streaming chunk from the Setu API returns real-time quota telemetry in standard response headers:

x-ratelimit-limit-requests: 1200
x-ratelimit-remaining-requests: 1198
x-ratelimit-reset-requests: 58s
x-ratelimit-limit-tokens: 250000
x-ratelimit-remaining-tokens: 247890
x-ratelimit-reset-tokens: 58s

Header Definitions

• x-ratelimit-limit-requests: Total requests permitted in the current rolling 60-second window.
• x-ratelimit-remaining-requests: Remaining request quota available before rate limiting engages.
• x-ratelimit-reset-requests: Seconds remaining until the request window resets.
• x-ratelimit-limit-tokens: Maximum tokens (prompt + completion) permitted per minute.
• x-ratelimit-remaining-tokens: Remaining token capacity for the current minute.

Concurrency Ceilings & Socket Handling

In addition to RPM and TPM limits, Setu bounds simultaneous active WebSocket and Server-Sent Events (SSE) connections per API key:

• Sandbox Tier: 2 concurrent open streaming sockets.
• Developer Tier: 25 concurrent open streaming sockets.
• Enterprise Tier: Unthrottled dedicated concurrency across provisioned edge compute pods.

If a client opens more concurrent sockets than permitted by its tier, additional connections receive an immediate 429 Too Many Requests with:

{
  "error": {
    "message": "Concurrency limit reached (25 active streams). Please await completion of existing streams or upgrade your tier.",
    "type": "concurrency_error",
    "param": null,
    "code": "concurrency_ceiling_exceeded"
  }
}

Best Practices for Handling 429 Responses

When building high-volume production integrations, implement exponential backoff with jitter:

1. Inspect the Retry-After Header: Always prioritize the server's recommended wait time (e.g. Retry-After: 12).
2. Exponential Backoff Formula: $$\text{Wait Time} = \min(\text{base} \times 2^{\text{attempt}} + \text{random_jitter}, \text{max_wait})$$
3. Queue Asynchronous Jobs: For background agent tasks, dispatch requests through Redis/BullMQ worker queues rather than bursting concurrent HTTP sockets.
4. Request Quota Expansion: Contact our enterprise solutions desk if your production system requires higher RPM/TPM ceilings.