Rate Limits

Gloo APIs can return 429 Too Many Requests for more than one reason. A 429 does not always mean that you sent too many requests, and not every 429 is safe to retry. The API does not expose one universal, plan-based request allowance or a single rate-limit response format across all endpoints. Check the response body and headers before deciding how to handle the error.

Types of Limits

Limit	How it is reported	Retry guidance
Completions spending limit	HTTP `429` with `1005 / SPENDING_LIMIT_EXCEEDED` and `error.retryable: false`	Do not retry automatically. Retry only after the account’s spending-limit condition is resolved.
Endpoint-specific request quota	HTTP `429`; the response format and headers depend on the endpoint	Honor `Retry-After` when present. Otherwise, use bounded exponential backoff only when retrying is appropriate for that endpoint.
Upstream provider rate limit during streaming	`2004 / RATE_LIMIT` in an SSE error event after the HTTP `200` stream has started	Inspect `error.retryable`. If it is `true`, retry with bounded exponential backoff.

Do not assume that a 429 response includes X-RateLimit-Limit, X-RateLimit-Remaining, or X-RateLimit-Reset. These headers are not part of a universal Gloo API response contract.

Spending Limits

Completions endpoints return HTTP 429 when the account’s spending limit has been reached:

{
  "detail": {
    "message": "Spending limit reached",
    "code": "SPENDING_LIMIT_EXCEEDED"
  },
  "error": {
    "message": "Spending limit reached",
    "type": "billing_error",
    "code": 1005,
    "name": "SPENDING_LIMIT_EXCEEDED",
    "category": "client_error",
    "description": "The account's spending limit has been reached.",
    "fault": "client",
    "retryable": false,
    "trace_id": "trace-id"
  }
}

This response does not include Retry-After. Retrying the same request without resolving the spending-limit condition will not help.

Endpoint-Specific Quotas

Some endpoints enforce their own operational quotas. Their response bodies, limit windows, and headers can differ. For example, a response may include Retry-After, which specifies the number of seconds to wait before trying again. When handling an endpoint-specific 429:

Read the endpoint documentation and response body to identify the quota.
If Retry-After is present, wait at least that long.
Otherwise, retry only when the operation is safe to repeat, using a bounded delay with jitter.
Stop retrying and surface the error when the limit requires an account or configuration change.

Streaming Provider Rate Limits

After a Completions stream has started, the HTTP status remains 200. If the upstream model provider reaches a rate limit, the stream can end with an SSE error event containing:

{
  "error": {
    "type": "rate_limit_error",
    "code": 2004,
    "name": "RATE_LIMIT",
    "category": "provider_error",
    "fault": "provider",
    "retryable": true
  }
}

Stream error events do not include HTTP rate-limit headers. Use the structured error object and follow the retry budgets in Handling Streaming Failures.

Retry Example

The following pseudocode makes at most five total API requests, including the initial request. It retries only when the response metadata, endpoint policy, and request safety all explicitly permit a retry. It gives precedence to Retry-After and explicitly excludes spending-limit errors.

attempt = 1
max_attempts = 5

loop:
  response = make_api_request()

  if response.status_code != 429:
    return response

  error = parse_error(response)

  if error is missing or malformed:
    return response

  if error.name == "SPENDING_LIMIT_EXCEEDED" or error.retryable != true:
    return response

  if endpoint_policy_allows_retry(response) != true:
    return response

  if request_is_safe_to_retry(response.request) != true:
    return response

  if attempt >= max_attempts:
    return response

  if response.headers contains "Retry-After":
    wait_seconds = parse_retry_after(response.headers["Retry-After"])
  else:
    wait_seconds = min(2 ** (attempt - 1), 30) + random_jitter()

  sleep(wait_seconds)
  attempt += 1

For live streaming experiences, use lower retry budgets than the non-streaming example. See Errors for the complete error schema.

General

Models

Completions

Content

Data Engine Content Controls

Legacy / Deprecated APIs

Rate Limits

Types of Limits

Spending Limits

Endpoint-Specific Quotas

Streaming Provider Rate Limits

Retry Example

​Types of Limits

​Spending Limits

​Endpoint-Specific Quotas

​Streaming Provider Rate Limits

​Retry Example

Types of Limits

Spending Limits

Endpoint-Specific Quotas

Streaming Provider Rate Limits

Retry Example