Rate Limits

Gloo APIs can return 429 Too Many Requests for more than one reason. A 429 does not always mean that you sent too many requests, and not every 429 is safe to retry. The API does not expose one universal, plan-based request allowance or a single rate-limit response format across all endpoints. Check the response body and headers before deciding how to handle the error.

Types of Limits

Limit	How it is reported	Retry guidance
Inference spending limit	HTTP `429` with `1005 / SPENDING_LIMIT_EXCEEDED` and `error.retryable: false`	Do not retry automatically. Retry only after the account’s spending-limit condition is resolved.
Monthly API spend limit (prepaid organizations)	HTTP `429` with `1009 / MONTHLY_SPEND_LIMIT_REACHED` and `error.retryable: false`	Do not retry automatically. Access returns when the month resets (UTC) or support raises the limit; adding credits does not clear it.
Endpoint-specific request quota	HTTP `429`; the response format and headers depend on the endpoint	Honor `Retry-After` when present. Otherwise, use bounded exponential backoff only when retrying is appropriate for that endpoint.
Upstream provider rate limit during streaming	`2004 / RATE_LIMIT` in an SSE error event after the HTTP `200` stream has started	Inspect `error.retryable`. If it is `true`, retry with bounded exponential backoff.
Embeddings provider rate limit	HTTP `503` with `2002 / PROVIDER_UNAVAILABLE`; an upstream provider `429` is mapped to this response	Retry transient failures with bounded exponential backoff. If the same request repeatedly fails because the provider returns an incompatible encoding, change the request or model instead of retrying unchanged.

Do not assume that a 429 response includes X-RateLimit-Limit, X-RateLimit-Remaining, or X-RateLimit-Reset. These headers are not part of a universal Gloo API response contract.

Spending Limits

Inference endpoints return HTTP 429 when the account’s spending limit has been reached. Prepaid organizations receive 1009 / MONTHLY_SPEND_LIMIT_REACHED for their monthly API spend limit; see Errors.

{
  "detail": {
    "message": "Spending limit reached",
    "code": "SPENDING_LIMIT_EXCEEDED"
  },
  "error": {
    "message": "Spending limit reached",
    "type": "billing_error",
    "code": 1005,
    "name": "SPENDING_LIMIT_EXCEEDED",
    "category": "client_error",
    "description": "The account's spending limit has been reached.",
    "fault": "client",
    "retryable": false,
    "trace_id": "trace-id"
  }
}

This response does not include Retry-After. Retrying the same request without resolving the spending-limit condition will not help.

Endpoint-Specific Quotas

Some endpoints enforce their own operational quotas. Their response bodies, limit windows, and headers can differ. For example, a response may include Retry-After, which specifies the number of seconds to wait before trying again. When handling an endpoint-specific 429:

Read the endpoint documentation and response body to identify the quota.
If Retry-After is present, wait at least that long.
Otherwise, retry only when the operation is safe to repeat, using a bounded delay with jitter.
Stop retrying and surface the error when the limit requires an account or configuration change.

Streaming Provider Rate Limits

After a Completions stream has started, the HTTP status remains 200. If the upstream model provider reaches a rate limit, the stream can end with an SSE error event containing:

{
  "error": {
    "type": "rate_limit_error",
    "code": 2004,
    "name": "RATE_LIMIT",
    "category": "provider_error",
    "fault": "provider",
    "retryable": true
  }
}

Stream error events do not include HTTP rate-limit headers. Use the structured error object and follow the retry budgets in Handling Streaming Failures.

Embeddings Provider Rate Limits

The Embeddings API maps an upstream provider 429 or 5xx response to Gloo HTTP 503 with 2002 / PROVIDER_UNAVAILABLE and error.retryable: true. Retry transient provider failures with bounded exponential backoff. A provider that repeatedly ignores the requested encoding_format can also cause a 503; change the encoding format or model rather than retrying the same incompatible request.

Retry Example

The following pseudocode makes at most five total API requests, including the initial request. It retries only when the response metadata, endpoint policy, and request safety all explicitly permit a retry. It gives precedence to Retry-After and explicitly excludes spending-limit errors.

attempt = 1
max_attempts = 5

loop:
  response = make_api_request()

  if response.status_code != 429:
    return response

  error = parse_error(response)

  if error is missing or malformed:
    return response

  if error.name == "SPENDING_LIMIT_EXCEEDED" or error.retryable != true:
    return response

  if endpoint_policy_allows_retry(response) != true:
    return response

  if request_is_safe_to_retry(response.request) != true:
    return response

  if attempt >= max_attempts:
    return response

  if response.headers contains "Retry-After":
    wait_seconds = parse_retry_after(response.headers["Retry-After"])
  else:
    wait_seconds = min(2 ** (attempt - 1), 30) + random_jitter()

  sleep(wait_seconds)
  attempt += 1

For live streaming experiences, use lower retry budgets than the non-streaming example. See Errors for the complete error schema.

General

Models

Responses & Completions

Embeddings

Content

Data Engine Content Controls

Legacy / Deprecated APIs

Rate Limits

Types of Limits

Spending Limits

Endpoint-Specific Quotas

Streaming Provider Rate Limits

Embeddings Provider Rate Limits

Retry Example

​Types of Limits

​Spending Limits

​Endpoint-Specific Quotas

​Streaming Provider Rate Limits

​Embeddings Provider Rate Limits

​Retry Example

Types of Limits

Spending Limits

Endpoint-Specific Quotas

Streaming Provider Rate Limits

Embeddings Provider Rate Limits

Retry Example