> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Error Handling and Retries

> Best practices for handling non-streaming API errors: which errors to retry, backoff with jitter, Retry-After, retry budgets, and what to log.

Most failed requests tell you what went wrong and whether trying again can help. Read that information to retry failures that clear on their own, stop quickly on the ones that won't, and give support what it needs when something keeps failing.

This guide covers **non-streaming** requests and the HTTP error responses that come back before a stream starts:

* reading the error object instead of the status alone
* deciding which errors to retry
* backing off with jitter and honoring `Retry-After`
* setting retry budgets
* logging the right fields

> [!NOTE]
>
> [Streaming](https://docs.gloo.com/ai-learning-center/gloo-ai-104/streaming-responses.md) requests can also fail **after** the HTTP `200` response has started, once your user may already be reading partial output. Those failures need lower retry ceilings and a different recovery strategy. See [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md). This page and that one share the same rules for reading `error.retryable` and the same [retry budgets](#retry-budgets).

For the full list of status codes and error codes, see the [Errors](https://docs.gloo.com/api-reference/general/errors.md) and [Rate Limits](https://docs.gloo.com/api-reference/general/limits.md) reference pages.

## The Retry Rule

One rule covers every Gloo AI endpoint, streaming or not:

* **Retry only what the error marks retryable** (`error.retryable: true`, or a dropped connection). Never retry `4xx` authentication, validation, or billing errors.
* **Background work** (batch jobs, [ingestion](https://docs.gloo.com/api-guides/upload-content.md), anything no one is watching): at most **5 requests in total**, including the first. Wait for `Retry-After` first; otherwise wait `min(2^(attempt-1), 30)` seconds plus jitter.
* **Someone is waiting** (interactive requests and live streams): at most **2 retries**, with short backoff capped at a couple of seconds. See [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md#retry-budgets-and-backoff).
* **When `Retry-After` is longer than the caller can wait**, such as the 30 seconds a `3003 / SERVICE_UNAVAILABLE` usually asks for, don't wait: fail fast with a clear message, or hand the work to a background job.
* **Honor `X-RateLimit-*` headers when present**, but don't assume a response includes them.

## Read the Error, Not Just the Status

Completions and Responses errors include a structured `error` object. Use it for programmatic handling instead of parsing human-readable messages. See [AI Error Object](https://docs.gloo.com/api-reference/general/errors.md#ai-error-object) for every field.

* **Branch on `error.retryable` first.** It says whether retrying the same request may succeed. Use `error.name` or `error.code` to tell exactly what happened.
* **Log `error.name` and `error.code`, not just the status.** The status gives only the category; the name and code give you precise alerting and debugging.
* **On the [direct endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#direct-endpoints), branch on the `X-Gloo-Error-Code` header and `retryable`**, not on the status alone. Show `message` to your end users, and log `developer_message` for yourself. Never parse `message` or `developer_message`: their wording can change; the codes do not. See [Direct Endpoint Errors](https://docs.gloo.com/api-reference/general/errors.md#direct-endpoint-errors).
* **Use `detail` to pinpoint validation failures.** For `422` responses, the `detail` array tells you exactly which field failed: `loc` is the field path, `msg` is the reason, and `input` is the value you sent.

## Which Errors to Retry

Retry only transient failures, and only when the request is safe to send again.

| Response | Retry? | What to do |
| - | - | - |
| `429 / 2004 RATE_LIMIT` (the model's provider is limiting requests) | Yes | Wait for `Retry-After`, then retry with bounded exponential backoff. This is not your account's quota. |
| `503 / 3003 SERVICE_UNAVAILABLE` | Yes | Retry after `Retry-After`. |
| `504` provider timeouts and other `5xx` with `retryable: true` | Yes | Bounded retries with backoff. |
| [Embeddings](https://docs.gloo.com/api-guides/embeddings.md) `503 / 2002 PROVIDER_UNAVAILABLE` | Yes | Bounded exponential backoff. If the same request keeps failing, change the request or model instead. |
| Endpoint-specific `429` quota | Depends | Honor `Retry-After` when present. Otherwise retry only when the operation is safe to repeat, with a bounded delay and jitter. |
| `3007 / GATEWAY_DEADLINE` (`408` on guarded and grounded, `504` on direct) | Not unchanged | Shorten the request or stream it. |
| `3001 / INTERNAL_ERROR` | Never | Surface the error; [contact support](https://docs.gloo.com/getting-started/support.md#contact-support) with the `trace_id` if it persists. |
| `429 / 1005 SPENDING_LIMIT_EXCEEDED` | No | Retry only after the account's spending-limit condition is resolved. |
| `429 / 1009 MONTHLY_SPEND_LIMIT_REACHED` | No | Access returns when the month resets (UTC) or support raises the limit. |
| `402 INSUFFICIENT_CREDIT` | No | Add credits, then retry. |
| `401` | No | The API key is missing, invalid, or revoked. A static key won't become valid on retry; fix the key. See [API Key Security](https://docs.gloo.com/best-practices/api-key-security.md). |
| `403` insufficient permissions | No | The key or organization lacks access to this resource. Check your access in [Studio](https://docs.gloo.com/studio/overview.md). See [403 Forbidden](https://docs.gloo.com/api-reference/general/errors.md#authentication-errors). |
| `403` from [guardrails](https://docs.gloo.com/ai-learning-center/gloo-ai-101/application-patterns.md#guardrails) | Not unchanged | Show your user a helpful message rather than resending the same text. See [Guarded endpoints](https://docs.gloo.com/best-practices/guarded-endpoints.md#handle-what-guarded-sends-back). |
| `400` / `422` | No | Fix the request; `detail` names the field. |

Not every `429` is retryable. Read the `error` object before you decide.

Network failures with no response, such as connection timeouts and dropped connections, are likely transient: retry them with the same backoff and budget, and keep your user informed while you do.

## Backoff with Jitter

Wait longer between each attempt, and add random jitter so many clients that failed at the same moment don't all retry at the same moment.

1. **If the response has `Retry-After`, wait at least that long.** On the direct endpoints the same wait is also in the error body as `retry_after`. If that is longer than the caller can wait, fail fast or defer the work instead.
2. **Otherwise, use exponential backoff with jitter**, capped so a single wait never grows without bound: `min(2^(attempt-1), 30)` seconds plus jitter for background work, a couple of seconds at most when someone is waiting.
3. **Stop retrying and surface the error** when the error is not retryable, the budget is spent, or the limit requires an account or configuration change.

Honor `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset` when present, but don't assume a `429` includes them; they aren't part of a universal Gloo API response contract.

### Retry Example

This pseudocode follows the background budget: at most five total requests, including the first. It retries only when the response metadata, endpoint policy, and request safety all permit it, gives precedence to `Retry-After`, and excludes spending-limit errors.

```text theme={null}
attempt = 1
max_attempts = 5

loop:
  response = make_api_request()

  if response.status_code != 429:
    return response

  error = parse_error(response)

  if error is missing or malformed:
    return response

  if error.name == "SPENDING_LIMIT_EXCEEDED" or error.retryable != true:
    return response

  if endpoint_policy_allows_retry(response) != true:
    return response

  if request_is_safe_to_retry(response.request) != true:
    return response

  if attempt >= max_attempts:
    return response

  if response.headers contains "Retry-After":
    wait_seconds = parse_retry_after(response.headers["Retry-After"])
  else:
    wait_seconds = min(2 ** (attempt - 1), 30) + random_jitter()

  sleep(wait_seconds)
  attempt += 1
```

The same structure applies to retryable `5xx` responses: let `5xx` statuses through the first check alongside `429`, and the `error.retryable` check after `parse_error` decides whether to retry.

> [!TIP]
>
> New to retries? [Production-Ready](https://docs.gloo.com/ai-learning-center/gloo-ai-104/production-ready.md) in the AI Learning Center builds a small retry helper that follows these rules.

## Retry Budgets

Every retry loop needs a ceiling, and a user waiting on a screen needs a much smaller one than a background job.

| Context | Ceiling | Source of the rule |
| - | - | - |
| Background or non-interactive work, streaming or not | At most 5 total requests, including the first, with waits capped at 30 seconds and an overall job timeout | [Retry Example](#retry-example) above |
| Someone is waiting: an interactive request, or a live stream before any visible content | 2 retries, with backoff capped at a couple of seconds | [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md#retry-budgets-and-backoff) |
| Live stream, after visible content | 1 automatic Continue, then user-initiated controls | [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md#retry-budgets-and-backoff) |
| `Retry-After` longer than the caller can wait | Don't wait: fail fast with a clear message, or defer to a background job | [The Retry Rule](#the-retry-rule) |

Stop early the moment an error reports `retryable: false`, whatever the remaining budget.

## Make Requests Safe to Retry

A retry sends the same request again. Before you retry, make sure doing it twice is harmless:

* **Requests that change data**, such as uploads, need a way to detect a duplicate. See [Duplicate Upload Results](https://docs.gloo.com/api-reference/general/errors.md#duplicate-upload-results) for how ingestion reports content it already has.
* **Tool calls with side effects**, such as sending an email, should carry idempotency keys or operation IDs so a replayed call can be detected and dropped. See [Tool Calls and Side Effects](https://docs.gloo.com/best-practices/completions-streaming-failures.md#tool-calls-and-side-effects).

## Batch and Ingestion Jobs

Jobs that send many requests in a row, such as uploading a content library, should:

* **Control call frequency** with delays between API calls, and back off when a rate limit arrives.
* **Retry failed items** with the same budget, and report success or failure for each item.
* **Track progress** so you can see each item's processing status.
* **Validate content before sending it**, so a validation error is caught once instead of on every retry.

## Log What Support Needs

For every failed request, log:

* `error.name`, `error.code`, `error.fault`, and `error.retryable`
* the `trace_id` from the error object (on the direct endpoints, also `X-Request-Id`, which every response carries)
* the attempt number and the wait before it
* the endpoint and model you called

Track success counts, error counts by `error.name`, and response times so you can see a rising failure rate before your users report it.

**Open a ticket with the `trace_id` for persistent inference issues.** If inference errors continue after retrying with backoff, contact Support in Gloo AI Studio (bottom of the sidebar; see [Get help](https://docs.gloo.com/getting-started/support.md)) and include the error response's `trace_id` so we can trace the exact request.

## Summary

| Practice | Minimum viable version |
| - | - |
| Read the error | Branch on `error.retryable`, then `error.name` / `error.code` (`X-Gloo-Error-Code` on direct) |
| Retry selectively | Only `retryable: true` and safe-to-repeat requests |
| Back off | `Retry-After` first, else exponential backoff with jitter and a cap |
| Bound it | At most 5 requests for background work; 2 retries when someone is waiting |
| Log it | `error.name`, `error.code`, `trace_id`, attempt number |

## Next steps

> **[Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md)**
>
> Retry, continuation, and partial-output UX for interrupted streams.

> **[Errors Reference](https://docs.gloo.com/api-reference/general/errors.md)**
>
> Every status code, error code, and the AI error object.

> **[Rate Limits](https://docs.gloo.com/api-reference/general/limits.md)**
>
> Spending limits, endpoint quotas, and provider rate limits.

> **[API Key Security](https://docs.gloo.com/best-practices/api-key-security.md)**
>
> Storing, using, and replacing API keys safely.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.