> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt Caching

> Reduce costs and latency with implicit and explicit prompt caching across Anthropic, OpenAI, DeepSeek, and Gemini models

[Prompt caching](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md#prompt-caching) lets you reuse processed prompt [tokens](https://docs.gloo.com/ai-learning-center/gloo-ai-101/prompting-and-generation.md#tokens) across multiple requests, reducing both cost and latency. When requests start with the same text (the same *prompt prefix*, such as a long [system prompt](https://docs.gloo.com/ai-learning-center/gloo-ai-101/prompting-and-generation.md#system-prompt) or a shared document), those tokens can be served from cache instead of being processed again.

Current per-token rates are available on the [models page](https://studio.ai.gloo.com/models).

> [!TIP]
>
> New to token pricing? [Cost and Speed](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md) in the AI Learning Center covers how requests are priced, how caching lowers the cost, and how to choose a model and [reasoning effort](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md#reasoning-effort).

## Providers at a glance

| Provider | Type | How to turn it on | cache\_read rate | cache\_write rate | Notes |
| :- | :- | :- | :- | :- | :- |
| Anthropic | Explicit | `X-Cache-TTL` header (`5m` or `1h`) | Usually 10%; lower on some newer models (5% Opus/Sonnet 5.5; 2.5% Fable 5.1) | 5m: 125%; 1h: 200% of input | Two billing tiers via `X-Cache-TTL` header |
| OpenAI | Implicit + Explicit | Automatic; optional `prompt_cache_key` | Usually 10% (GPT-5 and later); 25% GPT-4.1 | None on most; 125% on some newer models | Automatic for 1,024+ tokens |
| DeepSeek | Implicit | Automatic | Varies per model | None | Automatic, no user action |
| Gemini | Implicit | Automatic | Usually 10% | Charged; varies per model | Automatic, no user action required |
| Other (Qwen, MiniMax, Xiaomi, …) | Implicit | Automatic | Varies widely per model | Most none; some Qwen models about 1.25× | Automatic, set by upstream provider |

Percentages are of the model's input rate. The [models endpoint](#checking-your-rates) has the exact rates for every model.

### Types of caching

LLM providers use three main approaches:

* **Implicit caching**: the provider automatically caches repeated prompt prefixes, with no code changes or configuration. Requests with matching prefixes are routed to servers that recently processed the same content. This is the simplest approach and is used by OpenAI, DeepSeek, Gemini, and many open-source models (Qwen, MiniMax, and others).

* **Explicit caching**: you declare which content to cache using API parameters (such as headers or request body fields), for more control over what gets cached and when. Anthropic uses this model via the `X-Cache-TTL` header.

* **Storage-based caching**: you create managed caches via API that persist for a defined duration, billed per token-hour. This offers the most control but requires cache lifecycle management. Gemini offers this as a separate feature from its implicit caching.

Most providers combine approaches. The [provider details](#provider-details) cover what each provider supports and how to use it through Gloo AI.

## Turn caching on

To start using prompt caching:

### Implicit caching (OpenAI, DeepSeek, Gemini)

No code changes needed. Caching is automatic for prompts with repeated prefixes. Place static content (system prompts, instructions) at the beginning of your prompt to maximize cache hits. On Gemini, prefix-only caching works on the provider side with no special parameters.

### Explicit caching (Anthropic)

Add the `X-Cache-TTL: 5m` header to opt in to caching. System message caching is then automatic, with no content changes.

```bash theme={null}
curl -X POST 'https://platform.ai.gloo.com/ai/v2/guarded/chat/completions' \
  -H "Authorization: Bearer $GLOO_API_KEY" \
  -H 'Content-Type: application/json' \
  -H 'X-Cache-TTL: 5m' \
  -d '{ "model": "gloo-anthropic-claude-opus-5.5", "messages": [{"role": "user", "content": "..."}] }'
```

### Improving cache hit rates (OpenAI)

Add `"prompt_cache_key": "your-key"` to your request body with a stable key that matches your prompt prefix. This improves server routing for requests with shared prefixes.

## Provider details

> [tab] Anthropic
>
> Anthropic uses **explicit caching** via the `X-Cache-TTL` header. You opt in per request by setting the header, and Gloo AI caches your system messages for you, with no changes to your message content.
>
> #### How it works
>
> 1. Set the `X-Cache-TTL` header to either `"5m"` or `"1h"`.
>
> 2. Add at most **three** `cache_control` markers of your own on your message content.
>
> 3. If your request already contains four `cache_control` markers of its own, your system messages aren't cached on that call. The request still succeeds with no error.
>
> 4. Keep per-request values (such as timestamps or user-specific data) in the user message, so your system messages stay identical from call to call.
>
> #### Endpoint behavior
>
> | Endpoint | Breakpoint ownership | What happens |
> | :- | :- | :- |
> | **Routed** (`/ai/v2/guarded/chat/completions`) | Up to 3 of your own | Your system messages are cached automatically. You may add up to 3 of your own on your content. |
>
> Minimum cacheable token thresholds vary by model (e.g., 4,096 tokens for Claude Haiku 4.5). See [Anthropic's prompt caching docs](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching) for current thresholds.
>
> #### Billing
>
> | Tier | cache\_write rate | cache\_read rate |
> | :- | :- | :- |
> | `X-Cache-TTL: 5m` | 125% of input rate | 10% of input rate on most models |
> | `X-Cache-TTL: 1h` | 200% of input rate | 10% of input rate on most models |
>
> Cache writes are billed at a multiple of the standard input rate. Cache reads are billed at 10% of the input rate on most Claude models, and less on some newer ones: 5% on Claude Opus 5.5 and Sonnet 5.5, and 2.5% on Claude Fable 5.1. The [models endpoint](#checking-your-rates) lists the exact rate for each model.
>
> #### Supported models
>
> Anthropic caching works with all Claude models available through our API. See [Supported Models](https://docs.gloo.com/api-guides/supported-models.md) for the current model list.
>
> #### Example
>
> ```bash theme={null}
> curl -X POST 'https://platform.ai.gloo.com/ai/v2/guarded/chat/completions' \
>   -H "Authorization: Bearer $GLOO_API_KEY" \
>   -H 'Content-Type: application/json' \
>   -H 'X-Cache-TTL: 5m' \
>   -d '{
>     "model": "gloo-anthropic-claude-opus-5.5",
>     "messages": [
>       {"role": "user", "content": "What does the Bible say about forgiveness?"}
>     ],
>     "stream": false
>   }'
> ```

> [tab] OpenAI
>
> OpenAI uses **implicit caching** as its primary mechanism. Caching works automatically on all API requests, with no code changes. When your prompt contains 1,024 tokens or more, OpenAI routes requests to servers that recently processed the same prompt prefix. This can reduce latency by up to 80% and input token costs by up to 90%. See [OpenAI's prompt caching announcement](https://openai.com/index/api-prompt-caching/) for details.
>
> For finer control, you can use **explicit caching** by adding a `prompt_cache_key` to your request body. Requests sharing the same key and common prefix are then more likely to land on the same server, improving cache hit rates.
>
> #### How it works
>
> 1. Caching is **implicit and automatic** for prompts with 1,024+ tokens.
> 2. Requests are routed to servers that recently processed the same prompt prefix.
> 3. Optionally, add `"prompt_cache_key": "your-key"` to your request body to improve routing for requests with shared prefixes.
> 4. Cache hits reduce both latency and cost. Cached tokens are billed at a reduced rate that varies by model, usually 10% of the input rate on GPT-5 and later models and 25% on GPT-4.1. Most OpenAI models charge nothing to write the cache; some newer ones do.
>
> #### Billing
>
> The cache read discount depends on the model. Current OpenAI families:
>
> | Model family | cache\_read rate | cache\_write rate |
> | :- | :- | :- |
> | GPT-5 and later | Usually 10% of input rate; lower on some newer models | None on most; some newer models, such as GPT-5.6 and GPT-6, charge 125% of input rate |
> | GPT-4.1 family | 25% of input rate | None |
>
> On most OpenAI models you pay only for cache reads. Rates change as new models arrive, so see [Checking your rates](#checking-your-rates) for each model's exact `cache_read` and `cache_write` values.
>
> #### Supported models
>
> OpenAI caching works with OpenAI models available through our API. The `prompt_cache_key` parameter is silently ignored for other providers. See [Supported Models](https://docs.gloo.com/api-guides/supported-models.md) for the current model list.
>
> #### Best practice: stable cache keys
>
> Use deterministic, stable keys that match your prompt structure. When you change the system prompt, use a new cache key.
>
> ```
> Good:  "ministry-block-v1", "sermon-generator", "theology-assistant-v1"
> Avoid: "request-12345", "cache-abc"
> ```
>
> #### Example
>
> ```json theme={null}
> {
>   "model": "gloo-openai-gpt-5.2",
>   "messages": [
>     { "role": "system", "content": "You are a theological assistant..." },
>     { "role": "user", "content": "Explain the Trinity" }
>   ],
>   "prompt_cache_key": "theology-assistant-v1",
>   "stream": false
> }
> ```

> [tab] DeepSeek
>
> DeepSeek uses **implicit caching** entirely on the provider side, with no user action or configuration.
>
> * **No headers or request body fields required**: caching is automatic.
> * **Cache read rates** vary per model, from a few percent to about half of the input rate.
> * **No cache write fee**.
> * Savings from cache reads appear automatically in your billing.

> [tab] Gemini
>
> Gemini uses **implicit caching** on the provider side: a request that matches a previously seen prefix is served from cache automatically.
>
> #### How it works
>
> 1. **No customer action needed**: no `X-Cache-TTL` header, no `cache_control` markers, and no `prompt_cache_key` are required.
> 2. Repeated prefixes are cached by the provider.
> 3. **Cache reads are usually 10% of the input rate**, and lower on some newer models. Gemini models with caching also charge a cache write, at a rate that varies by model. Check the [models endpoint](#checking-your-rates) for each model's exact `cache_read` and `cache_write` rates. Cache hits surface as a separate `cache_read` segment in your billing, the same as the other providers.
>
> #### Supported models
>
> Gemini caching is available for all Gemini models in the Gloo AI catalog. Minimum token thresholds vary by model and are determined by the provider.
>
> | Model | Minimum token threshold |
> | :- | :- |
> | Gemini 2.5 Flash, 2.5 Flash Lite | 2,048 tokens |
> | Gemini 3 Flash, 3.1 Flash Lite, 3.5 Flash | 4,096 tokens |

> [tab] Other models
>
> Other models in the Gloo AI catalog, including **Qwen**, **MiniMax**, and **Xiaomi MiMo**, support **implicit caching** through their upstream provider. Caching is automatic; no headers, `cache_control` markers, or `prompt_cache_key` are required.
>
> * **Automatic**: repeated prompt prefixes are cached provider-side and surface as a `cache_read` segment in your billing, the same as the other providers.
> * **Cache read rates vary widely per model**, from under 1% to more than half of the input rate, so the discount is much smaller on some models than on others. Some models, including several Qwen models, also charge a cache **write**, usually about 1.25× the input rate; most do not.
> * Use the [models endpoint](#checking-your-rates) for exact per-model `cache_read` / `cache_write` values.
>
> Not every model in this group caches: support is set by the upstream provider, and the models endpoint is the source of truth for which models have cache pricing.

## Cost and rates

### How total cost is calculated

```
total_cost = fresh_tokens × input_rate
            + cached_tokens × cache_read_rate
            + cache_write_tokens × cache_write_rate
            + output_tokens × output_rate
```

> [!NOTE]
>
> The formula above applies to all four providers: cached tokens are billed as
> a separate `cache_read` segment. On a model with no cache write fee, such as
> most OpenAI models and the DeepSeek models, the `cache_write` term is zero.

The per-segment rates returned by the models endpoint are **list prices** (the model's published rate). [Direct endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#direct-endpoints) and [Grounded Responses](https://docs.gloo.com/api-guides/endpoint-types.md#grounded-endpoints) bill these rates as listed; [guarded endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#guarded-endpoints) and Grounded Completions are priced at the list price plus 6.5% on every segment, the guarded [platform price](https://docs.gloo.com/getting-started/pricing.md).

### Checking your rates

Use the models endpoint to see current per-model rates:

```
GET /platform/v2/models
```

## Best Practices

1. **Static content first**: For all providers, place static content (system prompts, instructions) at the beginning of your prompt. This maximizes prefix matches for implicit caching and ensures stable blocks for explicit caching.
2. **Use explicit caching when available**: For Anthropic models, add the `X-Cache-TTL` header to opt in. For OpenAI, use `prompt_cache_key` to improve routing for requests with shared prefixes.
3. **Pre-warm on startup**: Send an initial request with your standard system prompt to warm the cache before serving users.
4. **Monitor cache impact**: Compare your token usage and billing over time. Effective caching shows up as lower input costs. Check your usage in the [Gloo Studio billing dashboard](https://studio.ai.gloo.com/billing).
5. **Expect fewer cache hits after failover**: Caches are held by each provider, so a request that [fails over](https://docs.gloo.com/api-guides/provider-failover.md) to another provider does not read the cache built on the original one.

For request structure on the guarded endpoints, such as keeping your instructions in the system message and pinning a model, see [Building with Guarded Endpoints](https://docs.gloo.com/best-practices/guarded-endpoints.md).

To apply these step by step and measure the savings on your own traffic, follow the [Prompt Caching tutorial](https://docs.gloo.com/tutorials/prompt-caching.md). For caching alongside the other levers on your bill, see [Controlling Cost](https://docs.gloo.com/best-practices/cost-control.md).

## Next steps

> [card] [Step-by-step tutorial](https://docs.gloo.com/tutorials/prompt-caching.md)
>
> Restructure a prompt for caching and measure the savings.

> [card] [Completions V2](https://docs.gloo.com/api-guides/completions-v2.md)
>
> Core routing mechanisms and streaming details.

> [card] [Supported Models](https://docs.gloo.com/api-guides/supported-models.md)
>
> Model IDs, capabilities, and current pricing.

> [card] [Provider Failover](https://docs.gloo.com/api-guides/provider-failover.md)
>
> What happens when the provider serving your model fails.

For core routing mechanisms and [streaming](https://docs.gloo.com/ai-learning-center/gloo-ai-104/streaming-responses.md) details, see [Completions V2](https://docs.gloo.com/api-guides/completions-v2.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.