> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Controlling Cost

> Best practices for keeping Gloo AI inference costs predictable: choosing models by price, controlling reasoning and output tokens, prompt caching, conversation length, images, and tracking usage.

Most surprises on an inference bill come from a few places: a model larger than the task needs, reasoning tokens nobody planned for, prompts that never hit the cache, and conversations that grow with every turn. This page is for developers who want their Gloo AI spend to be predictable, and who want to know which levers change it.

> [!TIP]
>
> New to how requests are priced? [Cost and Speed](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md) in the AI Learning Center explains tokens, rates, caching and reasoning effort, and [Pricing at a Glance](https://docs.gloo.com/getting-started/pricing.md) lists what each product costs.

This page covers how to:

* know what a request costs, from the catalog and from `usage`
* match the model to each step of your app
* control reasoning and output tokens
* get more cache hits
* keep conversations and images from growing the bill
* track spend and set a ceiling

## Know what a request costs

Every inference request is billed per token, at the rates in the [model catalog](https://docs.gloo.com/api-guides/supported-models.md) ([`GET /platform/v2/models`](https://docs.gloo.com/api-reference/get-models/v2)):

* **Input tokens** are everything you send: instructions, conversation history, documents and images. They bill at `pricing.input`.
* **Output tokens** are everything the model writes, including any reasoning, whether or not you see it. They bill at `pricing.output`, which is usually several times the input rate.
* **Cached tokens** bill at `pricing.cache_read`, and on some models writing the cache bills at `pricing.cache_write`. See [Prompt Caching](https://docs.gloo.com/api-guides/prompt-caching.md#how-total-cost-is-calculated).

How the model's rate becomes your price depends on the endpoint family, not on the model. [Direct endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#direct-endpoints) bill the model's published rate. [Guarded endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#guarded-endpoints) are priced at the model's rate plus 6.5%, the guarded [platform price](https://docs.gloo.com/getting-started/pricing.md), which pays for [guardrails](https://docs.gloo.com/ai-learning-center/gloo-ai-101/application-patterns.md#guardrails), output moderation and values alignment. Choose between them by who should own safety in your app, then control cost with the levers on this page, which work the same on both. See [Pricing at a glance](https://docs.gloo.com/api-guides/endpoint-types.md#pricing-at-a-glance) for [grounded endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#grounded-endpoints), [embeddings](https://docs.gloo.com/ai-learning-center/gloo-ai-103/embeddings.md) and System One.

Every response reports what it used in `usage`. Estimate each request's cost from it, and log the result next to the model and the feature that made the call:

```python Python theme={null}
import requests

catalog = requests.get("https://platform.ai.gloo.com/platform/v2/models", timeout=30).json()["data"]
RATES = {m["id"]: m["pricing"] for m in catalog}


def per_token(pricing: dict, segment: str) -> float:
    return float(pricing[segment]["rate_per_1m_tokens"]) / 1_000_000


def estimate_cost(model: str, usage: dict, guarded: bool) -> float:
    """List-price estimate from a Responses or Chat Completions `usage` object."""
    pricing = RATES[model]
    input_tokens = usage.get("input_tokens", usage.get("prompt_tokens", 0))
    output_tokens = usage.get("output_tokens", usage.get("completion_tokens", 0))
    details = usage.get("input_tokens_details") or usage.get("prompt_tokens_details") or {}
    cached = details.get("cached_tokens") or 0

    cost = (input_tokens - cached) * per_token(pricing, "input")
    cost += output_tokens * per_token(pricing, "output")
    if cached and "cache_read" in pricing:
        cost += cached * per_token(pricing, "cache_read")
    return cost * (1.065 if guarded else 1.0)
```

```typescript TypeScript theme={null}
const res = await fetch("https://platform.ai.gloo.com/platform/v2/models");
const RATES = new Map<string, any>((await res.json()).data.map((m: any) => [m.id, m.pricing]));

function perToken(pricing: any, segment: string): number {
  return Number(pricing[segment].rate_per_1m_tokens) / 1_000_000;
}

/** List-price estimate from a Responses or Chat Completions `usage` object. */
function estimateCost(model: string, usage: any, guarded: boolean): number {
  const pricing = RATES.get(model);
  const inputTokens = usage.input_tokens ?? usage.prompt_tokens ?? 0;
  const outputTokens = usage.output_tokens ?? usage.completion_tokens ?? 0;
  const details = usage.input_tokens_details ?? usage.prompt_tokens_details ?? {};
  const cached = details.cached_tokens ?? 0;

  let cost = (inputTokens - cached) * perToken(pricing, "input");
  cost += outputTokens * perToken(pricing, "output");
  if (cached && pricing.cache_read) cost += cached * perToken(pricing, "cache_read");
  return cost * (guarded ? 1.065 : 1);
}
```

This leaves out cache writes, which some models bill separately (`cache_write_tokens` in the same details object), and the higher `pricing.long_context` rates some models charge once a prompt passes `min_prompt_tokens`, so treat it as an estimate. [Read the Cache Fields in `usage`](https://docs.gloo.com/tutorials/prompt-caching#step-2-read-the-cache-fields-in-usage) shows every field by endpoint, and your **Usage** page in Studio is the record of what you were charged.

## Match the model to each step

Rates differ widely across the catalog, and the most capable model is rarely needed for every step. As of November 2026, Claude Sonnet 5.5 lists \$2 per million input tokens and \$10 per million output tokens, while GPT-6 Luna lists \$0.10 and \$0.50.

* **Split your app into steps** (classify, extract, draft, check) and choose a model per step. Short classifications, routing decisions and extractions often run well on small, fast models; long reasoning and careful writing may need a larger one.
* **Test the cheaper model first** on your own examples, and move up only where quality falls short. See [Testing, Evaluation and Observability](https://docs.gloo.com/best-practices/testing-and-observability.md).
* **Pin the model you chose.** A pinned `model` is billed at that model's catalog rate on every call, rather than a rate that changes per request. On guarded [Completions](https://docs.gloo.com/api-guides/completions-v2.md#choose-how-the-model-is-selected), `model_family` and [auto-routing](https://docs.gloo.com/getting-started/glossary.md#auto-routing) can pick a different model, at a different price, per request.
* **Watch for replacements.** Pinning isn't a price guarantee: when a model you use is deprecated, its requests can be redirected to a replacement priced differently, and `deprecation_note` states any increase. Check catalog rates on a schedule, log the `X-Gloo-Model-Redirected-From` header, and follow [Model Updates](https://docs.gloo.com/model-updates.md). See [Choosing, Pinning and Updating Models](https://docs.gloo.com/best-practices/choosing-and-updating-models.md).

## Control reasoning and output tokens

Output tokens cost the most, and on [reasoning models](https://docs.gloo.com/ai-learning-center/gloo-ai-102/models-that-think.md) much of the output is thinking.

* **Know which models think unprompted.** A model with `reasoning.thinks_by_default: true` reasons, and bills that reasoning, even when you send no reasoning setting.
* **Lower the effort, not the limit.** The reasoning effort is the main control over how long a model thinks. Use `low`, or `none` on a model whose `effort_levels` include it, for short tasks. Lowering `max_output_tokens` (or `max_tokens`) doesn't make the model think less; it cuts the answer off when thinking uses the limit.
* **The limit is a ceiling, not a charge.** You pay for the output tokens the model uses, not for the limit you set. Size the limit so answers aren't cut off, then let effort set the cost. See [Size the limit](https://docs.gloo.com/api-guides/reasoning.md#size-the-limit).
* **Read where output went.** Where the model reports it, `usage.output_tokens_details.reasoning_tokens` (Responses) or `usage.completion_tokens_details.reasoning_tokens` (Chat Completions) is the part of the output that was reasoning. Claude models report it as `0` or leave it out on some endpoints, and their reasoning is still in the output count.
* **Ask for shorter answers.** "Answer in two sentences" or a tight output schema cuts output tokens on every model.

[Reasoning Models in Production](https://docs.gloo.com/best-practices/reasoning-models.md) covers effort and limits in more depth.

## Get more cache hits

[Prompt caching](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md#prompt-caching) bills the repeated start of a prompt at the much lower `cache_read` rate, usually around a tenth of the input rate. It only works when the start of the prompt is identical to a recent request, token for token.

* **Put stable content first.** Order each request from most stable to least: instructions, reference documents, tool definitions, conversation history, then the new question. Keep timestamps, user names and per-request snippets at the end.
* **Keep the prefix byte-identical.** Build it from the same template every time; don't reorder examples, tools or JSON keys between requests.
* **Turn on caching where it's opt-in.** Claude models cache when you send the `X-Cache-TTL` header (`5m` or `1h`); on Anthropic Messages use `cache_control`. OpenAI, DeepSeek, Gemini and most other models cache automatically above a minimum prompt length.
* **Use a stable `prompt_cache_key` on OpenAI models**, named after the prompt it caches, such as `"support-assistant-v3"`, and change it when the instructions change. Per-request keys defeat the cache.
* **Check that it works.** Log the cached-token count from `usage` with every request. A hit rate that drops to zero usually means something started changing the prefix. Expect fewer hits for a while after a [provider failover](https://docs.gloo.com/api-guides/provider-failover.md), since each provider holds its own cache.

The [Prompt Caching guide](https://docs.gloo.com/api-guides/prompt-caching.md#best-practices) lists the rates and rules for each provider, and the [Prompt Caching tutorial](https://docs.gloo.com/tutorials/prompt-caching.md) restructures a prompt and measures the savings step by step.

## Keep conversations and images in check

Your app sends the whole conversation on every request, so every turn bills all the turns before it again. A 30-turn conversation sends all 30 turns on turn 31.

* **Trim or summarise long histories.** Keep the instructions and the recent turns; drop or summarise the oldest. See [Conversations and Context Windows](https://docs.gloo.com/best-practices/conversations-and-context.md).
* **Append, don't rewrite.** Adding new turns to the end keeps the earlier conversation as a cacheable prefix.
* **Mind images.** An image is billed as input tokens at the model's input rate, and again on every turn it stays in the history. Scale images down before you send them, and describe an image once and keep the text when the conversation moves on. See [Images in Conversations](https://docs.gloo.com/best-practices/images-in-conversations.md#know-what-each-turn-costs).
* **Send only the context the question needs.** Retrieving the three most relevant passages costs less than pasting a whole document, and often answers better.

## Track spend and set a ceiling

* **Watch the Usage page.** [API Usage](https://docs.gloo.com/studio/api-usage.md) in [Gloo AI Studio](https://docs.gloo.com/studio/overview.md) shows spend, requests and tokens (input, output and cache) over time. Filter by API key to see which application is spending, so give each application and environment its own [API key](https://docs.gloo.com/studio/manage-api-credentials.md).
* **Log `usage` per feature.** Studio shows totals; your own logs tell you which feature or customer drove them. Record the model, the token counts and your cost estimate with every request.
* **Know your ceiling.** Organizations on [prepaid credits](https://docs.gloo.com/studio/prepaid-credits.md) have a monthly API [spend limit](https://docs.gloo.com/studio/billing.md#spend-limits), \$4,000 per calendar month (UTC) unless Gloo has set a different one. When it's reached, requests return `429` with `MONTHLY_SPEND_LIMIT_REACHED` until the month resets. [Contact support](https://docs.gloo.com/getting-started/support.md#contact-support) to raise or lower it, including lowering it as a safety net for a new app.
* **Cap your own loops.** Retries, tool loops and agents can multiply requests. Bound them with a retry budget and a step limit. See [Error Handling and Retries](https://docs.gloo.com/best-practices/error-handling-and-retries.md#retry-budgets) and [Tools, Structured Output and Agents](https://docs.gloo.com/best-practices/tools-and-structured-output.md).

## Summary

* Output costs more than input, and reasoning is output: control it with effort, not by shrinking the limit.
* Pick the smallest model that passes your tests for each step, and pin it.
* Put stable content first, turn on caching where it's opt-in, and check the cached-token count.
* Trim long conversations and shrink images, since every turn resends them.
* Log `usage` per feature, filter Studio usage by API key, and know your monthly spend limit.

## Next steps

> [card] [Cost and Speed](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md)
>
> The lesson: how a request is priced, caching and reasoning effort.

> [card] [Prompt Caching tutorial](https://docs.gloo.com/tutorials/prompt-caching.md)
>
> Restructure a prompt for caching and measure the savings.

> [card] [Reasoning Models in Production](https://docs.gloo.com/best-practices/reasoning-models.md)
>
> Effort, output limits, streaming and cut-off answers.

> [card] [API Usage](https://docs.gloo.com/studio/api-usage.md)
>
> Spend, requests and tokens in Studio.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.