> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Reasoning Models in Production

> Best practices for reasoning models on Gloo AI: choosing an effort per task, sizing output limits, streaming long generations, avoiding timeouts, and handling cut-off answers.

[Reasoning models](https://docs.gloo.com/ai-learning-center/gloo-ai-102/models-that-think.md) think before they answer. That makes them better at hard problems, and also slower, more expensive and more likely to run out of room if a request is sized for a model that doesn't think. This page is for developers putting reasoning models into production who want answers that arrive complete, on time and at a predictable cost.

> [!TIP]
>
> The [Reasoning guide](https://docs.gloo.com/api-guides/reasoning.md) is the reference for the settings, what each endpoint returns and how each family of models behaves. The [Reasoning tutorial](https://docs.gloo.com/tutorials/reasoning.md) walks through them step by step. This page is about using them well.

This page covers how to:

* decide how much reasoning each task needs
* read a model's reasoning behaviour from the catalog
* size the output limit so answers aren't cut off
* stream long generations and set timeouts
* detect and fix a cut-off answer
* handle reasoning you show to users, and reasoning across turns

## Decide how much reasoning each task needs

More effort usually helps on multi-step problems, careful analysis and code, and buys little on lookups, short rewrites and classifications. Effort is the main control over how long a model thinks, and so over its cost and latency; see [When More Effort Is Worth It](https://docs.gloo.com/ai-learning-center/gloo-ai-102/models-that-think.md#when-more-effort-is-worth-it).

* **Set an effort on every request** rather than relying on the model's own behaviour. Many models reason when you send nothing, and you pay for that reasoning as output tokens.
* **Choose per task, not per app.** A router that picks `low` for short questions and `high` for analysis spends far less than one setting for everything.
* **Turn reasoning off where you can and should.** For fast, short tasks, send `none` (`reasoning_effort: "none"` on Chat Completions, `reasoning: {"effort": "none"}` on Responses) to a model that can turn reasoning off.
* **Measure before you raise it.** Run your test set at two efforts and compare quality, `usage` and time. If `low` passes, keep it. See [Testing, Evaluation and Observability](https://docs.gloo.com/best-practices/testing-and-observability.md).

## Read the model's behaviour from the catalog

Each model in the [model catalog](https://docs.gloo.com/api-guides/supported-models.md#reasoning) has a `reasoning` object, or `null` when it doesn't reason. Read it before you choose settings:

| Field | What it tells you |
| :- | :- |
| `effort_levels` | The efforts the model runs as sent. Send one of these. |
| `thinks_by_default` | `true`: the model reasons, and bills reasoning, when you send no effort. |
| `can_disable` | `true`: `none` turns reasoning off. `false`: you can't rely on `none`, and the model still bills the reasoning it does. |
| `returns_summary` | Whether a request with an effort gets reasoning text back. |

A level the model doesn't list normally isn't refused: it runs at the nearest level the model has, and the response lists it in `ignored_params` (on Gemini 2.5 models, `xhigh` and `max` can be refused with a `400`, so send `high`). Check `ignored_params` in development so you know the effort you sent is the one that ran. See [Effort levels](https://docs.gloo.com/api-guides/reasoning.md#effort-levels).

```python Python theme={null}
import requests

ORDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"]

catalog = requests.get("https://platform.ai.gloo.com/platform/v2/models", timeout=30).json()["data"]
MODELS = {m["id"]: m for m in catalog}


def effort_for(model_id: str, wanted: str) -> str | None:
    """The listed effort closest to `wanted`, or None to send no effort."""
    reasoning = MODELS[model_id].get("reasoning")
    if reasoning is None:
        return None  # the model doesn't reason
    levels = reasoning["effort_levels"]
    if wanted == "none":
        return "none" if "none" in levels else None
    graded = [level for level in levels if level != "none"]
    if not graded:
        return None  # the model decides for itself
    return min(graded, key=lambda level: abs(ORDER.index(level) - ORDER.index(wanted)))
```

```typescript TypeScript theme={null}
const ORDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"];

const res = await fetch("https://platform.ai.gloo.com/platform/v2/models");
const MODELS = new Map<string, any>((await res.json()).data.map((m: any) => [m.id, m]));

/** The listed effort closest to `wanted`, or null to send no effort. */
function effortFor(modelId: string, wanted: string): string | null {
  const reasoning = MODELS.get(modelId)?.reasoning;
  if (!reasoning) return null; // the model doesn't reason
  const levels: string[] = reasoning.effort_levels;
  if (wanted === "none") return levels.includes("none") ? "none" : null;
  const graded = levels.filter((level) => level !== "none");
  if (graded.length === 0) return null; // the model decides for itself
  const distance = (level: string) => Math.abs(ORDER.indexOf(level) - ORDER.indexOf(wanted));
  return graded.reduce((best, level) => (distance(level) < distance(best) ? level : best));
}
```

When `wanted` is `none` and the helper returns nothing, check `thinks_by_default`: if it's `false`, sending no effort already means no reasoning; if it's `true`, the model can't be told not to reason, so pick a model with `can_disable: true` if the task must be fast and cheap.

## Size the output limit for thinking and answer together

On a reasoning model, the output limit (`max_tokens` on Chat Completions, `max_output_tokens` on Responses) holds the thinking and the answer. A limit sized for the answer alone, such as 500, can be used up by thinking before the answer starts.

* **Start generous, then tighten.** Begin from the table in [Size the limit](https://docs.gloo.com/api-guides/reasoning.md#size-the-limit), or leave the limit unset on Chat Completions, then lower it once `usage` from real requests shows how much your task needs.
* **Size per task.** A classification and a long report need very different limits.
* **Don't carry limits over** from models that don't reason, and recheck them when you change models or efforts.
* **Lower the effort, not the limit, to save tokens.** You're billed for the tokens the model uses, not for the limit, so a smaller limit doesn't make the model think less; it cuts the answer off.
* **Know the ceiling.** A model's `max_output_tokens` in the catalog is the most it can produce, thinking and answer together.

Gloo adjusts some limits for you, differently by endpoint; for example, on Chat Completions a limit below 8,192 on a reasoning model is raised to 8,192. See [What Gloo sends](https://docs.gloo.com/api-guides/reasoning.md#what-gloo-sends) rather than relying on it.

## Stream long generations and set timeouts

A high effort with a large limit can take minutes.

* **Stream anything long.** A non-streamed request that runs past Gloo's response deadline fails with `3007 / GATEWAY_DEADLINE` (`408` on guarded and [grounded endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#grounded-endpoints), `504` with the code `gateway_deadline` on direct). It isn't retryable: the same request meets the same deadline. [Streaming](https://docs.gloo.com/ai-learning-center/gloo-ai-104/streaming-responses.md) avoids it and shows progress as it comes. See [504 Gateway Timeout](https://docs.gloo.com/api-reference/general/errors.md#504-gateway-timeout).
* **Set your own timeouts to match.** For a non-streamed request, give your HTTP client a total timeout longer than your slowest expected answer. For a stream, use a timeout between events rather than a total timeout, so a long but healthy stream isn't cut off.
* **Show that the model is thinking.** Streams can carry the reasoning before the answer when you set an effort (and `reasoning.summary` on Responses). Even when you don't show the text, a "thinking" indicator tells users the request is alive.
* **Know what a cancelled stream costs.** Stopping a stream partway doesn't always stop the bill for thinking already done. See [Cancelled and failed streams](https://docs.gloo.com/api-guides/reasoning.md#cancelled-and-failed-streams).

For retries, continuation and partial output on streams, see [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md#summary-table).

## Detect and fix a cut-off answer

Check that a response finished before you use it. A response that reached the limit says so:

* **Chat Completions:** `finish_reason: "length"`.
* **Responses:** `status: "incomplete"` with `incomplete_details.reason` set to `"max_output_tokens"`.

```python Python theme={null}
def was_cut_off(body: dict) -> bool:
    """True when a Chat Completions or Responses body stopped at the output limit."""
    if "choices" in body:
        return body["choices"][0].get("finish_reason") == "length"
    return (
        body.get("status") == "incomplete"
        and (body.get("incomplete_details") or {}).get("reason") == "max_output_tokens"
    )
```

```typescript TypeScript theme={null}
/** True when a Chat Completions or Responses body stopped at the output limit. */
function wasCutOff(body: any): boolean {
  if (Array.isArray(body.choices)) return body.choices[0]?.finish_reason === "length";
  return body.status === "incomplete" && body.incomplete_details?.reason === "max_output_tokens";
}
```

If there's little or no answer text, the model spent the limit thinking. It isn't an outage, and resending the same request usually ends the same way. Raise the limit or lower the effort, then send it again. Don't show a cut-off answer as if it were complete, and never parse a cut-off answer as JSON. See [Spot and fix a cut-off](https://docs.gloo.com/api-guides/reasoning.md#spot-and-fix-a-cut-off).

## Handle reasoning you show to users

* **Treat reasoning as optional.** It comes back only when the model reasoned, and some models skip reasoning on easy questions even when you ask for it. Don't make your UI depend on it.
* **Reasoning isn't moderated.** On the [guarded endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#guarded-endpoints), moderation checks the answer, not the reasoning. On [direct endpoints](https://docs.gloo.com/api-guides/endpoint-types.md#direct-endpoints), nothing reviews either. If you show reasoning to end users, review it yourself or don't show it. See [Reasoning is not moderated](https://docs.gloo.com/api-guides/reasoning.md#reasoning-is-not-moderated).
* **Don't build on reasoning text.** It's a summary or raw thinking for people to read, not a stable format for code to parse. Put anything your code needs in the answer, or in a tool call.

## Reasoning across turns

Send the full conversation each turn, as text and output items. Reasoning from earlier turns isn't carried forward: the model reasons afresh each turn, and reasoning items you send back are removed and reported. Don't rely on `previous_response_id` or encrypted reasoning to keep the model's thinking between turns. See [Reasoning across turns](https://docs.gloo.com/api-guides/reasoning.md#reasoning-across-turns) and [Conversations and Context Windows](https://docs.gloo.com/best-practices/conversations-and-context.md).

## Summary

| Practice | Minimum viable version |
| :- | :- |
| Choose the effort | Set one on every request, per task; send a level in `effort_levels` |
| Know the model | Read `thinks_by_default` and `can_disable` before you rely on `none` |
| Size the limit | Thinking plus answer; start from the guide's table, then tighten from `usage` |
| Stay inside the deadline | Stream long generations; use an idle timeout on streams |
| Catch cut-offs | Check `finish_reason` or `status` before using an answer |
| Show reasoning carefully | Optional, unmoderated, never parsed |

## Next steps

> [card] [Reasoning guide](https://docs.gloo.com/api-guides/reasoning.md)
>
> Settings, effort levels and every model family's behaviour.

> [card] [Reasoning tutorial](https://docs.gloo.com/tutorials/reasoning.md)
>
> Pick an effort, read the reasoning back and size the limit, step by step.

> [card] [Models That Think Before They Answer](https://docs.gloo.com/ai-learning-center/gloo-ai-102/models-that-think.md)
>
> The lesson: what reasoning is and when it helps.

> [card] [Controlling Cost](https://docs.gloo.com/best-practices/cost-control.md)
>
> Reasoning tokens, caching and the other levers on your bill.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.