Reasoning Models in Production

Reasoning models think before they answer. That makes them better at hard problems, and also slower, more expensive and more likely to run out of room if a request is sized for a model that doesn’t think. This page is for developers putting reasoning models into production who want answers that arrive complete, on time and at a predictable cost.

The Reasoning guide is the reference for the settings, what each endpoint returns and how each family of models behaves. The Reasoning tutorial walks through them step by step. This page is about using them well.

This page covers how to:

decide how much reasoning each task needs
read a model’s reasoning behaviour from the catalog
size the output limit so answers aren’t cut off
stream long generations and set timeouts
detect and fix a cut-off answer
handle reasoning you show to users, and reasoning across turns

Decide how much reasoning each task needs

More effort usually helps on multi-step problems, careful analysis and code, and buys little on lookups, short rewrites and classifications. Effort is the main control over how long a model thinks, and so over its cost and latency; see When More Effort Is Worth It.

Set an effort on every request rather than relying on the model’s own behaviour. Many models reason when you send nothing, and you pay for that reasoning as output tokens.
Choose per task, not per app. A router that picks low for short questions and high for analysis spends far less than one setting for everything.
Turn reasoning off where you can and should. For fast, short tasks, send none (reasoning_effort: "none" on Chat Completions, reasoning: {"effort": "none"} on Responses) to a model that can turn reasoning off.
Measure before you raise it. Run your test set at two efforts and compare quality, usage and time. If low passes, keep it. See Testing, Evaluation and Observability.

Read the model’s behaviour from the catalog

Each model in the model catalog has a reasoning object, or null when it doesn’t reason. Read it before you choose settings:

Field	What it tells you
`effort_levels`	The efforts the model runs as sent. Send one of these.
`thinks_by_default`	`true`: the model reasons, and bills reasoning, when you send no effort.
`can_disable`	`true`: `none` turns reasoning off. `false`: you can’t rely on `none`, and the model still bills the reasoning it does.
`returns_summary`	Whether a request with an effort gets reasoning text back.

A level the model doesn’t list normally isn’t refused: it runs at the nearest level the model has, and the response lists it in ignored_params (on Gemini 2.5 models, xhigh and max can be refused with a 400, so send high). Check ignored_params in development so you know the effort you sent is the one that ran. See Effort levels.

import requests

ORDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"]

catalog = requests.get("https://platform.ai.gloo.com/platform/v2/models", timeout=30).json()["data"]
MODELS = {m["id"]: m for m in catalog}


def effort_for(model_id: str, wanted: str) -> str | None:
    """The listed effort closest to `wanted`, or None to send no effort."""
    reasoning = MODELS[model_id].get("reasoning")
    if reasoning is None:
        return None  # the model doesn't reason
    levels = reasoning["effort_levels"]
    if wanted == "none":
        return "none" if "none" in levels else None
    graded = [level for level in levels if level != "none"]
    if not graded:
        return None  # the model decides for itself
    return min(graded, key=lambda level: abs(ORDER.index(level) - ORDER.index(wanted)))

const ORDER = ["none", "minimal", "low", "medium", "high", "xhigh", "max"];

const res = await fetch("https://platform.ai.gloo.com/platform/v2/models");
const MODELS = new Map<string, any>((await res.json()).data.map((m: any) => [m.id, m]));

/** The listed effort closest to `wanted`, or null to send no effort. */
function effortFor(modelId: string, wanted: string): string | null {
  const reasoning = MODELS.get(modelId)?.reasoning;
  if (!reasoning) return null; // the model doesn't reason
  const levels: string[] = reasoning.effort_levels;
  if (wanted === "none") return levels.includes("none") ? "none" : null;
  const graded = levels.filter((level) => level !== "none");
  if (graded.length === 0) return null; // the model decides for itself
  const distance = (level: string) => Math.abs(ORDER.indexOf(level) - ORDER.indexOf(wanted));
  return graded.reduce((best, level) => (distance(level) < distance(best) ? level : best));
}

When wanted is none and the helper returns nothing, check thinks_by_default: if it’s false, sending no effort already means no reasoning; if it’s true, the model can’t be told not to reason, so pick a model with can_disable: true if the task must be fast and cheap.

Size the output limit for thinking and answer together

On a reasoning model, the output limit (max_tokens on Chat Completions, max_output_tokens on Responses) holds the thinking and the answer. A limit sized for the answer alone, such as 500, can be used up by thinking before the answer starts.

Start generous, then tighten. Begin from the table in Size the limit, or leave the limit unset on Chat Completions, then lower it once usage from real requests shows how much your task needs.
Size per task. A classification and a long report need very different limits.
Don’t carry limits over from models that don’t reason, and recheck them when you change models or efforts.
Lower the effort, not the limit, to save tokens. You’re billed for the tokens the model uses, not for the limit, so a smaller limit doesn’t make the model think less; it cuts the answer off.
Know the ceiling. A model’s max_output_tokens in the catalog is the most it can produce, thinking and answer together.

Gloo adjusts some limits for you, differently by endpoint; for example, on Chat Completions a limit below 8,192 on a reasoning model is raised to 8,192. See What Gloo sends rather than relying on it.

Stream long generations and set timeouts

A high effort with a large limit can take minutes.

Stream anything long. A non-streamed request that runs past Gloo’s response deadline fails with 3007 / GATEWAY_DEADLINE (408 on guarded and grounded endpoints, 504 with the code gateway_deadline on direct). It isn’t retryable: the same request meets the same deadline. Streaming avoids it and shows progress as it comes. See 504 Gateway Timeout.
Set your own timeouts to match. For a non-streamed request, give your HTTP client a total timeout longer than your slowest expected answer. For a stream, use a timeout between events rather than a total timeout, so a long but healthy stream isn’t cut off.
Show that the model is thinking. Streams can carry the reasoning before the answer when you set an effort (and reasoning.summary on Responses). Even when you don’t show the text, a “thinking” indicator tells users the request is alive.
Know what a cancelled stream costs. Stopping a stream partway doesn’t always stop the bill for thinking already done. See Cancelled and failed streams.

For retries, continuation and partial output on streams, see Handling Streaming Failures.

Detect and fix a cut-off answer

Check that a response finished before you use it. A response that reached the limit says so:

Chat Completions: finish_reason: "length".
Responses: status: "incomplete" with incomplete_details.reason set to "max_output_tokens".

def was_cut_off(body: dict) -> bool:
    """True when a Chat Completions or Responses body stopped at the output limit."""
    if "choices" in body:
        return body["choices"][0].get("finish_reason") == "length"
    return (
        body.get("status") == "incomplete"
        and (body.get("incomplete_details") or {}).get("reason") == "max_output_tokens"
    )

/** True when a Chat Completions or Responses body stopped at the output limit. */
function wasCutOff(body: any): boolean {
  if (Array.isArray(body.choices)) return body.choices[0]?.finish_reason === "length";
  return body.status === "incomplete" && body.incomplete_details?.reason === "max_output_tokens";
}

If there’s little or no answer text, the model spent the limit thinking. It isn’t an outage, and resending the same request usually ends the same way. Raise the limit or lower the effort, then send it again. Don’t show a cut-off answer as if it were complete, and never parse a cut-off answer as JSON. See Spot and fix a cut-off.

Handle reasoning you show to users

Treat reasoning as optional. It comes back only when the model reasoned, and some models skip reasoning on easy questions even when you ask for it. Don’t make your UI depend on it.
Reasoning isn’t moderated. On the guarded endpoints, moderation checks the answer, not the reasoning. On direct endpoints, nothing reviews either. If you show reasoning to end users, review it yourself or don’t show it. See Reasoning is not moderated.
Don’t build on reasoning text. It’s a summary or raw thinking for people to read, not a stable format for code to parse. Put anything your code needs in the answer, or in a tool call.

Reasoning across turns

Send the full conversation each turn, as text and output items. Reasoning from earlier turns isn’t carried forward: the model reasons afresh each turn, and reasoning items you send back are removed and reported. Don’t rely on previous_response_id or encrypted reasoning to keep the model’s thinking between turns. See Reasoning across turns and Conversations and Context Windows.

Summary

Practice	Minimum viable version
Choose the effort	Set one on every request, per task; send a level in `effort_levels`
Know the model	Read `thinks_by_default` and `can_disable` before you rely on `none`
Size the limit	Thinking plus answer; start from the guide’s table, then tighten from `usage`
Stay inside the deadline	Stream long generations; use an idle timeout on streams
Catch cut-offs	Check `finish_reason` or `status` before using an answer
Show reasoning carefully	Optional, unmoderated, never parsed

Next steps

Reasoning guide

Settings, effort levels and every model family’s behaviour.

Reasoning tutorial

Pick an effort, read the reasoning back and size the limit, step by step.

Models That Think Before They Answer

The lesson: what reasoning is and when it helps.

Controlling Cost

Reasoning tokens, caching and the other levers on your bill.

Best Practices

Security

Models and Cost

Requests

Reliability

Trust

Content

Reasoning Models in Production

Decide how much reasoning each task needs

Read the model’s behaviour from the catalog

Size the output limit for thinking and answer together

Stream long generations and set timeouts

Detect and fix a cut-off answer

Handle reasoning you show to users

Reasoning across turns

Summary

Next steps

Reasoning guide

Reasoning tutorial

Models That Think Before They Answer

Controlling Cost

​Decide how much reasoning each task needs

​Read the model’s behaviour from the catalog

​Size the output limit for thinking and answer together

​Stream long generations and set timeouts

​Detect and fix a cut-off answer

​Handle reasoning you show to users

​Reasoning across turns

​Summary

​Next steps

Reasoning guide

Reasoning tutorial

Models That Think Before They Answer

Controlling Cost

Decide how much reasoning each task needs

Read the model’s behaviour from the catalog

Size the output limit for thinking and answer together

Stream long generations and set timeouts

Detect and fix a cut-off answer

Handle reasoning you show to users

Reasoning across turns

Summary

Next steps