Skip to main content
Control how much a model reasons before it answers. You’ll pick an effort level the model supports, read the reasoning back, size the output limit so the thinking doesn’t crowd out the answer, and catch answers that were cut off. The last section has a working example for each of the six endpoints. Want the concepts first? Read the Reasoning guide. New to reasoning models? Models That Think Before They Answer explains what reasoning is and when more effort pays off.
One set of settings for every model. On the Responses and Chat Completions endpoints (guarded, grounded and direct), reasoning_effort and the reasoning object work the same way for Claude, Gemini, OpenAI and open-weight models. Gloo translates them for each provider, so the code in this tutorial works across model families.
Using the Anthropic SDK? Anthropic Messages uses Anthropic’s own thinking and output_config.effort instead. See Use Claude through the Anthropic Messages endpoint.

Prerequisites

You need:
  • A Gloo AI Studio account
  • Your API key from the API Keys page, exported as GLOO_API_KEY
  • Authentication setup - Complete the Authentication Tutorial first
  • Python 3.10+ or Node.js 18+, and the OpenAI SDK for your language. The Gloo endpoints accept its requests, and it sends reasoning_effort and reasoning natively:
The short TypeScript examples use top-level await. To run one on its own, save it with an .mts extension and run it with npx tsx example.mts. The Complete Example runs as a plain .ts file.

Understanding the settings

Effort levels, from least to most reasoning: none, minimal, low, medium, high, xhigh, max. Each model takes only some of them.

Step 1: Find the levels a model takes

Every model in the catalog that reasons has a reasoning object. Look one up (this endpoint is public, so no key is needed):
A model that doesn’t reason has "reasoning": null. For example, Claude 5 models take low to max, and GPT-5.4 takes none, low, medium, high and xhigh.

What happens with a level the model doesn’t list

You won’t get an error. A level the model doesn’t list runs at the nearest level below it that the model lists (or at its lowest level when none is below), and the response says so in ignored_params:
Chat Completions names the entry reasoning_effort=<level>; Responses names it reasoning.effort=<level>. On a stream, the X-Ignored-Params response header lists the names only. Match on params, not on the reason sentence. To be sure of the level you get, send one the model lists. The Reasoning guide has a ready-made pick_effort helper in Python and TypeScript that maps the level you want to the nearest one a model takes.

Step 2: Pick an effort for the task

Effort is your main control over how long the model thinks, and so over cost and time. Reasoning tokens are billed as output tokens, whether or not any reasoning comes back. Turn effort up for:
  • Multi-step problems where one early mistake ruins the answer
  • Math, scheduling, and anything with numbers that must add up
  • Planning, where the model needs to weigh options and trade-offs
  • Writing or reviewing code
Keep effort low (or off, with none, on a model whose can_disable is true) for:
  • Simple lookups and factual questions
  • Friendly back-and-forth chat
  • Anything where speed matters most, like a live chat window
When More Effort Is Worth It walks through the trade-off. Use a lower effort to save tokens rather than a small output limit, which can cut the answer off. With no effort set, the model follows its provider’s default. Many models, including Claude 5 models, reason by default, and you pay for that reasoning as output tokens even though you didn’t ask for it.
Asking a model to think step by step in your prompt is a different tool. Step-by-step prompting puts the steps in the answer itself; reasoning happens before the answer, separate from it. They combine well: the effort controls how much the model thinks, and your prompt still guides what it thinks about.
Send an effort on Chat Completions:
On Chat Completions you can also send the nested form, "reasoning": {"effort": "low"}. If you send both, reasoning_effort wins.

Step 3: Read the reasoning back

Reasoning comes back only when the model reasoned. Claude 5 models decide how much each request needs and can skip reasoning on easy ones, so even with an effort set, an answer can arrive with no reasoning. Always treat it as optional. Claude, Gemini and OpenAI models return the provider’s summary of their reasoning, not the full chain of thought. Open-weight models such as DeepSeek, GLM and gpt-oss return their raw reasoning text.

Chat Completions

Set any effort except none, and read message.reasoning_content. The OpenAI SDKs don’t type that field, so read it with getattr in Python and a wider type in TypeScript:
On a stream, reasoning arrives as delta.reasoning_content chunks before the first answer chunk.

Responses

Add "summary": "auto" (or concise or detailed) to reasoning. An effort alone makes the model think but returns no summary. The summary is a reasoning item in output[], before the message item:
On a stream, read the reasoning from response.reasoning_summary_text.delta events and the answer from response.output_text.delta. Models outside the Anthropic, Google and OpenAI families stream raw reasoning as response.reasoning_text.delta instead; handle both if you stream from more than one family. What comes back shows every event.
On the guarded endpoints, reasoning isn’t checked by output moderation the way the answer is. If you show reasoning to your end users, review it yourself first, or don’t show it. See Reasoning is not moderated.

Step 4: Size the output limit

On a reasoning model, the output limit holds the thinking and the answer. A limit sized for the answer alone, such as 500 or 1,024, can be used up by the thinking before the answer starts. Pick a starting point from the effort you set and the answer you expect, then adjust it from the usage your requests report: These are starting points, not guarantees: the same model can reason for much longer on a hard question than on an easy one. With no effort set, size for the model’s default level, which is often medium or high.

What Gloo does with your limit

The ceiling is the model’s max_output_tokens in the catalog: thinking and answer together. You’re billed only for the output tokens the model uses, not for the limit. Set your output limit for reasoning models has the full rules, including how none changes them.

Stream long generations

A large limit with a high effort can take minutes. A non-streamed request that runs past Gloo’s response deadline fails with 3007 / GATEWAY_DEADLINE. Streaming avoids it and shows progress as it comes. If a stream fails partway, Handling Streaming Failures explains how to keep what the user has already seen and when to retry.

Step 5: Detect and handle a cut-off

A response that reached the limit says so:
  • Chat Completions: finish_reason: "length", streamed or not.
  • Responses: status: "incomplete" with incomplete_details: {"reason": "max_output_tokens"}. A non-streamed response can also carry a suggestion field that names the tokens used and the limit you sent.
When that comes with little or no answer text, the model spent the limit thinking. It’s not an outage, and sending the same request again usually ends the same way. Fix it by raising the limit or lowering the effort. usage shows where the tokens went. Where the model reports it, usage.completion_tokens_details.reasoning_tokens (Chat Completions) or usage.output_tokens_details.reasoning_tokens (Responses) is the reasoning part of the output tokens. It isn’t reported everywhere: for Claude models it is 0 or missing on some endpoints, and their reasoning is still counted in the output tokens. This helper checks every answer before it’s used and, on a cut-off, tries once more with double the limit:
On Responses, check the same thing with status and incomplete_details:
Size the limit per task, not once for your whole app: a classification and a long report need very different limits. Start generous, look at usage across real requests, then tighten.

Examples by endpoint

Each example sets an effort and reads the reasoning and the answer, non-streamed and streamed. Read reasoning_content with getattr in Python and a wider type in TypeScript. The Responses examples set max_output_tokens, because Claude models on Responses otherwise use the model’s default limit (see Step 4). Parameters the SDKs don’t know, such as the grounding parameters, go in extra_body in Python and top-level with a cast in TypeScript. If your SDK version doesn’t type a level you send (such as max), pass it in extra_body as well. The Python and TypeScript snippets in each tab reuse the client created in that tab’s first example.
Completions guide · API referencePOST /ai/v2/guarded/chat/completions. The answer is moderated; the reasoning is not (details).
Streamed:

How the endpoints differ

The guide’s How the endpoints differ table also covers passing reasoning back across turns.

Complete Example

This script puts the steps together for one model: it reads the model’s levels from the catalog, picks the level to send, makes the request with a sized limit, and on a cut-off tries once more with double the limit before printing the answer.
Run it:

What You’ll See

A line with the effort, limit and output tokens used, a short excerpt of the reasoning when the model returned some, and the answer: about 387 distinct people.

Troubleshooting

The answer is empty or stops mid-sentence : Check finish_reason (Chat Completions) or status (Responses). "length" or "incomplete" with max_output_tokens means the model spent the limit thinking. Raise the limit or lower the effort; resending the same request usually ends the same way. See Step 5. My max_tokens of 1,024 came back as a bigger number of output tokens : On Chat Completions, a reasoning model’s limit below 8,192 is raised to 8,192 so it has room to think. You’re billed only for the tokens the model uses. No reasoning comes back : On Responses, add "summary": "auto" to reasoning; an effort alone returns no summary. On Chat Completions, set an effort other than none. Claude 5 models can also skip reasoning on easy requests, and models with returns_summary: false (such as Gemma) don’t guarantee it. ignored_params lists my effort : The model doesn’t list that level, so it ran at the nearest level below it that it does list. Send a level from the model’s effort_levels (Step 1). none didn’t turn reasoning off, and I’m still billed for reasoning : Only models with can_disable: true can turn it off. On others, such as Claude 5 and Gemini 2.5 Pro, none runs at the model’s lowest level and is listed in ignored_params. Use a model that can disable reasoning, or accept the lowest level. Claude 4 models ignore the effort : Claude 4 models such as Haiku 4.5, Sonnet 4.6 and Opus 4.5 to 4.8 don’t reason on these endpoints. To use their thinking, call them on Anthropic Messages. reasoning_tokens is 0 but output tokens are high : For Claude models, reasoning_tokens is 0 or missing on some endpoints. Their reasoning is still counted in the output tokens. A long non-streamed request fails with 3007 / GATEWAY_DEADLINE : It ran past Gloo’s response deadline. Stream it instead. The streamed reasoning seems to repeat itself : If a provider fails while the model is still reasoning, Gloo can retry on another provider, and the model reasons again from the start. On Chat streams both attempts’ reasoning is joined; on Responses streams the first reasoning item closes as incomplete and a new one opens. See When a provider fails mid-reasoning. 422 when I send reasoning back in an assistant message : Completions and Grounded Completions refuse thinking parts in assistant content. Send earlier assistant turns as text; the model reasons afresh each turn. TypeScript says 'max' isn’t a valid reasoning_effort : Older openai SDK versions don’t type that level. Upgrade the SDK, or cast it.

Next Steps

Reasoning guide

Defaults, effort rules, output limits and behavior by model family.

Model catalog reference

GET /platform/v2/models, with each model’s reasoning object.

Handling Streaming Failures

Retry, continue or surface a stream that fails partway.

Models That Think

What reasoning is, and when more effort pays off.

Direct Responses

Every model in the catalog with one request shape.

Anthropic Messages tutorial

Claude’s own thinking settings through the Anthropic SDK.