One set of settings for every model. On the Responses and Chat Completions endpoints (guarded, grounded and direct),
reasoning_effort and the reasoning object work the same way for Claude, Gemini, OpenAI and open-weight models. Gloo translates them for each provider, so the code in this tutorial works across model families.Using the Anthropic SDK? Anthropic Messages uses Anthropic’s own
thinking and output_config.effort instead. See Use Claude through the Anthropic Messages endpoint.Prerequisites
You need:- A Gloo AI Studio account
- Your API key from the API Keys page, exported as
GLOO_API_KEY - Authentication setup - Complete the Authentication Tutorial first
- Python 3.10+ or Node.js 18+, and the OpenAI SDK for your language. The Gloo endpoints accept its requests, and it sends
reasoning_effortandreasoningnatively:
await. To run one on its own, save it with an .mts extension and run it with npx tsx example.mts. The Complete Example runs as a plain .ts file.
Understanding the settings
Effort levels, from least to most reasoning:
none, minimal, low, medium, high, xhigh, max. Each model takes only some of them.
Step 1: Find the levels a model takes
Every model in the catalog that reasons has areasoning object. Look one up (this endpoint is public, so no key is needed):
A model that doesn’t reason has
"reasoning": null. For example, Claude 5 models take low to max, and GPT-5.4 takes none, low, medium, high and xhigh.
What happens with a level the model doesn’t list
You won’t get an error. A level the model doesn’t list runs at the nearest level below it that the model lists (or at its lowest level when none is below), and the response says so inignored_params:
reasoning_effort=<level>; Responses names it reasoning.effort=<level>. On a stream, the X-Ignored-Params response header lists the names only. Match on params, not on the reason sentence.
To be sure of the level you get, send one the model lists. The Reasoning guide has a ready-made pick_effort helper in Python and TypeScript that maps the level you want to the nearest one a model takes.
Step 2: Pick an effort for the task
Effort is your main control over how long the model thinks, and so over cost and time. Reasoning tokens are billed as output tokens, whether or not any reasoning comes back. Turn effort up for:- Multi-step problems where one early mistake ruins the answer
- Math, scheduling, and anything with numbers that must add up
- Planning, where the model needs to weigh options and trade-offs
- Writing or reviewing code
none, on a model whose can_disable is true) for:
- Simple lookups and factual questions
- Friendly back-and-forth chat
- Anything where speed matters most, like a live chat window
"reasoning": {"effort": "low"}. If you send both, reasoning_effort wins.
Step 3: Read the reasoning back
Reasoning comes back only when the model reasoned. Claude 5 models decide how much each request needs and can skip reasoning on easy ones, so even with an effort set, an answer can arrive with no reasoning. Always treat it as optional. Claude, Gemini and OpenAI models return the provider’s summary of their reasoning, not the full chain of thought. Open-weight models such as DeepSeek, GLM and gpt-oss return their raw reasoning text.Chat Completions
Set any effort exceptnone, and read message.reasoning_content. The OpenAI SDKs don’t type that field, so read it with getattr in Python and a wider type in TypeScript:
delta.reasoning_content chunks before the first answer chunk.
Responses
Add"summary": "auto" (or concise or detailed) to reasoning. An effort alone makes the model think but returns no summary. The summary is a reasoning item in output[], before the message item:
response.reasoning_summary_text.delta events and the answer from response.output_text.delta. Models outside the Anthropic, Google and OpenAI families stream raw reasoning as response.reasoning_text.delta instead; handle both if you stream from more than one family. What comes back shows every event.
Step 4: Size the output limit
On a reasoning model, the output limit holds the thinking and the answer. A limit sized for the answer alone, such as 500 or 1,024, can be used up by the thinking before the answer starts. Pick a starting point from the effort you set and the answer you expect, then adjust it from theusage your requests report:
These are starting points, not guarantees: the same model can reason for much longer on a hard question than on an easy one. With no effort set, size for the model’s default level, which is often
medium or high.
What Gloo does with your limit
The ceiling is the model’s
max_output_tokens in the catalog: thinking and answer together. You’re billed only for the output tokens the model uses, not for the limit. Set your output limit for reasoning models has the full rules, including how none changes them.
Stream long generations
A large limit with a high effort can take minutes. A non-streamed request that runs past Gloo’s response deadline fails with3007 / GATEWAY_DEADLINE. Streaming avoids it and shows progress as it comes. If a stream fails partway, Handling Streaming Failures explains how to keep what the user has already seen and when to retry.
Step 5: Detect and handle a cut-off
A response that reached the limit says so:- Chat Completions:
finish_reason: "length", streamed or not. - Responses:
status: "incomplete"withincomplete_details: {"reason": "max_output_tokens"}. A non-streamed response can also carry asuggestionfield that names the tokens used and the limit you sent.
usage shows where the tokens went. Where the model reports it, usage.completion_tokens_details.reasoning_tokens (Chat Completions) or usage.output_tokens_details.reasoning_tokens (Responses) is the reasoning part of the output tokens. It isn’t reported everywhere: for Claude models it is 0 or missing on some endpoints, and their reasoning is still counted in the output tokens.
This helper checks every answer before it’s used and, on a cut-off, tries once more with double the limit:
status and incomplete_details:
Examples by endpoint
Each example sets an effort and reads the reasoning and the answer, non-streamed and streamed. Readreasoning_content with getattr in Python and a wider type in TypeScript. The Responses examples set max_output_tokens, because Claude models on Responses otherwise use the model’s default limit (see Step 4). Parameters the SDKs don’t know, such as the grounding parameters, go in extra_body in Python and top-level with a cast in TypeScript. If your SDK version doesn’t type a level you send (such as max), pass it in extra_body as well.
The Python and TypeScript snippets in each tab reuse the client created in that tab’s first example.
- Completions
- Grounded Completions
- Direct Completions
- Responses
- Grounded Responses
- Direct Responses
Completions guide · API referenceStreamed:
POST /ai/v2/guarded/chat/completions. The answer is moderated; the reasoning is not (details).How the endpoints differ
The guide’s How the endpoints differ table also covers passing reasoning back across turns.
Complete Example
This script puts the steps together for one model: it reads the model’s levels from the catalog, picks the level to send, makes the request with a sized limit, and on a cut-off tries once more with double the limit before printing the answer.What You’ll See
A line with the effort, limit and output tokens used, a short excerpt of the reasoning when the model returned some, and the answer: about 387 distinct people.Troubleshooting
The answer is empty or stops mid-sentence : Checkfinish_reason (Chat Completions) or status (Responses). "length" or "incomplete" with max_output_tokens means the model spent the limit thinking. Raise the limit or lower the effort; resending the same request usually ends the same way. See Step 5.
My max_tokens of 1,024 came back as a bigger number of output tokens
: On Chat Completions, a reasoning model’s limit below 8,192 is raised to 8,192 so it has room to think. You’re billed only for the tokens the model uses.
No reasoning comes back
: On Responses, add "summary": "auto" to reasoning; an effort alone returns no summary. On Chat Completions, set an effort other than none. Claude 5 models can also skip reasoning on easy requests, and models with returns_summary: false (such as Gemma) don’t guarantee it.
ignored_params lists my effort
: The model doesn’t list that level, so it ran at the nearest level below it that it does list. Send a level from the model’s effort_levels (Step 1).
none didn’t turn reasoning off, and I’m still billed for reasoning
: Only models with can_disable: true can turn it off. On others, such as Claude 5 and Gemini 2.5 Pro, none runs at the model’s lowest level and is listed in ignored_params. Use a model that can disable reasoning, or accept the lowest level.
Claude 4 models ignore the effort
: Claude 4 models such as Haiku 4.5, Sonnet 4.6 and Opus 4.5 to 4.8 don’t reason on these endpoints. To use their thinking, call them on Anthropic Messages.
reasoning_tokens is 0 but output tokens are high
: For Claude models, reasoning_tokens is 0 or missing on some endpoints. Their reasoning is still counted in the output tokens.
A long non-streamed request fails with 3007 / GATEWAY_DEADLINE
: It ran past Gloo’s response deadline. Stream it instead.
The streamed reasoning seems to repeat itself
: If a provider fails while the model is still reasoning, Gloo can retry on another provider, and the model reasons again from the start. On Chat streams both attempts’ reasoning is joined; on Responses streams the first reasoning item closes as incomplete and a new one opens. See When a provider fails mid-reasoning.
422 when I send reasoning back in an assistant message
: Completions and Grounded Completions refuse thinking parts in assistant content. Send earlier assistant turns as text; the model reasons afresh each turn.
TypeScript says 'max' isn’t a valid reasoning_effort
: Older openai SDK versions don’t type that level. Upgrade the SDK, or cast it.
Next Steps
Reasoning guide
Defaults, effort rules, output limits and behavior by model family.
Model catalog reference
GET /platform/v2/models, with each model’s reasoning object.Handling Streaming Failures
Retry, continue or surface a stream that fails partway.
Models That Think
What reasoning is, and when more effort pays off.
Direct Responses
Every model in the catalog with one request shape.
Anthropic Messages tutorial
Claude’s own thinking settings through the Anthropic SDK.

