Skip to main content
Models change underneath every AI application: providers release better ones and retire old ones, sometimes at a different price. This page is for developers running Gloo AI in production who want to choose a model deliberately, keep it fixed, and move to the next one when they are ready rather than when a provider decides.
New to choosing models? Choosing a Model in the AI Learning Center explains the trade-offs, and Model Lifecycle is the policy this page builds on.
This page covers how to:
  • pick a model from the catalog by what your task needs
  • pin the model ID so it doesn’t change between calls
  • notice deprecations early, from the catalog and from response headers
  • move to a new model after testing it, on your own timeline

Pick from the catalog by what the task needs

The model catalog, GET /platform/v2/models, is public and needs no API key. Each entry describes what the model can do, so you can rule models in or out before you try them: Filter the catalog in code rather than copying IDs by hand. This picks every current chat model that calls tools and reads images, cheapest output first:
Then try two or three candidates on real examples from your app. The Chat Playground is a quick first look; for a decision you can repeat, run a small test set as described in Testing, Evaluation and Observability. A few rules of thumb:
  • Match the model to the step, not the app. A classification or a short extraction rarely needs the largest model. Use a smaller, faster one there and keep the large model for the steps that need it. See Controlling Cost.
  • Check reasoning behaviour, not just quality. A model with thinks_by_default: true reasons, and bills that reasoning as output, even when you send no reasoning setting. If you want short, fast answers, pick a model with can_disable: true. See Reasoning Models in Production.
  • Leave headroom in the context window. Size for your longest real conversation plus the documents you add to it, not your average one. See Conversations and Context Windows.

Pin the model ID

Send the exact id from the catalog as model on every request, and keep it in configuration rather than scattered through your code, so changing models is a one-line change you can test and roll back. A pinned ID gives you the same model on every call, so behaviour, cost and latency stay predictable. The Responses endpoints run the model you name (or its replacement, once that model is deprecated). On guarded Completions, you can instead let Gloo choose: model_family picks a model from a provider’s family, and opt-in auto-routing picks one per request. Both can change the model from one call to the next, so use them where that suits the task, and pin a model for tool calling, structured output, anything you test against, and anything you need to reproduce. Take IDs from the catalog’s id field. The gloo-<provider>-<model> pattern is a convention, not a guarantee, so don’t build IDs from parts.

Notice deprecations early

A deprecated model doesn’t stop working on the day it’s flagged. For a generation model with a replacement_model, requests for its ID are served by the replacement, so your app keeps running while you migrate. The replacement can behave differently and can cost more: a replacement’s price is normally capped at 1.3 times the old rate on input or output, and any exception is stated in deprecation_note. See How a replacement is chosen. Two signals tell you a model you use is on its way out. Watch both. To see what’s coming across the whole catalog, Model Updates lists every deprecation, retirement and redirect with its date and replacement model.

Check the catalog on a schedule

Run a check daily, or in CI before each deploy, against the model IDs your app uses. Fail loudly when one is deprecated or missing:
Read deprecation_note when a model is flagged: it says when the model retires, what replaces it and whether the price changes. When replacement_model is empty, nothing will serve the old ID after its retirement date, and choosing the next model is up to you.

Watch the response headers

A successful response served by a replacement carries X-Gloo-Model-Redirected-From with the ID you asked for, Deprecation: true, and Sunset with the retirement date when one is set. The response’s model names the model that actually answered. Log these on every response and alert when they appear:
Treat Sunset without X-Gloo-Model-Redirected-From as urgent. It means nothing replaces the model: from 00:00 UTC on that date, requests are refused with 410 and the code model_retired, which isn’t retryable. On Anthropic Messages, a retired Claude model returns 404 with model_retired instead of being served by its replacement. See When nothing replaces a model.

Move to a new model on your own timeline

Automatic routing to a replacement is there so a retirement doesn’t break your app. It isn’t a substitute for testing: the replacement may answer differently, reason differently or cost more. Move deliberately, whether a model was deprecated or a better one has arrived:
  1. Read the note and the catalog entry. Compare the new model’s price, context_window, max_output_tokens, capability flags and reasoning object with the old one. A replacement with a smaller context window is called out in deprecation_note.
  2. Run your test set on both. Compare quality on your own examples, the usage each request reports, and response times. Check structured output and tool calls especially, since small changes in format break code that reads them.
  3. Revisit your settings. Reasoning efforts and output limits that suited one model may not suit the next: check effort_levels and resize max_output_tokens from the new model’s usage.
  4. Switch the configured ID and watch. Roll out to a share of traffic first if you can, and compare error rates, cut-off answers and cost before moving everything.
  5. Keep the old ID only as long as it’s listed. An ID you can still see in the catalog is supported; once its entry is removed, continued routing is a courtesy, not something to build on. See Notice before removal.
You don’t have to migrate the moment a generation model is flagged, but don’t wait for the retirement date either.

Embeddings don’t follow replacements

Don’t assume a deprecated embedding model’s requests go to a replacement. Vectors from different models aren’t comparable, so changing an embedding model means re-embedding everything you’ve stored before you query it with the new model.
  • Store the embedding model ID, and the dimensions you asked for, alongside every vector.
  • When the catalog flags your embedding model, plan a re-embedding job, build the new index alongside the old one, and switch queries over only when it’s complete.
  • Embedding requests are not failed over to another provider. See Provider Failover.

Summary

  • Choose from the catalog by capability flags, context window, reasoning behaviour and price, then test candidates on your own examples.
  • Pin the exact catalog id in configuration; use model_family or auto-routing only where a changing model is fine.
  • Check is_deprecated on a schedule, and log X-Gloo-Model-Redirected-From, Deprecation and Sunset on every response.
  • Treat Sunset without a redirect header as urgent: 410 model_retired follows on that date.
  • Test a new model on your own examples before you switch, and re-embed stored content when an embedding model changes.

Next steps

Model Lifecycle

How deprecations are announced, how replacements are chosen and how much notice you get.

Model Updates

Every model added, deprecated, retired or redirected, with dates and replacements.

Supported Models

The live catalog with capabilities and rates.

Production-Ready

The lesson on routing, failover and models that come and go.

Testing, Evaluation and Observability

Compare models on your own examples before you switch.