> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM routing tier abstraction

> Categorical tiers replace hard-coded model strings. Authors declare a tier; the platform resolves (tier × provider × runtime conditions) into a concrete model at dispatch time.

Forge's LLM routing layer lets agent authors declare *intent* (this work is cheap classification; this work is heavy reasoning) instead of pinning a specific model string. The platform resolves the intent into a concrete model at dispatch time, applies fallback when the primary provider is unavailable, and writes telemetry that drives future arbitration decisions.

Callers use the platform resolver rather than duplicating provider and fallback selection. Agent composition consumes its routing decisions today; a procedure executor is planned (see [Consumers](#consumers)).

## Five tiers

The canonical tier set is ordered cheapest-to-most-capable:

| Tier | Typical use | Class |
| - | - | - |
| `nano` | Cheap classification, simple extractions | Haiku-class |
| `low` | Structured outputs over small contexts | Haiku-class |
| `mid` | General-purpose reasoning | Sonnet-class |
| `high` | Complex reasoning, large contexts | Opus-class |
| `frontier` | Best-available at dispatch time | Currently Opus-4.7 |

`nano` and `low` map to the same concrete model today; the distinction exists so authors can express intent. When a future model serves the `nano` band more cheaply than `low`, the map updates without code changes at consumer sites.

## Resolution pipeline

Routing walks four stages to turn a tier into a concrete model:

LLM ROUTING — TIER → CONCRETE MODEL

<ol />

## Reconciliation with legacy infrastructure

Three pre-existing pieces of routing infrastructure remain operational and are consulted by the new layer rather than replaced:

| Configuration | Purpose | How the new layer uses it | | | | |
| - | - | - | - | - | - | - |
| Model routing configuration | Per-agent or org-wide mapping of three legacy tiers (\`low | medium | high\`) to concrete models. | Routing reads it when the agent or org has one, then falls back to the built-in tier map. The new 5-tier scheme maps onto the 3-tier scheme: \`legacy.low ⇄ new\.nano | low`, `legacy.medium ⇄ new\.mid`, `legacy.high ⇄ new\.high | frontier\`. |
| [Fallback chains](https://docs.gloo.com/forge/console/platform/fallback-chains.md) | Ordered model lists with semantic `purpose` (`simple-reasoning`, `average-reasoning`, `heavy-reasoning`, `embedding`, `deep-research`). | Consumed by the fallback walk when the primary provider for a tier is unavailable. The purpose vocabulary maps onto the new tier set in the obvious way. | | | | |
| Model slots | Named slots (`primary-reasoner`, `fast-coder`) with environment-aware mappings. | Orthogonal to tiers — slots can override a specific dispatch site within a tier band without changing the tier itself. | | | | |

The two tier vocabularies (3-tier `low|medium|high` and 5-tier `nano|low|mid|high|frontier`) coexist deliberately. `mid` ≠ `medium` is the intentional rename — 5-tier expressiveness is what authors get, 3-tier parity is what legacy storage keeps.

## Consumers

| Consumer | What it asks for |
| - | - |
| [Agent composition](https://docs.gloo.com/forge/architecture/flows/workflow-execution.md#2-composition) | The agent's model configuration, with runtime context (workspace, dispatch source). Returns the concrete model to build. |
| Inference route | A model string the caller passed; the inference service resolves it against the active provider chain and applies the allowed fallback posture. |
| Future procedure executor | Same as agent composition — a tier declared on the procedure manifest. |

## What this is not

* **Not a load balancer.** The routing layer picks a provider per dispatch; it does not split traffic across providers for the same dispatch.
* **Not a model registry.** Model identities and capabilities live in `modelRegistry`. The routing layer consumes that registry but does not own it.
* **Not OpenAI-compatible by itself.** OpenAI-compat is a separate concern handled by the [LLM proxy](https://docs.gloo.com/forge/reference/inference.md#llm-proxy-/api/llm/v1/) and the [External App inference carve-out](https://docs.gloo.com/forge/reference/inference.md#external-app-inference-carve-out-/api/inference/v1/). The routing layer is the engine behind the LLM proxy's model-selection step.

***

## Related

* [Inference surfaces](https://docs.gloo.com/forge/reference/inference.md)
* [Models HTTP API](https://docs.gloo.com/forge/reference/http-api/models)
* [Agent composition](https://docs.gloo.com/forge/architecture/flows/workflow-execution.md#2-composition)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.