Skip to main content
Forge’s LLM routing layer lets agent authors declare intent (this work is cheap classification; this work is heavy reasoning) instead of pinning a specific model string. The platform resolves the intent into a concrete model at dispatch time, applies fallback when the primary provider is unavailable, and writes telemetry that drives future arbitration decisions. Callers use the platform resolver rather than duplicating provider and fallback selection. Agent composition consumes its routing decisions today; a procedure executor is planned (see Consumers).

Five tiers

The canonical tier set is ordered cheapest-to-most-capable: nano and low map to the same concrete model today; the distinction exists so authors can express intent. When a future model serves the nano band more cheaply than low, the map updates without code changes at consumer sites.

Resolution pipeline

Routing walks four stages to turn a tier into a concrete model:

Reconciliation with legacy infrastructure

Three pre-existing pieces of routing infrastructure remain operational and are consulted by the new layer rather than replaced: The two tier vocabularies (3-tier low|medium|high and 5-tier nano|low|mid|high|frontier) coexist deliberately. mid ≠ medium is the intentional rename — 5-tier expressiveness is what authors get, 3-tier parity is what legacy storage keeps.

Consumers

What this is not

  • Not a load balancer. The routing layer picks a provider per dispatch; it does not split traffic across providers for the same dispatch.
  • Not a model registry. Model identities and capabilities live in modelRegistry. The routing layer consumes that registry but does not own it.
  • Not OpenAI-compatible by itself. OpenAI-compat is a separate concern handled by the LLM proxy and the External App inference carve-out. The routing layer is the engine behind the LLM proxy’s model-selection step.