> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference Carve-Out

> Two OpenAI-compatible surfaces. /api/inference/v1/* is the External-App carve-out (JWKS-signed JWT). /api/llm/v1/* is the LLM proxy (virtual-key auth) backed by the routing tier abstraction.

Two surfaces, same OpenAI URL shape — do not confuse them. `/api/inference/v1/*` is the External-App carve-out authenticated via OAuth2-minted JWT; `/api/llm/v1/*` is the platform-internal LLM proxy authenticated via virtual-key bearer. The `v1` in either path is an OpenAI ecosystem convention, not the Forge management API version (which is `v2`).

Forge ships **two distinct OpenAI-compatible surfaces**, each with its own auth model, route prefix, and caller:

| Surface | Path prefix | Auth | Primary caller |
| - | - | - | - |
| **External App inference carve-out** | `/api/inference/v1/*` | JWKS-signed JWT minted via OAuth2 client-credentials | Third-party services that ship a Forge integration as if it were OpenAI/Anthropic. |
| **LLM proxy (routing tier abstraction)** | `/api/llm/v1/*` | Virtual-key bearer token | Platform-internal agents and procedures — composed via the [LLM routing tier abstraction](https://docs.gloo.com/forge/architecture/llm-routing.md). |

This page documents both. They follow the OpenAI URL convention so SDKs work against either with a base-URL swap and nothing else.

## External App inference carve-out: `/api/inference/v1/*`

### Why it's preserved

When the legacy `/api/v1/*` management surface was retired, the inference paths were deliberately kept. The reasoning: the OpenAI/Anthropic-compatible SDK ecosystem hardcodes `/v1/chat/completions`-style paths, and breaking that breaks every consumer that mounts Forge as an OpenAI base URL.

The `v1` in `/api/inference/v1/*` is **a third-party convention, not the Forge management API version**. The two version namespaces are independent:

* **Forge management API** is at `/api/v3/*` (and the `/api/v2/*` compatibility surface) and evolves under Forge's own versioning policy.
* **`/api/inference/v1/*`** mirrors the OpenAI URL shape and changes only when the upstream OpenAI surface changes.

### Endpoints

| Path | Purpose |
| - | - |
| `POST /oauth/token` | OAuth2 `client_credentials` grant (and the WorkOS token-exchange grant). Mints a JWT for inference and/or management use. Also mounted at `POST /api/inference/v1/oauth/token`, which is deprecated: it answers `Deprecation` and `Link: </oauth/token>; rel="successor-version"`. Use `/oauth/token`. |
| `GET /api/inference/v1/.well-known/jwks.json` | JWK set used to verify Forge-signed JWTs. |
| `POST /api/inference/v1/chat/completions` | OpenAI-compatible chat-completions endpoint. Accepts the standard `messages` / `model` / `tools` shape. |
| `GET /api/inference/v1/models` | OpenAI-compatible model catalog. |

Both mounts resolve to the **same handler instance**, so both grant types behave identically and the two paths cannot drift. The JWKS used to verify the minted JWT is likewise served at both `GET /.well-known/jwks.json` and `GET /api/inference/v1/.well-known/jwks.json`. Prefer the root paths — they are what every quickstart and SDK example teaches.

### Compatibility

Most OpenAI and Anthropic SDKs work out of the box if you point their base URL at `https://api.forge.gloo.com/api/inference/v1` and supply a Forge-issued bearer token.

```typescript theme={null}
// OpenAI SDK example
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.forge.gloo.com/api/inference/v1",
  apiKey: process.env.FORGE_INFERENCE_TOKEN!,
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "hello" }],
});
```

The Forge inference layer routes the request to the underlying provider configured for the model and applies governance, cost attribution, and audit logging on the way through. From the SDK's perspective, the call is indistinguishable from a direct OpenAI call.

### Uploading a file for knowledge

An app adds a file to its knowledge in three calls:

1. `POST /api/inference/v1/files` with `{ "filename": "...", "mime_type": "..." }` answers `201` with `{ "file_id", "upload_url" }`.
2. `POST` the file's raw bytes to `upload_url`. It answers `200` with `{ "storageId" }`. The URL carries a single-use token, so send no `Authorization` header to it and do not share it. It accepts one upload of 1 to 20,971,520 bytes (20 MiB) within an hour of the `POST /files` call. A larger body answers `413` `payload_too_large`, an empty body `400` `empty_upload`, a second upload `409` `upload_already_received`, an upload after the hour `403` `upload_url_expired`, and a URL that matches no pending file `403` `invalid_upload_url`. Like the other file routes, it answers `401` `app_not_active` when the app is no longer active and `503` `governance_door_stopped` while the `api_keys` door is stopped.
3. `POST /api/inference/v1/files/confirm` with `{ "file_id", "storage_id" }`, where `storage_id` is the `storageId` the upload answered. The file moves to `processing`.

Confirm accepts only the bytes this file's own upload delivered. Any other `storage_id` answers `400` `storage_id_mismatch` with a sentence ending `Reference: <id>.`, and the same id in `decision_id`. The refusal is recorded as a decision, and the file stays `pending_upload`.

### Access control

Every `/api/inference/v1/*` route an app token calls — chat completions, the agent card and the inference file routes (`/files`, `/files/confirm`, `/files/get`, `/files/delete`) — requires the `inference:use` permission. The token must carry the `inference:app` scope, and the app must hold `inference:use` through its role grants; chat completions and the agent card answer a token without the scope with `401` (`wrong_scope`), and every route answers `403` (`missing_permission`) when the app's roles, narrowed by the token's scopes, lack the permission. A token carrying only `files:read` or `files:write` therefore gets `403` on the inference file routes. See [Permissions and scopes](https://docs.gloo.com/forge/reference/conventions.md#permissions-and-scopes).

#### While Forge is stopped

When a person stops Forge in the Console (the global kill switch), `/chat/completions` and the inference file writes (`POST /files`, the upload to `upload_url`, `POST /files/confirm`, `DELETE /files/delete`) answer `403` with `code` `platform_killed` and the fixed message "Forge is stopped by an operator. A person resumes it in the Console." Chat completions refuse after the token's own scope and permission checks (which keep their `401`/`403` answers) and before any user is resolved or provisioned or any model runs; the file writes refuse before any upload URL, row or deletion. The operator's reason is never returned. File reads (`GET /files`, `/files/get`) keep working. Retry after a person resumes Forge.

The `/chat/completions` endpoint then enforces two further levels of access control sourced from the calling External App's configuration:

#### Agent gate — `allowAllAgents`

By default (`allowAllAgents: false`), an External App may only use agents listed in its `allowedAgentIds`. When `allowAllAgents: true` the app can address any active agent.

#### Model gate — `allowAllModels`

Each agent carries an `externalModelAllowlist` — the set of model IDs external callers may request through that agent. When a request specifies a `model` not in the list, the endpoint returns:

```json theme={null}
{
  "error": {
    "type": "model_not_found",
    "code": "model_not_permitted",
    "message": "...",
    "param": null,
    "docUrl": "https://docs.gloo.com/forge/reference/errors#bad_request"
  },
  "meta": { "requestId": "req_def456" }
}
```

Set `allowAllModels: true` on the External App row (via `PATCH /api/v3/external-apps/{externalAppId}`) to bypass this check for that app — the caller may then request any model string and Forge will route it directly. The agent's own `externalModelAllowlist` is still respected for apps without this flag; `allowAllModels` defaults to `false` (safest).

Both flags are managed through the [External Apps management API](https://docs.gloo.com/forge/reference/http-api/externalApps).

### Not part of the management API

Operations in this section are NOT covered by the v2 OpenAPI spec — they follow the OpenAI shape, not Forge's REST-RPC envelope. In particular:

* **No `data` / `meta` envelope.** Responses use OpenAI's native shape (`choices`, `usage`, etc.).
* **Streaming uses SSE in OpenAI's format**, not Forge's management envelope.
* **Errors keep OpenAI's structure**, `error.{ message, type, param, code }`, so an OpenAI SDK reads them. Each also carries `error.docUrl` and `meta.requestId` from Forge's [error envelope](https://docs.gloo.com/forge/reference/errors.md#details-on-the-other-surfaces), with the id in the `X-Request-Id` header.

For everything **operator-facing** — managing agents, dispatching workflows, reading runs, granting approvals — use the [v2 management API](https://docs.gloo.com/forge/reference.md).

***

## LLM proxy: `/api/llm/v1/*`

The LLM proxy is a separate OpenAI-compatible surface used by platform-internal callers — Forge's own agent turns, procedure sandboxes, and any future caller that wants Forge to act as a tier-routed LLM gateway rather than a transparent passthrough. It's distinct from the External App carve-out: the auth model, the access-control model, and the routing semantics all differ.

| Aspect | External App carve-out | LLM proxy |
| - | - | - |
| Path | `/api/inference/v1/*` | `/api/llm/v1/*` |
| Auth | JWKS-signed JWT (OAuth2 client-credentials) | Virtual-key bearer token (`Authorization: Bearer <vk-token>`) |
| Caller | External integrations | Internal agents, procedures |
| Model selection | Caller specifies the model string; allowlist enforced per agent | Caller specifies the model string; the routing tier layer handles fallback and arbitration |
| Routing | Direct provider passthrough | Tier-aware via [LLM routing](https://docs.gloo.com/forge/architecture/llm-routing.md) — fallback chains, tier downgrade, telemetry |

### Endpoints

| Path | Purpose |
| - | - |
| `POST /api/llm/v1/chat/completions` | OpenAI-compatible chat completions. Virtual-key auth required. Delegates to the `llmProxy.completions` action, which routes through the [LLM routing tier abstraction](https://docs.gloo.com/forge/architecture/llm-routing.md). |
| `GET /api/llm/v1/models` | Returns the model catalog from `modelRegistry` in OpenAI-compatible format. |

### Virtual keys

Virtual keys are minted server-side and bound to a calling principal (worker, factory, etc.). The token is opaque from the caller's perspective; Forge resolves it to a `virtualKeyId` server-side, enforces the principal's quotas and policies, and attributes the resulting token spend back through the token spend attribution chain.

Because the proxy is a platform-internal surface, virtual keys are not externally provisioned — they are minted by the dispatch path that needs them. Operators who want a long-lived OpenAI-compatible inference URL for a third-party SDK should use the [External App carve-out](#external-app-inference-carve-out-/api/inference/v1/) instead.

***

*See also: [LLM routing tier abstraction](https://docs.gloo.com/forge/architecture/llm-routing.md) | [External Apps](https://docs.gloo.com/forge/reference/http-api/externalApps) | [Models](https://docs.gloo.com/forge/reference/http-api/models)*


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.