Start an evaluation run
Queues a run of an active evaluation set against modelId and answers 201 with the pending run. Requires permissions agents:write and inference:use.
modelId and answers 201 with the pending run. Starting a run queues it. Each test case makes two metered model calls (the chosen model and the fixed judge model); cost accrues as cases run and is reported in totalCost (target plus judge) when the run finalizes. completed means finalized: testCasesCompleted below testCasesTotal means some cases never finished. Eval spend is recorded under the platform’s eval runner, not your organization’s budget.
Machine-started runs are limited to 200 test cases and 2 in-progress runs per credential, and are refused while the platform’s eval-runner, eval-judge or global budget is frozen or exhausted. See Evals.Authorizations
Headers
1–255 printable ASCII characters (0x21–0x7E, no spaces), scoped to the calling credential. A repeat with the same key and request never runs the write again, a 5xx included: once the operation's permissions pass again it answers the identical status and body of the first response (Idempotent-Replayed: true, also sent as Idempotency-Replayed: true), unless the operation's own replay policy withholds it; a first response that was not stored (over 32 KiB, or carrying a one-time secret) answers its first status with only data.id (Idempotency-Replay-Disclosure: id-only) when it named one; otherwise 409 idempotency_result_unavailable; a repeat while the first is still running is 409 idempotency_key_in_flight, and the same key with a different request is 422 idempotency_key_reuse. Honoured for External App JWTs and platform and Personal keys; an External App's static companion key is not protected.
1 - 255Body
The evaluation set to run and the model to run it against (a modelId from GET /api/v2/models/list). Starting a run queues it. Each test case makes two metered model calls (the chosen model and the fixed judge model); cost accrues as cases run and is reported in totalCost (target plus judge) when the run finalizes. completed means finalized: testCasesCompleted below testCasesTotal means some cases never finished. Cancelling stops cases that have not started; cost already incurred is not refunded and recorded results remain. Eval spend is recorded under the platform's eval runner, not your organization's budget. Machine-started runs are limited to 200 test cases and 2 in-progress runs per credential, and are refused while the platform's eval-runner, eval-judge or global budget is frozen or exhausted.

