Skip to main content
GET
Compare evaluation runs
Compares 2-10 runs you may read, computed on read and never stored. Any hidden or missing run is one 404 for the whole request. comparable says whether the deltas mean anything; see Comparing runs.

Authorizations

Authorization
string
header
required

An External App: the JWT it obtains from POST /oauth/token with OAuth2 client_credentials. Pass as Authorization: Bearer <jwt>. An app's static companion key also authenticates as the app, but is deprecated and is not protected by Idempotency-Key.

Query Parameters

ids
string
required

2-10 distinct run ids, comma-separated, in the order to compare. One ids parameter only.

Minimum string length: 1

Response

Run comparison

data
object
required

A comparison computed on read and never stored. runs and every scores array follow the requested order; delta and overallScoreDelta are last minus first when both are numbers, else null. comparable is false when the runs span evaluation sets (different_eval_sets), any run is not completed (not_completed), or any run has failed or unfinished test cases (incomplete: a run finalizes as completed even when some cases never finished, and failed cases are left out of its scores); the deltas are still returned with the reasons.

meta
object
required