Skip to main content
PATCH
Update an evaluation set
Changes a set’s name, description, test cases or tags. Send expectedUpdatedAt to refuse a stale edit with 409 conflict; an archived set cannot be edited (409 invalid_state), and the test cases of a set created in the Console or seeded (which the weekly regression evals run) can only be changed in the Console. The status, agent binding, dimensions and golden dataset cannot be changed. See Evals.

Authorizations

Authorization
string
header
required

An External App: the JWT it obtains from POST /oauth/token with OAuth2 client_credentials. Pass as Authorization: Bearer <jwt>. An app's static companion key also authenticates as the app, but is deprecated and is not protected by Idempotency-Key.

Headers

Idempotency-Key
string

1–255 printable ASCII characters (0x21–0x7E, no spaces), scoped to the calling credential. A repeat with the same key and request never runs the write again, a 5xx included: once the operation's permissions pass again it answers the identical status and body of the first response (Idempotent-Replayed: true, also sent as Idempotency-Replayed: true), unless the operation's own replay policy withholds it; a first response that was not stored (over 32 KiB, or carrying a one-time secret) answers its first status with only data.id (Idempotency-Replay-Disclosure: id-only) when it named one; otherwise 409 idempotency_result_unavailable; a repeat while the first is still running is 409 idempotency_key_in_flight, and the same key with a different request is 422 idempotency_key_reuse. Honoured for External App JWTs and platform and Personal keys; an External App's static companion key is not protected.

Required string length: 1 - 255

Path Parameters

evalId
string
required
Minimum string length: 1

Body

application/json

Changes to one evaluation set: at least one of name, description (null clears it), testCases (replaces every case) or tags. expectedUpdatedAt, when sent, must equal the set's updatedAt or the update is a 409 conflict. The status, agent binding, dimensions and golden dataset cannot be changed.

name
string
Required string length: 1 - 200
description
string | null
Maximum string length: 2000
testCases
object[]
Required array length: 1 - 200 elements
tags
string[]
Maximum array length: 20
Maximum string length: 64
expectedUpdatedAt
number

Response

Update an evaluation set

data
object
required

An EvalSet with its test cases, dimensions and golden dataset id instead of the test-case count. The creator is never returned.

meta
object
required