> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Author a judge

> Add a rubric-backed system Judge, integrate its subject and dispatch boundary, and verify verdict quality.

A [Judge](https://docs.gloo.com/forge/concepts/trust-fabric.md#judges) evaluates a declared subject against a rubric and returns `JudgeVerdictV1`. Adding a new system Judge is a platform contributor task: it requires source access, a seeded system agent, a subject contract, a dispatch integration, and tests. It is not the same as creating an ordinary Worker agent through the public agent-creation API.

For an operator reviewing an existing Judge, start with [Judges in Forge Console](https://docs.gloo.com/forge/console/platform/judges).

## 1. Define the subject and rubric

State exactly what the Judge receives, which evidence it may use, which decisions it informs, and how a caller handles missing or low-confidence evidence. Keep the subject bounded and canonical so the same input can be compared across rubric revisions.

Author a rubric skill with `SKILL.md` and `forge.yaml` using [skill authoring](https://docs.gloo.com/forge/build/author-a-skill.md). Describe the criteria, weights, score bands, examples, and output contract. The built-in architect and skill-quality rubric skills provide concrete examples.

Keep `rubricVersion` synchronized with the rubric that actually runs. A version label should identify an evaluative change, not just a file edit.

## 2. Register a system Judge

Use an existing system Judge as the complete pattern. The system-agent seed contract uses fields such as `displayName`, `modelRef`, `skillIds`, and `toolSetIds`; it is distinct from the stored agent shape and from HTTP request bodies.

The dispatch admission invariant requires an agent of type **SYSTEM**, subtype **JUDGE**, tagged `judge`. Its `judgeMetadata` includes the domain, rubric skill and version, output schema version, confidence requirement, and `minConfidenceForAuthoritative`. Configure the rubric skill for `always` activation and select an enabled model appropriate for the rubric. Grant only the tools the subject requires.

The threshold is a confidence cutoff: the dispatch marks a valid verdict authoritative when its confidence meets the configured minimum. Raising that minimum makes the cutoff stricter. It does not itself improve the Judge's calibration or make a low-confidence result authoritative.

For agent-facing evaluation tools, register the Judge ID in the Judge registry. Those wrappers use an allowlist to prevent arbitrary system-agent dispatch. Internal callers use a fixed Judge ID and the dispatch admission invariant rather than that name-level allowlist.

## 3. Integrate dispatch

The internal entry point is `dispatchJudge`, called through `ctx.runAction` with its generated internal reference. Its required arguments are:

| Argument | Meaning |
| - | - |
| `judgeAgentId` | The registered Judge's stable agent ID. |
| `subject` | The bounded, canonical subject defined by the caller's contract. |
| `subjectKind` | The subject category understood by this integration. |
| `subjectId` | The concrete subject's stable identity. |
| `triggeringDispatchSite` | The source integration responsible for the evaluation. |

Optional arguments include a criterion subset, triggering dispatch ID, and parent evidence handle. The result contains the parsed verdict and its `calibrationRowId`. Do not pass a subject-builder function as an action argument: build the serializable subject first.

Use the existing architect and skill-quality auto-triggers to understand triggering and deduplication. For panels, use `dispatchJudgePanel` and the current consensus contract; see [Judges](https://docs.gloo.com/forge/concepts/trust-fabric.md#judges). Recurring triggers belong in the platform cron manifest, following the repository's scheduling contract.

Dispatch governs the evaluation, composes the agent, parses the verdict, and records calibration evidence. Malformed output can receive one corrective retry; a budget timeout is a distinct failure. The caller must distinguish a valid verdict, a non-authoritative verdict, a held evaluation, and an execution failure instead of treating all returned or missing results as approval.

## 4. Verify the integration

Before enabling a Judge for a consequential decision:

1. Test representative accepted, rejected, ambiguous, and missing-evidence subjects.
2. Confirm the emitted verdict validates against the canonical Judge verdict schema and names the expected subject and rubric.
3. Exercise governance refusal, invalid output, timeout, and low-confidence handling.
4. Verify the caller consumes the authoritative/advisory distinction and records the expected result.
5. Inspect the verdict in Console and test any supported suggested action against its actual route.

Do not invent a new UI routing registry from an old example. New suggested-action kinds require the current renderer and action-routing contract to be checked together.

## 5. Calibrate and maintain

Use recorded operator overrides and representative evaluations to find systematic errors in criteria, bands, or evidence selection. Change the rubric deliberately and compare equivalent subject sets across versions. Judge metrics are rolled up by the manifest-backed `judge metrics rollup` job; their existence does not guarantee a consumer automatically disables a drifting Judge.

A confidence score is not a measured accuracy rate. Keep the consumer's decision policy explicit, and define what happens when model, rubric, evidence, or subject shape changes. See [Console Judges](https://docs.gloo.com/forge/console/platform/judges) for the available monitoring and override surfaces.

## Do it over the API

Registering and integrating a Judge is source work; there is no API call that creates one. Once it is deployed, these reads confirm that it is registered with the rubric and threshold you set and that its verdicts carry the expected subject. They use the `FORGE_BASE` set up in the Raw HTTP tab of the [quickstart](https://docs.gloo.com/forge/quickstart.md). The quickstart's `ACCESS_TOKEN` does not carry every scope they need, so mint it again with `scope=platform:read` from an app whose roles hold them (see [Requesting scopes at token mint](https://docs.gloo.com/forge/reference/conventions.md#requesting-scopes-at-token-mint)), or send a [Personal key](https://docs.gloo.com/forge/console/settings/personal-keys.md#choose-scopes) whose scopes include them.

Find the Judge and check its rubric version, confidence threshold and calibration health:

```bash theme={null}
curl -sS "$FORGE_BASE/api/v3/judges" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  | jq '.data.judges[] | {id, name, tier, rubricVersion, minConfidenceForAuthoritative, calibrationHealth}'
```

Then read its newest verdicts, using that `id` as `JUDGE_ID`:

```bash theme={null}
curl -sS "$FORGE_BASE/api/v3/judges/$JUDGE_ID/verdicts?limit=20" \
  -H "Authorization: Bearer $ACCESS_TOKEN" \
  | jq '.data.verdicts[] | {subjectKind, subjectId, rubricVersion, verdictLabel, confidence, isAuthoritative, hasOperatorOverride}'
```

Both reads need `platform:read` held at the platform scope; a credential whose grant covers only some org units gets `403`. Verdicts on a Personal agent, or on an agent that no longer exists, are left out, so a page can hold fewer than `limit` while `meta.hasMore` is true; follow `meta.nextCursor` to read on.

Over MCP, `forge_call_read_operation` reads the same verdicts. It is in the default `start` job:

```json theme={null}
{ "operationId": "listJudgeVerdicts", "input": { "judgeId": "<judge id>", "limit": 20 } }
```

## Related

* [Judges](https://docs.gloo.com/forge/concepts/trust-fabric.md#judges): what judges grade and how verdicts count.
* [Judges in Forge Console](https://docs.gloo.com/forge/console/platform/judges): the catalog and admissions an operator manages.
* [Author a skill](https://docs.gloo.com/forge/build/author-a-skill.md): the skill format a rubric builds on.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.