> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Tools, Structured Output and Agents

> Best practices for tool calling, JSON output and agent loops on Gloo AI: writing tool definitions, validating arguments, bounding loops, guarding side effects, and getting reliable structured output.

[Tool calling](https://docs.gloo.com/ai-learning-center/gloo-ai-104/tools-and-mcp.md) and [structured output](https://docs.gloo.com/ai-learning-center/gloo-ai-104/structured-output.md) are where a model's answer stops being text for a person and becomes input to your code. That's when mistakes get expensive: malformed JSON breaks a parser, a loop runs forever, a retried request sends the same email twice. This page is for developers building features and [agents](https://docs.gloo.com/ai-learning-center/gloo-ai-104/coding-agents.md) that act on what the model returns.

> [!TIP]
>
> For the request and response shapes, see the [Tool Use guide](https://docs.gloo.com/api-guides/tool-use.md). The [Tool Use tutorial](https://docs.gloo.com/tutorials/completions-tool-use.md) builds a working tool loop step by step.

This page covers how to:

* choose a model that calls tools, and pin it
* write tool definitions the model uses well
* validate every argument before you act
* bound the tool loop and make side effects safe
* get structured output you can rely on
* treat tool results and connected servers as untrusted

## Choose a model that calls tools, and pin it

Check `supports_tools` in the [model catalog](https://docs.gloo.com/api-guides/supported-models.md) before you send `tools`. A request with `tools` to a model that doesn't support them is refused with a `400` that says to remove the `tools` parameter or choose another model. Most chat models qualify; image-generation models and some open-weight models don't.

Pin a `model` for anything that calls tools. On guarded [Completions](https://docs.gloo.com/api-guides/completions-v2.md#choose-how-the-model-is-selected), `model_family` and [auto-routing](https://docs.gloo.com/getting-started/glossary.md#auto-routing) can pick a different model per request, and models differ in how they choose tools and format arguments. The [Tool Use guide](https://docs.gloo.com/api-guides/tool-use.md#tool-calling-with-v2-routing) covers how each routing mode handles tools.

## Write tool definitions for the model

The model decides which tool to call, and with what, from your definitions alone.

* **Name the action:** `get_service_times`, not `tool1` or `api_call`.
* **Say when to use it** in the description, in a sentence or two, and when not to.
* **Make parameters specific.** Use `enum` for fixed sets, mark what's `required`, give formats in descriptions ("ISO 8601 date, for example 2026-11-15"), and keep nesting shallow.
* **Offer few tools per request.** A handful of clearly different tools works better than dozens of overlapping ones. Send only the tools that make sense for the current step.
* **Keep definitions stable.** Identical tool definitions in the same order help [prompt caching](https://docs.gloo.com/ai-learning-center/gloo-ai-104/cost-and-speed.md#prompt-caching); see [Controlling Cost](https://docs.gloo.com/best-practices/cost-control.md#get-more-cache-hits).
* **Use the nested schema on Responses too.** Define tools as `{"type": "function", "function": {...}}` on both APIs; see [Defining tools](https://docs.gloo.com/api-guides/tool-use.md#defining-tools).

## Validate every argument before you act

`arguments` is a JSON string written by the model. Treat it like input from a form on the internet: parse it, check it against the schema you sent, and check the values make sense for this user, before you run anything. When a check fails, don't crash and don't guess: return the error to the model as the tool's result, so it can correct the call.

## Bound the loop and guard side effects

A tool loop runs until the model answers in text. Without a limit, a confused model can call tools indefinitely, and every turn resends the whole conversation and bills it again.

* **Cap the steps.** Stop after a fixed number of model turns, such as 8, and tell the user the task didn't finish.
* **Run every call the model asks for.** A turn can contain several tool calls. Return one result per call, matched by its ID.
* **Separate reads from writes.** Run read-only tools freely. For tools that change something (send a message, charge a card, update a record), ask the user to confirm first, or at least log them and limit how often they can run.
* **Make writes idempotent.** Give each side-effecting call an idempotency key derived from the conversation and the call ID, and have your tool ignore a key it has already processed.
* **Never auto-retry a request after a side effect has run.** If the model's request fails after your tool has already acted, resending the whole turn can make the model call it again. See [Tool Calls and Side Effects](https://docs.gloo.com/best-practices/completions-streaming-failures.md#tool-calls-and-side-effects).

This loop applies all of these on a Chat Completions endpoint. It works the same on the direct path shown and on `/ai/v2/guarded/chat/completions`:

```python Python theme={null}
import json
import os

import jsonschema
import requests

API_URL = "https://platform.ai.gloo.com/ai/v2/direct/chat/completions"
HEADERS = {"Authorization": f"Bearer {os.environ['GLOO_API_KEY']}"}
MODEL = "gloo-openai-gpt-6-luna"
MAX_STEPS = 8

SERVICE_TIMES_SCHEMA = {
    "type": "object",
    "properties": {"day": {"type": "string", "enum": ["saturday", "sunday"]}},
    "required": ["day"],
    "additionalProperties": False,
}


def get_service_times(day: str) -> dict:
    return {"day": day, "times": ["9:00", "11:00"]}


TOOLS = {
    "get_service_times": {
        "schema": SERVICE_TIMES_SCHEMA,
        "run": get_service_times,
        "description": "Get this church's service times for a weekend day.",
    },
}


def run_tool(call: dict) -> str:
    """Run one tool call and return its result as a string, including errors the model can fix."""
    tool = TOOLS.get(call["function"]["name"])
    if tool is None:
        return json.dumps({"error": f"Unknown tool {call['function']['name']}"})
    try:
        args = json.loads(call["function"]["arguments"] or "{}")
        jsonschema.validate(args, tool["schema"])
    except (json.JSONDecodeError, jsonschema.ValidationError) as err:
        return json.dumps({"error": f"Invalid arguments: {err}"})
    return json.dumps(tool["run"](**args))


def answer(question: str) -> str:
    messages = [{"role": "user", "content": question}]
    tools = [
        {"type": "function", "function": {"name": name, "description": t["description"], "parameters": t["schema"]}}
        for name, t in TOOLS.items()
    ]
    for _ in range(MAX_STEPS):
        r = requests.post(API_URL, headers=HEADERS, timeout=120, json={
            "model": MODEL, "messages": messages, "tools": tools, "max_tokens": 16384,
        })
        r.raise_for_status()
        message = r.json()["choices"][0]["message"]
        calls = message.get("tool_calls") or []
        if not calls:
            return message.get("content") or ""
        messages.append({"role": "assistant", "content": message.get("content"), "tool_calls": calls})
        for call in calls:
            messages.append({"role": "tool", "tool_call_id": call["id"], "content": run_tool(call)})
    raise RuntimeError(f"No answer after {MAX_STEPS} steps")
```

```typescript TypeScript theme={null}
const API_URL = "https://platform.ai.gloo.com/ai/v2/direct/chat/completions";
const HEADERS = {
  Authorization: `Bearer ${process.env.GLOO_API_KEY}`,
  "Content-Type": "application/json",
};
const MODEL = "gloo-openai-gpt-6-luna";
const MAX_STEPS = 8;

type ToolCall = { id: string; function: { name: string; arguments: string } };

const TOOLS: Record<string, { description: string; parameters: object; run: (args: any) => unknown; check: (args: any) => string | null }> = {
  get_service_times: {
    description: "Get this church's service times for a weekend day.",
    parameters: {
      type: "object",
      properties: { day: { type: "string", enum: ["saturday", "sunday"] } },
      required: ["day"],
      additionalProperties: false,
    },
    check: (args) => {
      if (typeof args !== "object" || args === null || Array.isArray(args)) return "arguments must be an object";
      const extra = Object.keys(args).filter((key) => key !== "day");
      if (extra.length > 0) return `unexpected properties: ${extra.join(", ")}`;
      return ["saturday", "sunday"].includes(args.day) ? null : "day must be saturday or sunday";
    },
    run: (args) => ({ day: args.day, times: ["9:00", "11:00"] }),
  },
};

/** Run one tool call and return its result as a string, including errors the model can fix. */
function runTool(call: ToolCall): string {
  const tool = Object.hasOwn(TOOLS, call.function.name) ? TOOLS[call.function.name] : undefined;
  if (!tool) return JSON.stringify({ error: `Unknown tool ${call.function.name}` });
  let args: unknown;
  try {
    args = JSON.parse(call.function.arguments || "{}");
  } catch {
    return JSON.stringify({ error: "Arguments were not valid JSON" });
  }
  const problem = tool.check(args);
  if (problem) return JSON.stringify({ error: `Invalid arguments: ${problem}` });
  return JSON.stringify(tool.run(args));
}

async function answer(question: string): Promise<string> {
  const messages: any[] = [{ role: "user", content: question }];
  const tools = Object.entries(TOOLS).map(([name, t]) => ({
    type: "function",
    function: { name, description: t.description, parameters: t.parameters },
  }));
  for (let step = 0; step < MAX_STEPS; step++) {
    const res = await fetch(API_URL, {
      method: "POST",
      headers: HEADERS,
      body: JSON.stringify({ model: MODEL, messages, tools, max_tokens: 16384 }),
    });
    if (!res.ok) throw new Error(`Request failed: ${res.status} ${await res.text()}`);
    const message = (await res.json()).choices[0].message;
    const calls: ToolCall[] = message.tool_calls ?? [];
    if (calls.length === 0) return message.content ?? "";
    messages.push({ role: "assistant", content: message.content ?? null, tool_calls: calls });
    for (const call of calls) {
      messages.push({ role: "tool", tool_call_id: call.id, content: runTool(call) });
    }
  }
  throw new Error(`No answer after ${MAX_STEPS} steps`);
}
```

The Python version uses the `jsonschema` package (`pip install jsonschema requests`) to check arguments against the same schema the model saw. In TypeScript, a schema library such as Ajv or Zod does the same job as the hand-written `check`.

## Get structured output you can rely on

When your code reads the answer, ask for a shape and then check it. There are two ways to ask:

1. **A JSON schema in the request.** On the Responses endpoints, send `text.format` with `"type": "json_schema"`, a `name`, the `schema` and `"strict": true`; on Chat Completions, send the same in `response_format`. See [Two Ways to Send a Schema on Gloo](https://docs.gloo.com/ai-learning-center/gloo-ai-104/structured-output.md#two-ways-to-send-a-schema-on-gloo).
2. **A tool the model must call.** Define one function whose `parameters` is your schema and force it with `tool_choice`. The arguments are your JSON. This works on both APIs and on the widest range of models; see [Get JSON with a forced tool call](https://docs.gloo.com/best-practices/guarded-endpoints.md#get-json-with-a-forced-tool-call).

Whichever you use:

* **Test it on the model you pin.** Support varies by model and provider. Some models don't apply a schema sent in `response_format`, and return text without an error. Some can't be forced to call a tool, including the Claude 5.5 models: on most of these the request still runs as if `tool_choice` were `"auto"`, so the model may answer in text instead, and a few refuse a forced tool choice with a `400`.
* **Check you got what you asked for.** Confirm the tool call arrived, or that the text parses, before you use it. If it didn't, retry once with a clearer instruction, then fail gracefully.
* **Validate the content, not just the shape.** A schema makes the right shape likely; it doesn't make the values right. Check ranges, IDs and dates against what you know, and allow `null` for "not found" so the model doesn't have to guess.
* **Check for cut-offs first.** An answer that hit the output limit is truncated JSON. Check `finish_reason` or `status` before parsing; see [Reasoning Models in Production](https://docs.gloo.com/best-practices/reasoning-models.md#detect-and-fix-a-cut-off-answer).
* **Mention JSON when you ask for `json_object`.** Some models refuse `{"type": "json_object"}` with a `400` unless your messages mention JSON. A plain "Reply in JSON" in the instructions avoids it.

## Treat tool results and connected servers as untrusted

A tool result is text the model reads, and it can contain instructions: a web page, an email or a document fetched by a tool can say "ignore your instructions and...". This is [prompt injection](https://docs.gloo.com/ai-learning-center/gloo-ai-101/challenges-and-risks.md#prompt-injection), and it applies whether the tool is your own function or a server connected through [MCP](https://docs.gloo.com/ai-learning-center/gloo-ai-104/tools-and-mcp.md#what-mcp-is).

* **Give tools the least access they need.** A tool that reads one user's calendar shouldn't hold a key that reads everyone's.
* **Decide permissions in your code, not in the prompt.** Check that the signed-in user may do what a tool call asks, every time.
* **Keep a person in the loop for writes** that are hard to undo.
* **Connect only MCP servers you trust,** and review what their tools can do before you let an agent use them.

[Direct Endpoints and Untrusted Content](https://docs.gloo.com/best-practices/untrusted-content.md) covers instruction layout and output checks in depth.

## Summary

* Check `supports_tools`, and pin the model for anything that calls tools.
* Write specific tool definitions and keep them stable.
* Parse and validate every argument; return errors to the model as tool results.
* Cap the loop, confirm and deduplicate writes, and never auto-retry after a side effect.
* Ask for a schema or force a tool, then check what came back, including whether it was cut off.
* Treat tool results and MCP servers as untrusted input.

## Next steps

> [card] [Tool Use guide](https://docs.gloo.com/api-guides/tool-use.md)
>
> Tool definitions and results on Responses and Completions.

> [card] [Tool Use tutorial](https://docs.gloo.com/tutorials/completions-tool-use.md)
>
> Build a working tool loop step by step.

> [card] [Structured Output](https://docs.gloo.com/ai-learning-center/gloo-ai-104/structured-output.md)
>
> The lesson: schemas, forced tools and why to validate.

> [card] [Direct Endpoints and Untrusted Content](https://docs.gloo.com/best-practices/untrusted-content.md)
>
> Prompt injection, least privilege and output checks.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.