> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gloo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Lesson 1: Streaming Responses

> Show the answer as the model writes it: how streaming works on Gloo, how to read a stream with the OpenAI SDK, and how to handle a stream that fails.

## Don't Make People Wait

A model writes its answer a few words at a time. Without streaming, your app waits until the whole answer is finished and then shows it in one go. For a long answer, that can mean several seconds of a blank screen.

**Streaming** sends each piece of the answer to your app as soon as the model writes it. The first words appear almost at once, and the rest fill in as the model goes. It's the reason chat apps feel quick, and it's the first thing most real AI apps need.

***

## Core Concepts

### How a Stream Arrives

When you set `"stream": true`, the API keeps the connection open and sends a series of small messages called **Server-Sent Events** (SSE). Each event carries one piece of the answer, called a **delta**. Your app adds each delta to what it already has.

On the Responses endpoints, each event has a type. The ones you'll use most:

| Event | What it means |
| :- | :- |
| `response.output_text.delta` | A new piece of answer text |
| `response.completed` | The response is finished, with `usage` |
| `response.incomplete` | The answer stopped at `max_output_tokens` |
| `response.failed` | Something went wrong after the stream started (Direct Responses) |
| `error` (or another `error.` event) | Something went wrong after the stream started (guarded Responses) |

On the Chat Completions endpoints, each chunk carries its text in `choices[0].delta.content`, and the stream ends with a `finish_reason` such as `"stop"`.

### When to Stream

**Stream when** a person is waiting to read the answer: chat windows, writing assistants, anything interactive.

**Don't bother when** your code needs the whole answer before it can do anything, such as a background job or a step that parses JSON (Lesson 3). A normal request is simpler there.

### Errors Can Arrive Mid-Stream

A stream can fail at two points:

* **Before any text arrives.** On the direct endpoints you get a normal HTTP error, such as a `503`, and the OpenAI SDK raises it as an exception.
* **After text has started.** The HTTP status is already `200`, so the error arrives in the stream: a `response.failed` event on Direct Responses, or an error chunk on Chat Completions. The OpenAI SDK raises an error chunk as an exception from the loop, so catch it.

Gloo retries some of these for you. If a stream fails before the answer starts, Gloo can retry it on another provider, so the stream just starts a little later. For models that support it, a stream cut off mid-answer is continued by another provider in the same response. Lesson 9 covers what your app should still handle.

***

## Get Started

This example streams a short answer from Direct Responses and prints it as it arrives.

```bash theme={null}
pip install openai
```

```python Python theme={null}
import os
import openai
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GLOO_API_KEY"],
    base_url="https://platform.ai.gloo.com/ai/v2/direct",
)

stream = client.responses.create(
    model="gloo-anthropic-claude-sonnet-5.5",
    input="Write a short, encouraging note for a volunteer's first day.",
    stream=True,
)

try:
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)
        elif event.type == "response.incomplete":
            print("\n[Stopped at the output limit]")
        elif event.type == "response.failed":
            print("\n[The answer was interrupted]")
except openai.APIError:
    # The connection dropped after text had started. Keep what was printed.
    print("\n[The answer was interrupted]")
print()
```

```bash cURL theme={null}
curl -N -X POST 'https://platform.ai.gloo.com/ai/v2/direct/responses' \
  -H "Authorization: Bearer $GLOO_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gloo-anthropic-claude-sonnet-5.5",
    "input": "Write a short, encouraging note for a volunteer'\''s first day.",
    "stream": true
  }'
```

With cURL, `-N` turns off buffering so you see each event as it arrives. The raw stream looks like this (trimmed):

```text theme={null}
event: response.output_text.delta
data: {"type":"response.output_text.delta","output_index":0,"delta":"Welcome "}

event: response.output_text.delta
data: {"type":"response.output_text.delta","output_index":0,"delta":"to the team! "}

event: response.completed
data: {"type":"response.completed",...}
```

### The Same Thing on Chat Completions

If your app uses the Chat Completions shape, read `delta.content` from each chunk. Add `stream_options` to get token usage in a final chunk:

```python theme={null}
import openai

stream = client.chat.completions.create(
    model="gloo-anthropic-claude-sonnet-5.5",
    messages=[{"role": "user", "content": "Write a short, encouraging note for a volunteer's first day."}],
    stream=True,
    stream_options={"include_usage": True},
)

try:
    for chunk in stream:
        if not chunk.choices:
            print("\nUsage:", chunk.usage)  # the final usage chunk
            continue
        if chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)
except openai.APIError:
    # An error chunk after text has started. Keep what was printed.
    print("\n[The answer was interrupted]")
```

***

## Good to Know

* **Keep your API key on the server.** Don't call the API from browser code, where anyone can read the key. Stream from your server and pass the text on to the browser. The [streaming tutorial](https://docs.gloo.com/tutorials/completions-streaming.md) builds a small proxy that does this. See [API Key Security](https://docs.gloo.com/best-practices/api-key-security.md).
* **Check the status before reading.** A request that fails before the stream starts returns a JSON error, not events, so check for it first so a `401` shows a clear message instead of a parsing error.
* **Some chunks have no text.** The first and last chunks often carry no content. Skip them quietly.
* **Keep what the user has seen.** If a stream fails partway, leave the partial text on screen and mark it as interrupted, rather than letting it vanish. [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md#partial-output-ux) covers this in production.
* **Reasoning can stream too.** Reasoning models can send their reasoning before the answer, in separate events. [Reasoning](https://docs.gloo.com/api-guides/reasoning.md#what-comes-back) shows how to read them.

***

## Key Takeaway

Streaming shows the answer as the model writes it, so your app feels fast. Set `"stream": true`, add each text delta as it arrives, and be ready for a stream to end early: at the output limit, or with an error after some text has already appeared.

> [!TIP]
>
> **Try it on Gloo:** the [Streaming tutorial](https://docs.gloo.com/tutorials/completions-streaming.md) builds a full streaming client and server-side proxy in several languages. The [Direct Responses streaming reference](https://docs.gloo.com/api-guides/direct-responses.md#streaming) lists every event type.

***

## Learn more

* [Direct Responses: Streaming](https://docs.gloo.com/api-guides/direct-responses.md#streaming): every event type on the Responses stream
* [Streaming AI Responses in Real Time](https://docs.gloo.com/tutorials/completions-streaming.md): a step-by-step tutorial with a server-side proxy
* [Handling Streaming Failures](https://docs.gloo.com/best-practices/completions-streaming-failures.md): retry budgets, continuing an interrupted answer and partial-output UX
* [Streaming on the direct endpoints](https://docs.gloo.com/api-reference/general/errors.md#streaming-on-the-direct-endpoints): how errors arrive before and after output starts

***

## What's next

Next, you'll keep a conversation going across many turns, and learn what to do when it gets too long.

> [card] [Lesson 2: Conversations That Remember](https://docs.gloo.com/ai-learning-center/gloo-ai-104/conversations-that-remember.md)
>
> Keep history, stay inside the context window and trim old turns.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.