What you’ll build
You’ll build:- SSE stream parser that reads tokens as they arrive from the API
- Token accumulator that assembles the full response with timing and token count
- Streaming-aware error handler that catches auth and rate-limit errors before reading the stream
- Terminal renderer that displays tokens in real time with a typing effect
- Server-side proxy that relays the stream to browser clients without exposing your credentials
How it works: Server-Sent Events, direct vs. proxy, and architecture
How it works: Server-Sent Events, direct vs. proxy, and architecture
"stream": true in a completions request, the API switches from a single JSON response to an SSE stream. Each token arrives as a line formatted data: <json>, with blank lines separating events:finish_reason (typically "stop").Two approaches: direct vs. proxyPrerequisites
You need:- A Gloo AI Studio account with an API key
- Your API key from the API Keys page. Keep it on your server and out of source control; see API Key Security. Authentication uses your
GLOO_API_KEYenvironment variable — set it in your.envfile and it’s used directly as a Bearer token. No token exchange needed. - Authentication setup — complete the Authentication Tutorial first
Set up the starter project
You’ll build the streaming client step by step. The starter code provides the scaffolding, with TODO markers guiding each step.Download the starter code
Python
JavaScript
TypeScript
PHP
Go
Java
Install dependencies and add your key
Test your setup
NotImplementedError (or equivalent) from the first stub. That means setup is complete.Step 1: Environment Setup & Auth Verification
Before writing any streaming logic, confirm yourGLOO_API_KEY works with the streaming endpoint.
What You’ll Verify
- Credentials load correctly from
.env - Your API key is set and valid
- A request to the completions endpoint returns
200 OKwithContent-Type: text/event-stream
Testing Your Setup
Run the Step 1 checkpoint:Expected output
Expected output
.envfile exists in the language directory (not just.env.example)GLOO_API_KEYis set correctly.- You’ve completed the Authentication Tutorial prerequisites
Step 2: Streaming-Aware Error Handling
Implement the stream error handler, which maps HTTP status codes to descriptive exceptions before any stream data is read.Key Concepts
Two-Phase Error Handling
Streaming errors arrive in two phases. Pre-stream errors are HTTP status codes you can check before reading any bytes, which is what this step’s handler does. Mid-stream errors happen while you read (a network drop, server restart or timeout), so Step 4 catches them around the read loop and keeps any partial text.Implementation Guide
In your streaming client file, find the error handler method (one case per status code), review the TODO comments, then implement it:Expected output
Expected output
- Status 200 must not raise an exception
- The error message for non-200 includes the status code
- The response body is truncated (first 200 chars) to avoid enormous error messages
Step 3: Streaming Requests & SSE Parsing
Next, open a persistent HTTP connection to the completions API and write the parser that turns raw SSE lines into usable data.What You’ll Implement
- A function to initiate a streaming request
- A function to parse individual SSE lines
Making the Streaming Request
Why stream: true Changes Everything
Without stream: true, the API buffers the entire response and returns it as a single JSON object. With stream: true, it switches to SSE mode: the connection stays open and bytes arrive incrementally as the model generates them.
That’s why you return the raw response object rather than parsed JSON: the body isn’t fully available yet. The caller reads it line by line in the next steps.
Fail Fast Before Reading
Check the HTTP status code before reading the stream. A 401 response never produces SSE data; it returns a JSON error body. Parsing it as SSE gives confusing parse errors instead of a clear “authentication failed” message.Implementation Guide
In the same file, find the streaming request method, review the TODO comments, then implement it:Authorization and Content-Type headers, and the payload sets stream: true to enable SSE mode and model: "gloo-anthropic-claude-opus-5.5" to pin the model.
Parsing SSE Lines
The SSE Wire Format
SSE is a simple text protocol. Each event is one line starting withdata: , terminated by a blank line. In practice, the Gloo AI stream looks like:
data: , such as event: or : comment lines.
Defensive Parsing
Wrap the JSON parse in a try/catch. Mid-stream network hiccups can produce partial lines, and one malformed chunk shouldn’t crash the stream. Returnnull for unparseable lines and let the accumulation loop move on.
Implementation Guide
In the same file, find the SSE line parser method, review the TODO comments, then implement it:null for blank and non-data: lines, strips the data: prefix, and detects the [DONE] sentinel before parsing JSON, returning it as a string to signal the end of the stream.
Expected output
Expected output
- The streaming request function sets
streamtotruein the payload - The SSE line parser strips exactly 6 characters (
"data: "has a space after the colon) - The
[DONE]check happens before the JSON parse
Step 4: Token Extraction & Accumulation
Next, pull the token out of each parsed SSE chunk and write the loop that assembles the complete result.What You’ll Implement
- A function to extract token content from a parsed SSE chunk
- A function to collect the full stream into a result object
Extracting Token Content
Why Content Can Be Absent
Not every SSE chunk carries text. The first chunk establishes the role (delta: {"role": "assistant"}), while the final chunk carries the finish reason with an empty or absent delta. Only chunks in the middle carry actual content.
An absent content field is normal, so return an empty string rather than throwing. The accumulation loop skips empty strings when counting tokens.
Null-Safe Navigation
Different languages handle missing keys differently. In Python,.get() returns None without raising; in JavaScript/TypeScript, optional chaining (?.) does the same. In Go and Java the struct is fully typed, so missing content simply maps to the zero value. The goal in all languages is the same: never throw when a field is absent.
Implementation Guide
In the same file, find the token content extractor, review the TODO comments, then implement it:delta.content from the first choice and returns an empty string if choices is empty, the field is absent or null, or the chunk has an unexpected shape.
Accumulating the Full Response
Two Ways to Consume a Stream
You can either accumulate all tokens into a string (what the function in this step does) or print each token immediately as it arrives (what the function in Step 5 does). The choice depends on whether you need the full text before taking action:- Accumulate: useful when you need to parse the full response, log it, or return it from an API
- Print immediately: useful for CLI tools and browser UIs where you want the typing effect
The Line Buffer (JS/TS/PHP)
In Python and Go, the HTTP libraries provide line-at-a-time iteration. In JavaScript, TypeScript, and PHP, you read raw bytes and split on\n yourself. This requires a line buffer: keep any incomplete final chunk in a variable and prepend it to the next read’s output. Without it, tokens near chunk boundaries get split across two parse calls.
Implementation Guide
In the same file, find the accumulation loop method. It brings together everything from the previous steps, with TODO comments marking each stage. Trace through the structure before implementing it.null results, stops on a non-null finish_reason or a [DONE] sentinel and returns the text, token count, elapsed milliseconds and finish reason.
Expected output
Expected output
- The token content extractor returns
""(notNone/null) when content is absent - The accumulation loop reads
finish_reasonfromchoices[0], not from the top-level chunk - The line buffer (
buffer = lines.pop()) is in place for JS/TS/PHP
Step 5: Typing-Effect Terminal Renderer
Implement the terminal renderer, which prints each token to stdout as it arrives, without a newline, for a live typing effect. It reuses the streaming request, SSE parsing and token extraction functions but skips the accumulation loop.Key Concepts
Unbuffered Output
Most languages buffer stdout, holding output until the buffer fills or the program exits. For a typing effect, force each token out immediately:Direct Stream Consumption vs. Accumulation
Step 4’s function returns once the stream is complete; the renderer prints as it goes, so the user sees output before the model finishes. Pick based on whether the output must be complete before it’s useful.Implementation Guide
Open the renderer file referenced in the code block. It has a single method to implement. Review the TODO comments, then implement it:Expected output
Expected output
- Each token is written with no trailing newline
flush()or equivalent is called after each write- The summary line format is
[N tokens, finish_reason=X]
Step 6: Server-Side Proxy
Implement the proxy server’s stream handler: the route that receives browser requests, forwards them to Gloo AI with the server-side API key, and pipes the SSE response back.Key Concepts
Why a Proxy?
Browser JavaScript can’t safely hold an API key: anyone can see client code in DevTools. With a proxy, the browser POSTs to your server, your server adds the API key and calls Gloo AI, and the SSE stream flows back through your server. The proxy can also add rate limiting, logging and multi-tenant auth without touching client code.SSE Headers That Matter
Three headers tell the browser (and any reverse proxies like nginx) that this is a live stream, not a buffered response:Language-Specific Flushing
Each language needs an explicit flush to push bytes to the client immediately:Implementation Guide
Open the proxy server file referenced in the code block; server setup and routing are already in place. Find the stream handler (or route handler) method, review the TODO comments, and implement the relay logic:OPTIONS preflight requests so browsers can POST cross-origin, injects stream: true into the forwarded body, and relays and flushes each non-blank SSE line. If the upstream request fails, it writes a structured error SSE frame instead of closing silently.
Expected output
Expected output
- CORS headers are set before sending the response headers (Java)
X-Accel-Buffering: nois present (required to disable nginx buffering)- Go: the flusher interface assertion must succeed — this panics if the
ResponseWriterdoesn’t support flushing - PHP: clear any existing output buffers before setting SSE headers
Step 7: Testing Your Complete Implementation
With all six steps implemented, run the demo, test the proxy via API, and try the browser demo.Run the Demo Script
The entry point runs both examples back-to-back: first it accumulates a full response and prints it, then it streams a second response to the terminal with a typing effect.Test the Proxy Server via API
Start the proxy server in one terminal:finish_reason.
Browser Demo
The browser demo is a standalone HTML file separate from the language starter projects — no install step required.frontend-example/
frontend-example/ directory with any of these:
http://localhost:3000 in your browser, type a question, and click Send — tokens appear one by one as they arrive from the proxy.

How the Browser Connects to the Stream
The browser’s built-inEventSource API only supports GET, and the completions API needs a POST body, so the demo page uses fetch() with a ReadableStream, which supports any HTTP method:
Markdown Rendering
AI responses often contain markdown. Inserting raw tokens into the DOM breaks mid-stream (**bo appears before ld** closes the bold span). Instead, accumulate tokens and re-parse the full buffer on each token:
marked.parse() runs on every token — slightly redundant but always produces valid HTML. DOMPurify.sanitize() prevents XSS from any HTML in the AI response.
Complete code
A working reference is in the tutorial repository:Completed Project
Troubleshooting
These fixes cover the tutorial code. For stream failures in general, see the decision tree in Handling Streaming Failures.Stream hangs and never produces output
Stream hangs and never produces output
"stream": true is in the request payload. Without it, the API returns a single buffered JSON response, so the connection may appear to hang.Garbled or split tokens
Garbled or split tokens
\n before parsing. Make sure buffer = lines.pop() saves the incomplete last fragment.Authentication failed (401)
Authentication failed (401)
.env file is missing GLOO_API_KEY, or the values are incorrect. Run the Step 1 checkpoint to verify credentials load correctly.Browser blocks direct API calls (CORS error)
Browser blocks direct API calls (CORS error)
platform.ai.gloo.com will be blocked. Use the proxy server (Step 6) so API calls happen server-side.Failed to fetch when serving the browser demo on a port other than 3000
Failed to fetch when serving the browser demo on a port other than 3000
http://localhost:3000 by default. If your file server uses a different port (e.g. VS Code / Cursor Live Server on port 5500, or python -m http.server 8080), the browser’s Origin header won’t match and the proxy blocks the request. Fix: set PROXY_CORS_ORIGIN in your .env to the exact origin shown in your browser’s address bar, then restart the proxy.http://localhost:5500 and http://127.0.0.1:5500 are treated as different origins by the browser even though they resolve to the same address. Copy the origin directly from the address bar to avoid a mismatch.PHP output appears all at once
PHP output appears all at once
ob_end_flush() (or while (ob_get_level() > 0) ob_end_flush()) before the SSE loop to disable buffering.Go panics on w.(http.Flusher)
Go panics on w.(http.Flusher)
http.ResponseWriter doesn’t implement http.Flusher. This shouldn’t happen with the standard net/http server, but will happen with some test wrappers. Make sure you’re using http.ResponseWriter directly.Mid-stream disconnect loses all output
Mid-stream disconnect loses all output
fullText already has content when the error occurs, return it rather than re-raising — partial responses are usually more useful than nothing.Stream ends with finish_reason: "error", or you're unsure whether to retry
Stream ends with finish_reason: "error", or you're unsure whether to retry
Broken markdown mid-stream
Broken markdown mid-stream
innerHTML. Accumulate the full buffer and call marked.parse(buffer) on every token — this ensures the markdown is always valid HTML at each step.Next steps
New to function calling? See Tools and MCP in the AI Learning Center.Grounded Completions
Tool Use
Completions API reference
tradition, model_family, and model.
