Prompt
What it means: A prompt is the input or question you give to an AI model. It tells the model what to generate or how to respond. Examples:- “Summarize this paragraph.”
- “Write a prayer for a new mother.”
- “What are 5 ways a church could use data in outreach?”
System Prompt
What it means: A system prompt is an instruction that sets the model’s overall behavior, tone, or rules before it interacts with a user. Use case: A system prompt might say: “You are a friendly assistant who always answers concisely and avoids strong opinions.” This tells the model how to behave for the entire session.The people using an app often don’t see its system prompt, but it shapes every response.
system message on Completions, or the instructions field on Responses. Use it to set your app’s voice, rules, and boundaries. Put your task in the system message shows how.
Go deeper: Context is King covers the background worth putting in a system prompt.
Prompt Engineering
What it means: Prompt engineering is the skill of writing prompts that get better, more accurate, or more useful responses from an AI model. Why it’s useful: A vague prompt often gets a vague answer. Prompt engineering means stating clearly what you expect, so the model doesn’t have to guess. Example: Instead of saying “Explain photosynthesis,” a better prompt might be “Explain photosynthesis in simple terms for a middle school science student.” How it shows up in Gloo: Clear prompts get better results from every Gloo AI endpoint, and on grounded endpoints they help Gloo find the right passages. Go deeper: AI 102: Prompting is a full course on writing prompts that work.Inference
What it means: Inference is a trained model generating a response to your prompt. It’s what happens every time you use a model, after training is done. Analogy: Training is like studying for an exam. Inference is answering the question during the test. Why it matters: How fast and how well a model runs inference decides how useful it is in real-time tools like chat apps or search. How it shows up in Gloo: Every API request you send to a Gloo AI model is an inference request. Inference is billed per token at each model’s published rate. Guarded endpoints are priced at that rate plus 6.5% for guardrails, moderation and values alignment; see Pricing.Tokens
What it means: Tokens are the small pieces of text a model reads and writes. A token can be as short as one character or as long as a whole word. In English, a token averages about three-quarters of a word. Example: The sentence “Hello there!” is about 3 tokens: “Hello,” “there,” and “!” Why it matters: Models count everything in tokens. Each model has a limit on how many tokens it can take in at once, and longer prompts and responses use more of them. Limits vary a lot between models, so check a model’s limit before you send it a long document. How it shows up in Gloo: Gloo AI prices model usage per token, with separate rates for input and output tokens. You can cap a response’s length withmax_tokens (Completions) or max_output_tokens (Responses). On a reasoning model that cap holds the thinking as well as the answer; see Set your output limit for reasoning models. When you upload content to the Data Engine, it’s split into smaller chunks before it’s made searchable.
Go deeper: Cost and Speed explains how tokens set the price of each request.
Context Window
What it means: The context window is the most tokens a model can work with at once. It covers your prompt, any earlier conversation you send, and the model’s reply. Analogy: Think of it like a whiteboard. When it’s full, something has to be erased to make room. A bigger context window lets the model refer to more of the conversation or a longer document. How it shows up in Gloo: Each model on Gloo AI has its own context window, shown on the Studio Models page as Max Input / Context. Pick a model with a larger window when you need to send long documents or long conversations, or trim older turns to stay inside it. Go deeper: Conversations That Remember shows what to do when a conversation outgrows it.Temperature
What it means: Temperature controls how focused or varied a model’s responses are. How it works:- Lower temperature (like 0.2): More focused and predictable answers
- Higher temperature (like 0.9): More random and creative responses
temperature on Gloo AI’s Completions and Responses endpoints, or move the Temperature slider in the Studio Chat Playground. If you don’t set it, the model’s own default applies. Some models, such as Claude 5 models, don’t take a temperature; the request still runs, and the response tells you your value wasn’t applied. See Sampling parameters.
Top-k / Top-p Sampling
What it means: These settings control which words the model can choose from at each step, balancing variety and coherence. Quick comparison:- Top-k looks at the top K likely words and picks from them
- Top-p looks at the smallest number of words whose probabilities add up to P (like 90 percent), then chooses from those
top_p. Most apps leave it at the model’s default and adjust temperature instead.
Stop Sequence
What it means: A stop sequence is text that makes the model stop writing when it produces it. Use case: You might set “###” as a stop sequence so the model stops there and doesn’t keep talking after a section. How it shows up in Gloo: You can set stop sequences withstop on Direct Completions and the Responses endpoints, or stop_sequences on Gloo AI’s Anthropic Messages endpoint. Chat Completions responses also include a finish_reason: stop means the model finished on its own or hit a stop sequence.
What’s next
When you build with Gloo’s API, Using the Completions API walks through your first prompt in code, and Controlling Cost shows how tokens turn into price.How AI Uses and Stores Knowledge
What are vectors, embeddings, and retrieval systems, and how do they help AI remember or reason?

