Benchmarks (e.g., MMLU, HellaSwag, TruthfulQA)
What it means: Benchmarks are standardized sets of test questions used to measure and compare how well models handle specific types of tasks or reasoning. Examples:- MMLU: Evaluates knowledge and reasoning across 57 academic and professional subjects.
- HellaSwag: Tests the model’s ability to apply common sense and continue sentences in plausible ways.
- TruthfulQA: Measures whether a model produces accurate and truthful responses.
Latency
What it means: Latency is how long an AI model takes to start responding after you submit a prompt. Why it matters: Low latency feels fast. High latency can make even a smart model frustrating in real-time apps. How it shows up in Gloo: Many models on Gloo AI can be served by more than one provider. If the provider serving a model fails or times out, Gloo sends the request to another provider of the same model, so your app keeps responding. Streaming a response also helps it feel fast, because people see the first words right away. Go deeper: Streaming Responses shows how to show the first words sooner.Throughput
What it means: Throughput is how many requests a system can handle in a given time. Why it matters: Higher throughput means the system can serve more users at once—critical for apps that rely on AI for live chat, search, or customer service. How it shows up in Gloo: If your app serves many people at once, through large-scale search or chat, Gloo AI’s published rate limits tell you how many requests you can send.Tokens per Second (TPS)
What it means: TPS tells you how fast a model writes. Models read and write in tokens, which are small pieces of text, often a word or part of a word. Example: If a model runs at 50 TPS, it can generate a 250-token reply (about 2–3 paragraphs) in 5 seconds. Why it matters: Higher TPS means less waiting, especially for long or complex responses. How it shows up in Gloo: The Studio Models page shows a Speed column for each model, so you can compare how fast models are before you choose one.BLEU / ROUGE / F1 Score
What it means: These automated scores measure how closely a model’s outputs match expected answers.- BLEU: Used mostly in translation tasks.
- ROUGE: Commonly used to evaluate summarization.
- F1 Score: Balances precision (how much of the output is correct) with recall (how much of the correct output was included).
Human Eval
What it means: Human evaluation is people judging model outputs on qualities like helpfulness, clarity, and tone. Why it matters: People catch things automated scores miss, like whether an answer is kind, well organized, or appropriate. Human evaluation matters most for AI used in education, counseling, or creative writing. How it shows up in Gloo: Gloo’s FAI framework scales up this kind of judgment: a judge model applies defined expert criteria to score each response. For your own app, have people review a sample of real answers before you launch, and keep checking after. Testing, Evaluation and Observability shows how to build a test set and score answers you can compare. Go deeper: When RAG Works (and When It Doesn’t) covers how to test a grounded app with real questions.What’s next
To measure your own app, see Testing, Evaluation and Observability and Benchmarking for how Gloo evaluates models.Review and What's Next
What have you learned and what can I do next with Gloo and AI?

