Skip to main content
How Do We Know if an AI Model Is Good? AI models aren’t all equal. Some are fast but less accurate; others are accurate but slow or expensive. To decide which models to use, and how to improve them, developers and researchers use a range of tests, scores, and real-world measures. In this section, we explain the most common ways to evaluate AI: standard benchmarks, speed measures, and human feedback. Together, they help teams check that a model is useful and safe.

Benchmarks (e.g., MMLU, HellaSwag, TruthfulQA)

What it means: Benchmarks are standardized sets of test questions used to measure and compare how well models handle specific types of tasks or reasoning. Examples:
  • MMLU: Evaluates knowledge and reasoning across 57 academic and professional subjects.
  • HellaSwag: Tests the model’s ability to apply common sense and continue sentences in plausible ways.
  • TruthfulQA: Measures whether a model produces accurate and truthful responses.
Why it matters: Benchmarks let researchers compare different models in a consistent way and track improvements over time. How it shows up in Gloo: Alongside standard benchmarks, Gloo evaluates models with the Flourishing AI Initiative (FAI) framework. It measures how well a model’s responses support seven dimensions of human flourishing: character, happiness, relationships, meaning, health, finances, and faith. Go deeper: Benchmarking describes the FAI framework and how Gloo scores models with it.

Latency

What it means: Latency is how long an AI model takes to start responding after you submit a prompt. Why it matters: Low latency feels fast. High latency can make even a smart model frustrating in real-time apps. How it shows up in Gloo: Many models on Gloo AI can be served by more than one provider. If the provider serving a model fails or times out, Gloo sends the request to another provider of the same model, so your app keeps responding. Streaming a response also helps it feel fast, because people see the first words right away. Go deeper: Streaming Responses shows how to show the first words sooner.

Throughput

What it means: Throughput is how many requests a system can handle in a given time. Why it matters: Higher throughput means the system can serve more users at once—critical for apps that rely on AI for live chat, search, or customer service. How it shows up in Gloo: If your app serves many people at once, through large-scale search or chat, Gloo AI’s published rate limits tell you how many requests you can send.

Tokens per Second (TPS)

What it means: TPS tells you how fast a model writes. Models read and write in tokens, which are small pieces of text, often a word or part of a word. Example: If a model runs at 50 TPS, it can generate a 250-token reply (about 2–3 paragraphs) in 5 seconds. Why it matters: Higher TPS means less waiting, especially for long or complex responses. How it shows up in Gloo: The Studio Models page shows a Speed column for each model, so you can compare how fast models are before you choose one.

BLEU / ROUGE / F1 Score

What it means: These automated scores measure how closely a model’s outputs match expected answers.
  • BLEU: Used mostly in translation tasks.
  • ROUGE: Commonly used to evaluate summarization.
  • F1 Score: Balances precision (how much of the output is correct) with recall (how much of the correct output was included).
Why it matters: These scores give a fast, repeatable way to check accuracy, especially while a model or app is being developed. How it shows up in Gloo: Scores like these are most useful for tasks with a clear right answer, such as extracting topics, classifying documents, or creating metadata. You can use them to check that results stay consistent across a whole content library.

Human Eval

What it means: Human evaluation is people judging model outputs on qualities like helpfulness, clarity, and tone. Why it matters: People catch things automated scores miss, like whether an answer is kind, well organized, or appropriate. Human evaluation matters most for AI used in education, counseling, or creative writing. How it shows up in Gloo: Gloo’s FAI framework scales up this kind of judgment: a judge model applies defined expert criteria to score each response. For your own app, have people review a sample of real answers before you launch, and keep checking after. Testing, Evaluation and Observability shows how to build a test set and score answers you can compare. Go deeper: When RAG Works (and When It Doesn’t) covers how to test a grounded app with real questions.

What’s next

To measure your own app, see Testing, Evaluation and Observability and Benchmarking for how Gloo evaluates models.

Review and What's Next

What have you learned and what can I do next with Gloo and AI?