Skip to main content
What Are the Real Risks with AI? AI is powerful, but it also brings risks to understand and manage. AI systems learn patterns from data, not meaning or truth, so they can behave in unexpected or harmful ways if nobody plans for it. Understanding these risks helps us build AI that supports people rather than confusing or misleading them. This section introduces the most common risks in modern AI and why they matter in real-world applications. Each risk below ends with how it shows up in Gloo. Many of those notes mention Gloo’s guarded and grounded endpoints. Direct endpoints send your request straight to the model you choose, and your app takes care of these protections itself. Guarded endpoints have Gloo’s safety and values alignment built in, and grounded endpoints answer from your own content. Guardrails and Values walks through that choice, and Endpoint Types compares the endpoints.

Hallucinations

What it means: A hallucination is when an AI generates incorrect, fabricated, or misleading information while sounding confident. Why it matters: Hallucinations can spread false information and cause users to rely on inaccurate outputs, especially in contexts involving education, leadership, counseling, or faith. How it shows up in Gloo: Gloo AI’s grounded endpoints reduce hallucinations by giving the model passages from your own content to answer from, instead of relying on what it remembers from training. They also tell you whether an answer was grounded. No AI system can rule out hallucinations completely, so for important answers, check them against the sources. Go deeper: How AI Thinks explains why they happen, and The Problem RAG Solves shows how answering from your own content reduces them.

Bias

What it means: Bias is when an AI reflects unfair patterns, stereotypes, or imbalanced representation found in its training data. Why it matters: If not monitored, AI systems can unintentionally reinforce harmful assumptions or favor certain groups over others. How it shows up in Gloo: Grounding answers in your own curated content reduces how much a response depends on broad internet training data. Guarded endpoints also give you guardrails and values-aligned (tradition) responses.

Toxicity

What it means: Toxicity means outputs that are rude, offensive, harmful, or inappropriate. Why it matters: Toxic or harmful content can make AI unsafe in community environments and may violate organizational or platform guidelines. How it shows up in Gloo: Gloo AI’s guarded endpoints moderate model output to help keep responses appropriate. Grounding responses in your own approved content also helps keep language and tone on track.

Model Collapse

What it means: Model collapse happens when models begin training on data generated by other models instead of human-created data, causing quality to degrade over time. Why it matters: Over-reliance on AI-generated data reduces originality, accuracy, and reliability in future models. How it shows up in Gloo: Gloo AI’s grounded endpoints use your content at request time, as context for the model. The model itself isn’t retrained on it. See Trust.

Data Poisoning

What it means: Data poisoning is when harmful or misleading data is intentionally inserted into training sets to manipulate how the AI behaves. Why it matters: If poisoned data influences training, it can cause models to behave incorrectly, unpredictably, or in ways that are vulnerable to exploitation. How it shows up in Gloo: Because grounded endpoints use your content as context rather than training data, what you upload doesn’t change the model. The quality of grounded answers still depends on your content, so keep your knowledge base curated and up to date.

Prompt Injection

What it means: Prompt injection is when someone writes input designed to make the model ignore its instructions or behave in unintended ways. The input can come from a user, or from a document or web page the model reads. Why it matters: Prompt injection can lead to leaks of internal instructions, bypassing of guardrails, or harmful model actions. How it shows up in Gloo: Gloo AI’s guarded endpoints run input guardrails on each request and moderate each response. In your own app, treat text the model reads as untrusted, and limit what tools the model can use without a person’s approval. Building with Guarded Endpoints shows how to pass outside text as data rather than instructions. Go deeper: Coding Agents explains why outside content should be treated as data, not instructions.

Misalignment

What it means: Misalignment is a gap between what the AI is designed to do and what it actually does in real interactions. Why it matters: If a model’s behavior does not match organizational values or user intent, it can harm trust and produce outcomes that conflict with the mission or goals. How it shows up in Gloo: On Gloo AI, you can steer models toward your values in a few ways: the tradition setting on guarded endpoints, a system prompt you write, and grounding in content you’ve chosen. Go deeper: Guardrails and Values explains values alignment and how Gloo measures it.
Try it on Gloo: choose a guarded endpoint when you want Gloo’s safety and values alignment built in, with guardrails and output moderation around each request. Choose a direct endpoint when you want the provider’s model as-is and handle safety yourself. Grounded Completions answer from your own content, which helps reduce hallucinations.

What’s next

When you build an app, Endpoint Types compares guarded and direct endpoints, and Direct Endpoints and Untrusted Content covers the safeguards your app owns.

What Types of AI Models Exist, and Why Do They Work Differently?

What distinguishes foundation models, frontier models, open-weight models, multilingual models, and instruction-tuned models, and when do you use each one?