Why Your AI Assistant Doesn’t Know About Yesterday
Here’s a frustrating scenario you might have experienced: You ask an AI assistant about a news event from last week, and it confidently tells you it doesn’t have information about events after its training cutoff date. Or worse, it makes something up that sounds plausible but is completely wrong. You’ve just hit one of AI’s basic limits. Understanding it is the first step toward solving it. RAG (Retrieval Augmented Generation) means the AI retrieves relevant information first, augments (adds to) its prompt with what it found, and then generates an answer. By the end of this course, you’ll know how to make AI much more useful by connecting it to accurate, up-to-date information.Core Concepts
The Knowledge Cutoff Problem
Every AI model was trained on data up to a certain date, called its knowledge cutoff. The model knows nothing about what happened after that date. It’s like a brilliant colleague who has been away for a long time: they know a lot about everything up to the day they left, and nothing about what happened since. This creates real problems:- Ask about recent events, and the AI is clueless
- Ask about your company’s latest product, and it has no idea
- Ask about a paper published last month, and it can’t help
When AI Confidently Makes Things Up
Here’s something trickier than “I don’t know”: sometimes AI fills gaps with fiction that sounds confident. AI predicts the most likely next words based on patterns (as earlier lessons covered). When it lacks good information, it doesn’t stop and say “I’m uncertain.” It generates text that sounds like a reasonable answer based on similar patterns. This is called hallucination, and it’s one of the biggest challenges with AI assistants. The AI might:- Invent statistics that don’t exist
- Create fake citations to papers that were never written
- Describe events that never happened
- Attribute quotes to people who never said them
The Difference Between Knowing Everything and Knowing the Right Things
Here’s a surprising point: we don’t need AI to know everything. We need it to know the right things for our specific situation. Imagine you’re a customer support agent for a software company. You don’t need an AI that knows every fact in the world. You need an AI that knows:- Your product’s features and limitations
- Your company’s policies and procedures
- Your documentation and help articles
- Common customer questions and their answers
What If AI Could Look Things Up?
Instead of relying solely on what it learned during training, what if the AI could:- Receive your question
- Search through relevant documents to find information that might help
- Read that information
- Generate a response based on what it just found
- Closed-book exam: Answer from memory only (traditional AI)
- Open-book exam: Look up information before answering (RAG)
Try It Yourself
Exercise 1: Spot the Limitation
Open your favorite AI assistant and try these prompts:- “What were the top news stories from last week?”
- “What’s the current price of [any stock]?”
- “What’s the weather like right now in New York?”
Exercise 2: Domain-Specific Questions
Now try asking about something very specific to a niche domain:- “What are the specific return policies for [a local store you know]?”
- “What’s the process for submitting expense reports at [your company]?”
- “What are the side effects of [a specific medication] according to the latest research?”
Exercise 3: Hallucination Hunting
Ask the AI to provide specific citations or statistics on a topic you know well. For example:“Give me three academic papers about [topic you’re expert in] with authors and publication years.”Then verify: Do those papers actually exist? You might be surprised how often the AI invents plausible-sounding but completely fictional sources.
Common Pitfalls
Pitfall 1: Assuming AI “Just Knows”
Many people treat AI like an all-knowing oracle and assume its answers are accurate. But AI’s knowledge is limited to its training data, and even within that data, it can make mistakes. The fix: Always consider whether your question requires information the AI might not have (recent events, specific organizations, specialized domains).Pitfall 2: Not Recognizing Hallucinations
Hallucinated content often sounds perfectly reasonable. It uses proper grammar, follows logical structure, and presents information confidently. The only way to catch hallucinations is to verify important claims. The fix: For any information that matters, verify it against authoritative sources. Don’t assume confident-sounding text is accurate. In an app, Building Trustworthy Grounded Applications shows checks you can run in code.Pitfall 3: Thinking RAG Is Just “Better Search”
Some people hear about RAG and think it’s just a fancy search engine. But RAG does more than find information. It combines what it finds into a clear answer to your question. The fix: Think of RAG as search plus writing: search finds the information, and the AI turns it into an answer.Pitfall 4: Expecting RAG to Fix Everything
RAG helps with many questions, but not all of them. If your knowledge base has bad information, RAG will use that bad information. If your question requires reasoning beyond what’s in the documents, RAG alone won’t solve it. The fix: Think of RAG as a tool that makes AI more informed, not omniscient. The quality of outputs depends heavily on the quality of inputs.Level Up
Here’s a challenge to test your understanding: Scenario: You’re building a customer support chatbot for a software company. The chatbot needs to answer questions about the product, troubleshoot common issues, and explain company policies. Your task: List three specific types of questions where a standard AI would struggle but a RAG-enabled AI would excel. For each, explain:- Why standard AI would fail
- What information RAG would need to retrieve
- How the answer would be better with RAG
Key Takeaway
AI models don’t know about recent events, lack domain-specific knowledge, and sometimes make things up. RAG addresses these limits by letting AI look up relevant information before it answers: an open-book exam instead of a closed-book one.What’s next
You now understand why RAG exists and what problems it solves, but how does the AI actually “look things up” and use what it finds? To see the fix on Gloo, read the Grounded Responses guide, try the RAG chatbot tutorial, and see the failure modes to design against.Lesson 2: How RAG Actually Works
Break the RAG process into three clear steps (Retrieve, Augment, and Generate), and see how to use RAG effectively and where things can go wrong.

