Skip to main content

Beyond Keyword Matching

Quick question: If you’re searching for information about “automobiles,” should a document about “cars” show up in your results? Yes. But traditional keyword search wouldn’t find it, because the word “automobiles” doesn’t appear in a document that only uses “cars.” Embeddings solve this problem. They let computers recognize that “car” and “automobile” mean the same thing, that “happy” is similar to “joyful,” and that a question about “vacation time” is related to a document about “paid time off.” Embeddings are what make RAG’s retrieval step useful. Here’s how they work.

Core Concepts

Traditional search engines (and simple database queries) look for documents containing the exact words you typed. This has serious limitations:
  • Synonyms: “Car” vs. “automobile” vs. “vehicle”
  • Phrasing variations: “How do I reset my password?” vs. “Password reset instructions”
  • Conceptual relationships: A document about “employee burnout” is relevant to a search about “workplace stress,” but shares few keywords
Keyword search is like a very literal-minded assistant who only recognizes exact matches. Useful, but frustrating when you need flexibility.

What Are Embeddings?

An embedding is a way of representing text as a list of numbers (called a vector) that captures its meaning. The key idea: similar meanings end up as similar numbers. Imagine you could place every word, sentence, or document somewhere on a map. Words with similar meanings would be close together on the map. “Happy” would be near “joyful” and “cheerful.” “Sad” would be in a different area, near “unhappy” and “melancholy.” Embeddings create this map. A paper map has two dimensions (left-right and up-down); an embedding has hundreds or thousands. Each dimension is one number in the list, and together they capture the meaning.

Numbers That Capture Meaning

When you convert a sentence to an embedding, you get something like:
These numbers aren’t random. They’re calculated so that:
  • “The cat sat on the mat” and “A feline rested on the rug” produce similar number sequences
  • “The stock market crashed yesterday” produces a very different sequence
Because they’re numbers, we can compare them with math to measure how similar two pieces of text are. Sequences that are close together have similar meanings.

Similarity in Embedding Space

“Close together” means mathematical distance in embedding space. Think of it like coordinates on a map: If you’re at coordinates (3, 4) and your friend is at (3.1, 4.2), you’re very close together. If another person is at (100, 200), they’re far away. Embeddings work the same way, just with many more dimensions. We can calculate the “distance” between any two embeddings and use it to measure how similar their meanings are. A common way to do this is cosine similarity, which compares the direction of two vectors: the closer it is to 1, the more alike the meanings.
  • Small distance = similar meaning
  • Large distance = different meaning

How Embeddings Power RAG

Here’s why embeddings are essential for RAG:
  1. Index time: Every chunk (short passage) of your knowledge base is converted to an embedding and stored
  2. Query time: When a user asks a question, that question gets converted to an embedding
  3. Search: The system finds stored embeddings that are closest to the question embedding
  4. Retrieval: The text chunks corresponding to those close embeddings get retrieved
This is semantic search: finding information based on meaning, not just keywords. So when someone asks “What’s the policy on working from home?” the system can find a document titled “Remote Work Guidelines” even though the words don’t match, because the embeddings are semantically similar.

The Embedding Model

Embeddings are created by an embedding model: an AI model trained to turn text into vectors that capture meaning. These models have learned from massive amounts of text to understand:
  • That synonyms should have similar embeddings
  • That questions and their answers should be related
  • That context matters (the word “bank” means different things in “river bank” vs. “bank account”)
Different embedding models have different strengths. Some are better at short text, others at long documents. Some are tuned for fields like law or medicine. Gloo offers embedding models from several providers through one Embeddings API. One rule matters whichever model you pick: embed your documents and your questions with the same model. Vectors from different models can’t be compared, so changing models means re-embedding everything; see Embeddings don’t follow replacements.

Dimensions and Trade-offs

Embedding vectors typically have hundreds to thousands of dimensions. Common sizes are 384, 768, 1024, or 1536 dimensions. More dimensions can capture more nuanced meanings but require more storage and computation. It’s a trade-off:
  • More dimensions: More nuanced understanding, but larger storage and slower search
  • Fewer dimensions: Faster and smaller, but might miss subtle differences in meaning
For most RAG applications, modern embedding models strike a good balance, so you only need to think hard about this for very specific requirements.

Try It Yourself

Exercise 1: Think in Similarities

Without any technology, just using your intuition: Rate how similar these pairs are on a scale of 1-10 (1 = completely different, 10 = almost identical):
  1. “The dog chased the ball” vs. “The puppy ran after the toy”
  2. “The dog chased the ball” vs. “Stock prices rose sharply”
  3. “How do I return an item?” vs. “What’s your refund policy?”
  4. “Python programming tutorial” vs. “Learn to code in Python”
Embedding models are trained to match human intuitions like yours. The pairs you rated as similar would have close embeddings; dissimilar pairs would be far apart.

Exercise 2: Synonym Exploration

Think of a topic and list five different ways to phrase a question about it: Topic: Getting time off from work
  1. “How do I request vacation days?”
  2. “What’s the PTO policy?”
  3. “Can I take time off next month?”
  4. “How do I schedule annual leave?”
  5. “What’s the process for booking holiday?”
A good embedding model would recognize all of these as semantically related, even though they share few keywords. That’s what makes flexible retrieval possible.

Exercise 3: When Keywords Fail

Think of a scenario where keyword search would completely fail: Example: You search your company wiki for “getting reimbursed for client dinners” but the relevant document is titled “Expense Policy for Business Entertainment.” Can you think of three similar scenarios in your own work or life where the words you’d naturally use don’t match the words in the source documents?

Common Pitfalls

Pitfall 1: Assuming Embeddings Are Perfect

Embeddings are powerful but not perfect. They can miss nuances, misunderstand context, or produce unexpected similarity scores. Edge cases exist. The fix: Test your retrieval with real queries. Don’t assume it will always find the right documents; verify it does.

Pitfall 2: Ignoring Domain Mismatch

An embedding model trained on general text might not perform well on highly specialized content. Legal jargon, medical terminology, or industry-specific language can confuse general-purpose models. The fix: For specialized domains, consider embedding models tuned for that domain, or test thoroughly with domain-specific queries.

Pitfall 3: Embedding the Wrong Units

What you embed matters. Embedding entire long documents produces less useful results than embedding appropriately sized chunks. The embedding represents the “average meaning” of the text; too much text and that meaning becomes diluted. The fix: Chunk your documents thoughtfully (we’ll cover this in Lesson 4) so each embedding represents a coherent, focused piece of information.

Pitfall 4: Forgetting That Embeddings Need Updates

If your knowledge base changes, the embeddings need to be updated too. New documents need to be embedded; deleted documents need their embeddings removed. The fix: Build embedding updates into your content management process. Stale embeddings mean stale search results.

Level Up

Here’s a thought experiment to deepen your understanding: Scenario: You’re building a RAG system for a multilingual customer support team. Customers write in English, Spanish, and French, but your documentation is only in English. Questions to consider:
  1. How might embeddings help here? (Hint: Some embedding models are multilingual, meaning “car” in English and “coche” in Spanish produce similar embeddings.)
  2. What challenges might still exist even with multilingual embeddings?
  3. How would you test whether the system works well across languages?

Key Takeaway

Embeddings convert text into numbers that capture meaning. Similar meanings produce similar embeddings, so RAG retrieval finds what you mean, not just what you typed.
Try it on Gloo: create embeddings with the Embeddings API, then use them for search in the Semantic Search with the Embeddings API tutorial.

What’s next

You now understand how embeddings enable smart retrieval, but before retrieval can happen, you need something to search through: a knowledge base. To use embeddings on Gloo, see the Embeddings guide, the Semantic Search with the Embeddings API tutorial and, when a model is retired, Choosing and Updating Models.

Lesson 4: Building a Knowledge Base

Prepare your information for RAG: chunking (breaking documents into searchable pieces), why chunk size matters, and how to turn raw content into a well-organized knowledge base.