Step by Step
R
Retrieve — looking up relevant documents
The system retrieves relevant documents from a knowledge base, based on the user's query, rather than relying purely on what the model memorized during training.
Example: retrieving the 5 most relevant support articles from a company's knowledge base for a specific customer question.
A
Augment — adding retrieved context to the prompt
The retrieved documents are added directly into the LLM's prompt as additional context, augmenting what the model has to work with beyond its own memorized training data.
Example: inserting the text of those 5 retrieved support articles directly into the prompt, right alongside the user's original question.
G
Generate — answering grounded in retrieved facts
The LLM generates its final answer grounded in this retrieved, augmented context, rather than purely from its internal memorized knowledge — resulting in far fewer hallucinations, more up-to-date answers, and the ability to cite sources.
Example: the model generating an answer that directly references and is grounded in the specific retrieved support articles, rather than potentially hallucinating an answer purely from training memory.
Applied Walkthrough
1
A user asks a chatbot a question about a very recent company policy change that occurred after the underlying LLM's training cutoff.
2
Using a pure LLM (a "closed-book exam"), the model would have to rely purely on its memorized training data, which doesn't include this recent policy change, risking either an outdated or hallucinated answer.
3
Using RAG (an "open-book exam") instead, the system retrieves the current, up-to-date policy document from the company's knowledge base and includes it directly in the prompt.
4
The LLM then generates its answer grounded in this retrieved, current document, producing an accurate, up-to-date response rather than relying on potentially outdated or hallucinated memorized knowledge — exactly why most enterprise AI systems use RAG.
Exam Application
Exams test whether you can describe the three RAG stages (retrieve, augment, generate) and whether you understand the closed-book vs. open-book exam analogy for explaining why RAG reduces hallucinations and enables up-to-date, citable answers.
⚠ Common Trap
The most common trap is assuming RAG changes the underlying LLM itself. RAG doesn't retrain or modify the model's weights at all — it simply provides additional retrieved context at inference time, which the existing model then uses to generate a more grounded answer.
✓ Quick Self-Check
1. What does the R in RAG stand for, and what does that stage do?
Retrieve — fetching relevant documents from a knowledge base based on the user's query.
Tap to reveal / hide
2. What does the A in RAG stand for, and what does that stage do?
Augment — adding the retrieved documents directly into the LLM's prompt as additional context.
Tap to reveal / hide
3. What does the G in RAG stand for, and what does that stage do?
Generate — the LLM produces its final answer, grounded in the retrieved, augmented context.
Tap to reveal / hide
4. What is the closed-book vs. open-book exam analogy meant to illustrate?
A pure LLM (closed-book) relies only on memorized training data; RAG (open-book) allows the model to look up relevant information before answering.
Tap to reveal / hide
5. Does RAG modify or retrain the underlying LLM's weights?
No — RAG only provides additional retrieved context at inference time; the model's weights remain unchanged.
Tap to reveal / hide