Step by Step
Q
Query — the user's question arrives
The pipeline begins when a user's query arrives, initiating the retrieval process.
Example: a user asking "What is our company's current parental leave policy?" as the initiating query.
R
Retrieve — semantic search finds relevant chunks
Semantic search over a vector database finds the top-K most relevant document chunks related to the query.
Example: semantic search retrieving the 5 most relevant chunks from the company's HR policy documents related to parental leave.
A
Augment — retrieved chunks injected into the prompt
The retrieved chunks are injected directly into the prompt, typically in a format like "Answer based on the following documents: [chunks]. Question: [query]".
Example: constructing a prompt that includes the 5 retrieved policy document chunks, followed by the original user question, before sending this combined prompt to the LLM.
G
Generate — the LLM produces a grounded answer
The LLM generates its final answer grounded in these retrieved documents, providing benefits including factual grounding, knowledge updates without retraining, source citations, and improved privacy (since data stays in your own vector database).
Example: the LLM generating an accurate answer about parental leave policy, grounded specifically in the retrieved HR documents rather than relying purely on its own potentially outdated training memory.
Applied Walkthrough
1
An employee asks a company chatbot about a recent policy change that occurred after the underlying LLM's training cutoff.
2
The RAG pipeline retrieves the most relevant, up-to-date policy document chunks from the company's own vector database, rather than relying on the model's outdated training memory.
3
These retrieved chunks are injected into the prompt alongside the employee's question, and the LLM generates its answer grounded in this current, retrieved information.
4
Even with this grounding, the LLM could still occasionally misread or misinterpret the retrieved context, illustrating why RAG reduces hallucination significantly but doesn't fully eliminate it — the underlying generation step can still introduce errors even with accurate source material provided.
Exam Application
Exams test whether you can correctly sequence the four RAG stages (query, retrieve, augment, generate) and whether you understand RAG's genuine benefits (factual grounding, knowledge updates without retraining, citations, privacy) alongside its key limitation — it reduces but does not eliminate hallucination, since the model can still misread or misinterpret retrieved context.
⚠ Common Trap
The most common trap is assuming RAG completely eliminates hallucination, since it provides real, retrieved facts as context. RAG significantly reduces hallucination but does not eliminate it entirely — the model can still misread, misinterpret, or incorrectly synthesize the retrieved context, producing an inaccurate answer despite having access to accurate source material.
✓ Quick Self-Check
1. What are the four stages of the RAG pipeline, in order?
Query, Retrieve, Augment, Generate.
Tap to reveal / hide
2. What happens during the Retrieve stage?
Semantic search over a vector database finds the top-K most relevant document chunks related to the query.
Tap to reveal / hide
3. What happens during the Augment stage?
The retrieved chunks are injected directly into the prompt, alongside the original question.
Tap to reveal / hide
4. Name one genuine benefit of RAG beyond just reducing hallucination.
Knowledge updates without retraining, source citations, or improved privacy (data stays in your own vector database) — any one.
Tap to reveal / hide
5. Does RAG completely eliminate hallucination?
No — it significantly reduces hallucination, but the model can still misread or misinterpret the retrieved context, so hallucination is not fully eliminated.
Tap to reveal / hide