RAG (Retrieval-Augmented Generation)

Gen AI · Retrieval & Embeddings

Theory

Retrieval-Augmented Generation (RAG) combines search with generation. Before answering, the system retrieves the most relevant pieces of your own data (documents, tickets, manuals, wiki pages), inserts them into the prompt as context, and only then asks the model to answer. The model can now use specific, current or private information it was never trained on, such as last week's policy change.

The usual pipeline has two halves. Offline: split documents into chunks, embed each chunk and store the vectors. At question time: embed the question, find the top-matching chunks, and send them with the question and an instruction like "answer only from this context and cite the source." Updating knowledge is as simple as re-indexing changed documents, with no model retraining. Answer quality depends heavily on retrieval: if the right chunk is not found, the model cannot use it.

Why we use it

Models do not know your private or recent data and may guess. RAG grounds answers in real retrieved text, which reduces (but does not eliminate) hallucination and lets you show sources.

When to use it

Use RAG when answers must come from your own or frequently changing documents, such as policies, product docs or support history. If the problem is style or format rather than missing knowledge, better prompting or fine-tuning fits better.

How it works

  1. User asks a question
  2. Embed the question with the same model as the documents
  3. Search the vector store for the top-k chunks
  4. Were relevant chunks found? Yes: Build prompt: question + chunks + rules. No: Say the docs lack the answer.
  5. Model generates an answer from the given context
  6. Return the answer with source references

Example

User question: "What is our refund window for annual plans?"

1. Embed the question, search company docs vector store
2. Retrieve top match: "Annual plans: 30-day refund window..."
3. Build prompt:
   "Using ONLY this context: [retrieved text]\nAnswer: What is our refund window for annual plans?"
4. Model answers: "Annual plans have a 30-day refund window."

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic