Prompting vs RAG vs Fine-Tuning

Gen AI · Customizing Model Behavior

Theory

When a model does not behave the way you need, there are three main levers, and they fix different problems. Prompting (clear instructions, examples, output format) changes behaviour for one request and costs nothing to try. RAG gives the model facts it does not have by retrieving your documents at question time, which fixes missing or changing knowledge. Fine-tuning changes the model's default behaviour through extra training, which fixes consistent style, format or a narrow task at high volume.

A useful rule: if the model lacks information, use RAG; if it has the information but behaves wrongly, improve the prompt first and fine-tune only if that is not enough. Try them in order of cost: prompt, then RAG, then fine-tuning, measuring each step on the same test set. Many production systems use all three together.

Why we use it

Teams often jump to fine-tuning when a better prompt or retrieval would solve the problem faster and cheaper. Knowing what each lever fixes saves weeks of effort and keeps knowledge easy to update.

When to use it

Use this decision whenever an LLM feature falls short. Diagnose the failure first: wrong facts or missing private data points to RAG; wrong format, tone or task behaviour points to prompting, then fine-tuning.

How it works

  1. The model's answers are not good enough
  2. Improve the prompt: instructions, examples, format
  3. Is the problem missing or changing knowledge? Yes: Add RAG over your documents. No: Keep the improved prompt.
  4. Is behaviour still inconsistent at scale? Yes: Fine-tune on curated examples. No: Ship the current setup.
  5. Measure every change on the same test set

Example

Problem 1: Bot does not know this month's refund policy
  -> knowledge gap -> RAG over the policy documents

Problem 2: Bot knows the answer but replies in long paragraphs
  -> behaviour -> add format rules + 2 examples to the prompt

Problem 3: 50,000 tickets/day must be tagged in a strict house format,
           prompt works but is long and costly
  -> fine-tune a smaller model on 2,000 labelled tickets

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic