Hallucination

Gen AI · Customizing Model Behavior

Theory

Hallucination is when a model confidently produces information that sounds fluent and plausible but is false or unsupported, such as a book that does not exist, a made-up legal case, a wrong statistic or a fake citation. It is not a simple bug to patch. The model generates likely next tokens from learned patterns and has no built-in step that checks claims against a source, so a wrong answer can read just as smoothly as a right one.

Several techniques reduce it, though none remove it: grounding answers in retrieved documents (RAG), asking for citations you can check, using low temperature for factual tasks, giving the model tools for lookups and calculations, and explicitly allowing "I don't know." Always verify high-stakes facts, numbers and citations before relying on them.

Why we use it

A fluent wrong answer is more dangerous than an obvious error, because people trust it. Knowing why hallucination happens tells you which design choices lower it and where human checks are essential.

When to use it

Watch for it whenever outputs contain facts, numbers, names, dates, code APIs or citations, especially in legal, medical, finance and reporting. For creative writing where nothing must be factual, it matters much less.

How it works

  1. Model produces an answer containing facts
  2. Is it based on given sources? Yes: Check claims against their sources. No: Treat claims as unverified.
  3. Verify numbers, names and citations against a trusted source
  4. Did any claim fail verification? Yes: Correct it or ask the model to answer from sources. No: Accept the answer.
  5. Use the verified answer

Example

Prompt: "Give me the exact page number where Einstein discusses quantum entanglement in his 1905 paper."

Model (hallucinating): "See page 47 of his 1905 paper, where he states..."
→ Einstein’s 1905 papers don’t discuss quantum entanglement at all — this citation is fabricated and must be verified against a real source before being trusted or published.

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic