Hybrid Search & Reranking

Gen AI · Retrieval & Embeddings

Theory

Hybrid search runs two kinds of retrieval for the same question and merges the results. Keyword search (such as BM25) is strong on exact terms like product codes, error IDs and names. Vector search is strong on meaning and paraphrases. A popular way to merge the two ranked lists is reciprocal rank fusion (RRF): each document scores 1 / (k + rank) in each list, with k often set to 60, and the scores are added, so documents ranked high in both lists rise to the top.

Reranking is an optional second stage. A reranker model reads the question together with each of, say, the top 20 candidates and scores how well each one answers it; the best few are sent to the LLM. It is slower per document than vector search, which is why it only runs on a short list. Together these steps often improve RAG accuracy more than switching LLMs.

Why we use it

Pure vector search can miss exact terms like 'ERR-504' or a SKU, and pure keyword search misses paraphrases. Combining them, then reranking, puts the truly relevant chunks into the prompt.

When to use it

Use hybrid search when your documents mix natural language with codes, IDs or names, which is common in support, product and data documentation. Add a reranker when the right chunk is often retrieved but not ranked in the top few.

How it works

  1. User question arrives
  2. Run keyword search and vector search in parallel
  3. Merge the two ranked lists with reciprocal rank fusion
  4. Is a reranker configured? Yes: Rerank the top candidates with question-aware scoring. No: Keep the fused order.
  5. Take the top few chunks
  6. Send them to the LLM as context

Example

Question: "How do I fix ERR-504 on export?"

Keyword search : finds the page that literally contains "ERR-504"
Vector search  : finds "Export times out on large files" (same meaning)
RRF merge      : both pages appear near the top
Reranker       : puts the ERR-504 troubleshooting section first

LLM receives the 3 best chunks and answers with steps + source links.

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic