Hybrid Search & Reranking
Gen AI · Retrieval & Embeddings
Theory
Hybrid search runs two kinds of retrieval for the same question and merges the results. Keyword search (such as BM25) is strong on exact terms like product codes, error IDs and names. Vector search is strong on meaning and paraphrases. A popular way to merge the two ranked lists is reciprocal rank fusion (RRF): each document scores 1 / (k + rank) in each list, with k often set to 60, and the scores are added, so documents ranked high in both lists rise to the top.
Reranking is an optional second stage. A reranker model reads the question together with each of, say, the top 20 candidates and scores how well each one answers it; the best few are sent to the LLM. It is slower per document than vector search, which is why it only runs on a short list. Together these steps often improve RAG accuracy more than switching LLMs.
Why we use it
Pure vector search can miss exact terms like 'ERR-504' or a SKU, and pure keyword search misses paraphrases. Combining them, then reranking, puts the truly relevant chunks into the prompt.
When to use it
Use hybrid search when your documents mix natural language with codes, IDs or names, which is common in support, product and data documentation. Add a reranker when the right chunk is often retrieved but not ranked in the top few.
How it works
- User question arrives
- Run keyword search and vector search in parallel
- Merge the two ranked lists with reciprocal rank fusion
- Is a reranker configured? Yes: Rerank the top candidates with question-aware scoring. No: Keep the fused order.
- Take the top few chunks
- Send them to the LLM as context
Example
Question: "How do I fix ERR-504 on export?"
Keyword search : finds the page that literally contains "ERR-504"
Vector search : finds "Export times out on large files" (same meaning)
RRF merge : both pages appear near the top
Reranker : puts the ERR-504 troubleshooting section first
LLM receives the 3 best chunks and answers with steps + source links.Key points
- Keyword search finds exact terms; vector search finds meaning
- Reciprocal rank fusion merges ranked lists without tuning score scales
- A reranker scores question and chunk together for sharper ranking
- Rerank only a short list because rerankers are slower per document
- Measure retrieval quality before and after each change