Naive RAG works for "What is the refund policy?" It collapses on "Compare our 2024 returns policy with our competitors' and tell me where we lose customers." Production RAG layers on hybrid search, HyDE, multi-query expansion, cross-encoder rerank, and query routing — exactly the stack Perplexity uses to push from 70% to 95% answer accuracy. These are the patterns that separate the demo from the product.
Learning Objectives
After this lesson, you will be able to:
Build a hybrid search system that combines exact keyword matching (BM25) with meaning-based vector search, and merge their results intelligently using Reciprocal Rank Fusion (RRF)
Apply cross-encoder re-ranking to promote truly relevant documents after initial retrieval, and understand why bi-encoders are used for retrieval while cross-encoders are used for re-ranking
Use HyDE (Hypothetical Document Embeddings) and multi-query retrieval to improve recall for short or ambiguous queries, and know when each technique helps or hurts
Design multi-step retrieval pipelines for complex questions that need information from several different documents, and build a query router that sends different query types to the right retrieval strategy
This lesson takes your basic RAG skills and levels them up. The techniques here are what separate a weekend prototype from a product people actually rely on. If some concepts feel dense on first read, that is expected -- revisit after building your first RAG app and they will snap into focus.
Cross-reference results to find books covering ALL topics
Filter by date
Rank by overall relevance, not just similarity to one aspect
Advanced RAG techniques are the strategies this expert librarian uses: searching from multiple angles, re-evaluating results, expanding the search when initial results are poor, and even generating hypothetical answers to search more effectively.
The basic RAG pipeline -- embed query, find similar chunks, generate answer -- works surprisingly well for simple factual questions. But it fails on complex queries that require nuance, multi-hop reasoning, or domain-specific vocabulary. This lesson covers the techniques that close these gaps.
A user query enters the advanced RAG pipeline. It could be a specific error code like "ERR_CONNECTION_REFUSED" (needs exact keyword matching) or a conceptual question like "how do I fix login problems" (needs semantic understanding). Hybrid search handles both.
The query is processed by a BM25 keyword search engine. BM25 scores documents based on exact term frequency, weighted by how rare each term is in the corpus (IDF). It excels at finding documents containing specific identifiers, error codes, product names, and technical terms.
Simultaneously, the query is embedded into a dense vector and searched against the vector database. This semantic search finds documents with similar meaning even when different words are used. "Automobile" matches "car." "Hypertension treatment" matches "lowering high blood pressure."
The two ranked lists are combined using Reciprocal Rank Fusion (RRF). Each document's fused score is the sum of 1/(k + rank) across both lists. Documents that rank well in both keyword and semantic search bubble to the top. Documents that only appear in one list get lower fused scores.
The top candidates from the fused list are passed through a cross-encoder reranker. Unlike bi-encoders (which encode query and document separately), the cross-encoder processes query and document together through all transformer layers, capturing fine-grained relevance signals like negation and implicit relationships.
The reranker produces the final, high-precision ranking. The top 3-5 chunks are sent to the LLM as context. These chunks have been vetted by three systems: keyword matching confirmed exact term presence, semantic search confirmed meaning alignment, and the reranker confirmed detailed relevance.
Pure vector search excels at semantic similarity but can miss exact keyword matches. Pure keyword search (BM25) finds exact matches but misses semantic relationships. Hybrid search combines both.
What Do You Think?
A user searches for 'ERR_CONNECTION_REFUSED troubleshooting'. Which search type would find the most relevant results?
Figure
Two retrieval strategies run in parallel and are then merged. BM25 keyword search matches exact terms with TF-IDF weighting and produces its own ranked list — strong on names, codes, and rare vocabulary. Dense vector search matches on embedding similarity and produces a second ranked list — strong on paraphrase and meaning. A fusion step combines the two rankings into one, so a document that either method found can surface. Each covers the other's blind spot.
Hybrid search: combining keyword and semantic results
For technical queries with specific error codes, keywords, or identifiers, BM25 is essential. The embedding model might not distinguish "ERR_CONNECTION_REFUSED" from "ERR_TIMEOUT" -- both are "connection errors" semantically. But the user wants the exact error code.
Initial retrieval (whether dense, sparse, or hybrid) is fast but approximate. Re-ranking uses a more powerful (but slower) model to re-score the top candidates.
The standard pattern: use a bi-encoder to retrieve the top 50-100 candidates (fast), then a cross-encoder to re-rank to the top 5-10 (accurate).
Try it: Compare re-ranking strategiesInteractive
Step through the hybrid search + re-ranking pipeline. Watch how BM25 keyword search, dense vector search, reciprocal rank fusion, and cross-encoder re-ranking each reorder the results. Green documents are relevant; red are not. Notice how precision improves at each stage.
Bi-encoders compress all information into a single fixed-size vector. A 1536-dim vector cannot capture every nuance of a 500-token passage. Cross-encoders process query and document tokens together through full transformer attention, capturing nuanced relationships like:
"Python 3.9 compatibility" matching a document that mentions "works with Python 3.8+" (the bi-encoder might not catch that 3.9 is included in 3.8+)
"Not recommended for production" being anti-relevant to a query about "production best practices"
Use an LLM to generate multiple reformulations of the original query, search with each, and combine results.
pythonreference · read-only
1
2
3
4
5
6
7
8
9
10
11
def multi_query_expand(original_query, llm):
prompt = f"""Generate 3 different search queries that would
help answer: "{original_query}"
Return one query per line."""
expanded = llm.generate(prompt).split("\n")
all_results = []
for q in [original_query] + expanded:
results = vector_search(embed(q), k=5)
all_results.extend(results)
return deduplicate_and_rerank(all_results)
Try it! Take any question you might ask a RAG system (e.g., "How does photosynthesis work?") and manually write 3 different versions of the same question ("What is the process of photosynthesis?", "Explain how plants convert sunlight to energy", "Photosynthesis mechanism in plants"). Notice how different phrasings might match different documents. That is exactly what multi-query retrieval automates.
When HyDE helps: Short queries, questions using different vocabulary than the documents, complex conceptual queries.
When HyDE hurts: Factual queries about specific identifiers (the LLM might hallucinate wrong identifiers in its hypothetical answer).
Self-RAG and Corrective RAG (CRAG) take the agentic approach a step further — the model decides at inference time whether to retrieve at all, then critiques what it found. These patterns are covered in depth in the next lesson on Agentic RAG, where you'll see the full decision flow with interactive examples.
Generate: "The company that acquired TechCorp (MegaInc) had a 23% revenue growth rate."
pythonreference · read-only
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def multi_step_rag(query, max_steps=3):
context = []
current_query = query
for step in range(max_steps):
# Retrieve
docs = retrieve(current_query)
context.extend(docs)
# Check if we can answer
can_answer = llm.evaluate(
f"Given this context, can you fully answer: {query}?"
f"\nContext: {context}"
)
if can_answer:
break
# Generate a follow-up query
current_query = llm.generate(
f"Based on this context, what additional info is needed"
f" to answer: {query}?\nContext: {context}"
)
return llm.generate(query, context)
What Do You Think?
What is the main risk of multi-step RAG compared to single-step RAG?
Hybrid search combines the best of sparse and dense retrieval. BM25 excels at exact keyword matching while vector search captures semantic similarity; fusing both with reciprocal rank fusion consistently outperforms either alone
Re-ranking improves precision after initial retrieval. A cross-encoder re-ranker scores each query-document pair jointly, producing much more accurate relevance rankings than the initial bi-encoder retrieval
Query expansion and HyDE improve recall for ambiguous queries. Generating hypothetical answers (HyDE) or expanding queries with related terms helps retrieve documents that naive queries miss
Multi-step retrieval handles complex reasoning. For questions that span multiple documents, iterative retrieval (retrieve, reason, retrieve again) lets the system build up context step by step
Why is hybrid search (BM25 + vector) generally better than either approach alone?
You have now mastered the advanced techniques that make RAG production-ready. But what about knowledge that is inherently relational -- entities connected by relationships? Next up: Knowledge Graphs & Graph RAG -- combining structured knowledge with LLMs.