Advanced RAG Patterns

This lesson takes your basic RAG skills and levels them up. The techniques here are what separate a weekend prototype from a product people actually rely on. If some concepts feel dense on first read, that is expected — revisit after building your first RAG app and they will snap into focus.

#Beyond Naive RAG

The basic RAG pipeline — embed query, find similar chunks, generate answer — works surprisingly well for simple factual questions. But it fails on complex queries that require nuance, multi-hop reasoning, or domain-specific vocabulary. This lesson covers the techniques that close these gaps.

#How Hybrid Search Works: End to End

#Hybrid Search: Best of Both Worlds

Pure vector search excels at semantic similarity but can miss exact keyword matches. Pure keyword search (BM25) finds exact matches but misses semantic relationships. Hybrid search combines both.

For technical queries with specific error codes, keywords, or identifiers, BM25 is essential. The embedding model might not distinguish "ERR_CONNECTION_REFUSED" from "ERR_TIMEOUT" — both are "connection errors" semantically. But the user wants the exact error code.

#BM25: The Classic

#Reciprocal Rank Fusion (RRF)

The simplest and most effective way to combine results from multiple search methods:

#Weighted Combination

An alternative to RRF: normalize scores from each system to [0, 1] and take a weighted average.

#Re-Ranking: The Second Opinion

Initial retrieval (whether dense, sparse, or hybrid) is fast but approximate. Re-ranking uses a more powerful (but slower) model to re-score the top candidates.

#Bi-Encoder vs Cross-Encoder

The standard pattern: use a bi-encoder to retrieve the top 50-100 candidates (fast), then a cross-encoder to re-rank to the top 5-10 (accurate).

#Why Re-Ranking Works

Bi-encoders compress all information into a single fixed-size vector. A 1536-dim vector cannot capture every nuance of a 500-token passage. Cross-encoders process query and document tokens together through full transformer attention, capturing nuanced relationships like:

• "Python 3.9 compatibility" matching a document that mentions "works with Python 3.8+" (the bi-encoder might not catch that 3.9 is included in 3.8+)
• "Not recommended for production" being anti-relevant to a query about "production best practices"

#Query Expansion & Transformation

Sometimes the user's query is too short, ambiguous, or uses different vocabulary than the documents. Query transformation techniques bridge this gap.

#Multi-Query Retrieval

Use an LLM to generate multiple reformulations of the original query, search with each, and combine results.

def multi_query_expand(original_query, llm):
    prompt = f"""Generate 3 different search queries that would
    help answer: "{original_query}"
    Return one query per line."""

    expanded = llm.generate(prompt).split("\n")
    all_results = []
    for q in [original_query] + expanded:
        results = vector_search(embed(q), k=5)
        all_results.extend(results)
    return deduplicate_and_rerank(all_results)

Try it! Take any question you might ask a RAG system (e.g., "How does photosynthesis work?") and manually write 3 different versions of the same question ("What is the process of photosynthesis?", "Explain how plants convert sunlight to energy", "Photosynthesis mechanism in plants"). Notice how different phrasings might match different documents. That is exactly what multi-query retrieval automates.

#HyDE: Hypothetical Document Embeddings

#Self-RAG & Corrective RAG

Self-RAG and Corrective RAG (CRAG) take the agentic approach a step further — the model decides at inference time whether to retrieve at all, then critiques what it found. These patterns are covered in depth in the next lesson on Agentic RAG, where you'll see the full decision flow with interactive examples.

#Multi-Step RAG: Reasoning Across Documents

Some questions require information from multiple documents combined with reasoning. Basic RAG retrieves once; multi-step RAG retrieves iteratively.

#Example: Multi-Hop Question

"What was the revenue growth rate of the company that acquired TechCorp in 2023?"

This requires:

1. Retrieve: "Who acquired TechCorp in 2023?" -> "MegaInc acquired TechCorp in 2023"
2. Retrieve: "What was MegaInc's revenue growth rate?" -> "MegaInc grew revenue 23% YoY"
3. Generate: "The company that acquired TechCorp (MegaInc) had a 23% revenue growth rate."

def multi_step_rag(query, max_steps=3):
    context = []
    current_query = query

    for step in range(max_steps):
        # Retrieve
        docs = retrieve(current_query)
        context.extend(docs)

        # Check if we can answer
        can_answer = llm.evaluate(
            f"Given this context, can you fully answer: {query}?"
            f"\nContext: {context}"
        )

        if can_answer:
            break

        # Generate a follow-up query
        current_query = llm.generate(
            f"Based on this context, what additional info is needed"
            f" to answer: {query}?\nContext: {context}"
        )

    return llm.generate(query, context)

#Try It Yourself

#Key Takeaways

• Hybrid search combines the best of sparse and dense retrieval. BM25 excels at exact keyword matching while vector search captures semantic similarity; fusing both with reciprocal rank fusion consistently outperforms either alone
• Re-ranking improves precision after initial retrieval. A cross-encoder re-ranker scores each query-document pair jointly, producing much more accurate relevance rankings than the initial bi-encoder retrieval
• Query expansion and HyDE improve recall for ambiguous queries. Generating hypothetical answers (HyDE) or expanding queries with related terms helps retrieve documents that naive queries miss
• Multi-step retrieval handles complex reasoning. For questions that span multiple documents, iterative retrieval (retrieve, reason, retrieve again) lets the system build up context step by step

#Quick Check