The RAG Pipeline

Most teams ship a RAG demo in a weekend and a broken RAG product in production. The pipeline looks identical on a whiteboard, but the difference is the seven stages where things silently go wrong. This lesson walks the full pipeline end-to-end — extraction, chunking, embedding, retrieval, reranking, prompt assembly, generation — and shows you exactly where Perplexity, Glean, and Notion AI invested to make theirs actually work.

Learning Objectives

After this lesson, you will be able to:

Follow a user's question through every step of the RAG pipeline -- from loading documents, to searching, to generating an answer -- and understand what happens at each stage
Spot how a failure at any one stage (bad chunking, poor retrieval, weak prompt) cascades and ruins the final answer
Write prompt templates that tell the AI 'answer ONLY from these documents' so it stays grounded instead of making things up
Debug RAG failures systematically from left to right -- extraction first, then chunking, then retrieval, then generation -- instead of jumping straight to prompt tuning

Prerequisites

Before You Start

Review

You know why RAG exists. Now let's go deep on how each stage actually works in code.

This is the "putting it all together" lesson. Everything you have learned so far -- embeddings, chunking, vector search -- connects here into one working system. Take your time and enjoy watching the pieces fit together.

# Common document loaders from langchain.document_loaders import ( PyPDFLoader, # PDFs UnstructuredHTMLLoader, # HTML NotionDBLoader, # Notion CSVLoader, # CSV files ) docs = PyPDFLoader("report.pdf").load()

from langchain.text_splitter import RecursiveCharacterTextSplitter splitter = RecursiveCharacterTextSplitter( chunk_size=500, chunk_overlap=50, separators=["\n\n", "\n", ". ", " "] ) chunks = splitter.split_documents(docs)

from openai import OpenAI client = OpenAI() def embed_batch(texts, model="text-embedding-3-small"): response = client.embeddings.create(input=texts, model=model) return [e.embedding for e in response.data] # Process in batches of 100 for API efficiency vectors = [] for i in range(0, len(chunks), 100): batch = [c.page_content for c in chunks[i:i+100]] vectors.extend(embed_batch(batch))

import chromadb client = chromadb.Client() collection = client.create_collection("knowledge_base") collection.add( ids=[f"chunk_{i}" for i in range(len(chunks))], embeddings=vectors, documents=[c.page_content for c in chunks], metadatas=[{ "source": c.metadata["source"], "page": c.metadata.get("page", 0), "chunk_index": i, } for i, c in enumerate(chunks)] )

def process_query(raw_query): # Optional: use an LLM to rewrite for better retrieval rewritten = llm.rewrite( f"Rewrite for document retrieval: {raw_query}" ) return rewritten or raw_query

# Re-rank with a cross-encoder from sentence_transformers import CrossEncoder reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2") pairs = [(query, doc) for doc in retrieved_docs] scores = reranker.predict(pairs) reranked = sorted(zip(scores, retrieved_docs), reverse=True) top_docs = [doc for _, doc in reranked[:3]]

prompt = f"""Answer the user's question based ONLY on the following context. If the context doesn't contain the answer, say "I don't have enough information to answer that." Context: {chr(10).join(top_docs)} Question: {user_query} Answer:"""

response = llm.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": augmented_prompt} ], stream=True, temperature=0.1 # Low temp for factual answers )

The RAG Pipeline

After this lesson, you will be able to:

Before You Start

Your Reflection

Continue Learning

Discussion

#The Full RAG Pipeline: End to End

#Step 1: User Question

#Step 2: Embed the Query

#Step 3: Vector Search

#Step 4: Retrieve Top-K Chunks

#Step 5: Rerank Results

#Step 6: Assemble the Prompt

#Step 7: LLM Generates the Answer

#Step 8: Return with Citations

#Phase 1: Offline Ingestion

#Stage 1: Document Loading

#Stage 2: Text Cleaning & Preprocessing

#Stage 3: Chunking

#Stage 4: Embedding

#Stage 5: Indexing & Storage

#Phase 2: Online Query

#Stage 6: Query Processing

#Stage 7: Query Embedding

#Stage 8: Retrieval

#Stage 9: Post-Retrieval Processing

#Stage 10: Prompt Construction

#Stage 11: Generation

#The Cascade Effect

#Prompt Engineering for RAG

#Basic RAG Prompt

#Citation-Aware Prompt

#Structured Output Prompt

#The Lost in the Middle Problem

#Try It Yourself

#Key Takeaways

#Quick Check

Deep Dive

Related Lessons