Building a RAG App

This is the capstone — every concept in this track wired into one working Perplexity-style app: hybrid retrieval, Cohere rerank, Claude/GPT-4 with citations, streaming, query caching, RAGAS in CI, A/B testing two retrievers. The complexity of production RAG is in the integration, not any single component — and the integration is exactly what solo founders turned into billion-dollar products in 2024–2026. ~200 lines of Python, end-to-end.

Learning Objectives

After this lesson, you will be able to:

Architect a production RAG app end-to-end — ingestion, embedding, hybrid retrieval, reranking, generation, citations, evaluation, monitoring
Stitch together LangChain or LlamaIndex + Pinecone/Weaviate + Cohere Rerank + Claude/GPT-4 + RAGAS into a working Perplexity-style application
Apply production patterns: streaming responses, query caching, fallback handling, cost tracking, A/B testing different retrievers
Identify the most common production failure modes and the production hardening that prevents them

Prerequisites

Before You Start

Review

Why

Why This Matters

Why solo developers built billion-dollar products on RAG in 2024-2026 -- Perplexity, Glean, Mendable, You.com all started with the same RAG playbook; the complexity is in the integration, not in any single component
How a Perplexity-style search works under the hood -- query → web crawl + cached docs → hybrid retrieval → cross-encoder rerank → Claude/GPT-4 with citation prompt → streaming response with footnotes; this lesson builds exactly that
Why production RAG looks nothing like the LangChain quickstart -- caching, fallbacks, streaming, monitoring, eval pipelines, A/B testing infrastructure are all required; we'll cover the full stack
Build this --> Build a Perplexity-clone in ~200 lines: FastAPI endpoint that takes a query, hybrid-retrieves from a vector DB, reranks, generates an answer with inline citations, streams to the user, logs to RAGAS for offline eval

Try it: Step through the full production RAG stackInteractive

USER QUERY │ ▼ ┌────────────────────────────────────┐ │ 1. Query Preprocessing │ │ - Normalization │ │ - PII redaction │ │ - Query rewriting (optional) │ └────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────┐ │ 2. Hybrid Retrieval │ │ - BM25 search (sparse) │ │ - Dense vector search │ │ - RRF fusion │ │ - Top-50 candidates │ └────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────┐ │ 3. Reranker │ │ - Cohere Rerank or BGE │ │ - Top-50 → top-10 │ └────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────┐ │ 4. LLM Generation │ │ - System prompt with rules │ │ - Top-10 chunks as context │ │ - Streaming response │ │ - Inline citations │ └────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────┐ │ 5. Post-processing │ │ - Citation validation │ │ - Refusal detection │ │ - Cache write │ └────────────────────────────────────┘ │ ▼ USER STREAMING RESPONSE WITH FOOTNOTES │ └──► Logged to eval pipeline (RAGAS)

Concern	Solution
Streaming latency	Use SSE (Server-Sent Events); first-token < 1s
Cost overruns	Per-user rate limits, semantic caching, model tiering (mini for cheap queries, premium for hard ones)
Hallucination	Citation requirement in system prompt; post-process to verify cited claims actually appear in retrieved chunks
Refusals	Detect "I don't have information" responses; route to fallback (web search, escalate to human)
Stale corpus	Incremental re-embedding pipeline; track doc updated_at; re-embed on change
PII leakage	Redact PII before indexing; check responses for accidental PII regurgitation
Prompt injection	Sanitize retrieved chunks; treat them as untrusted; never let chunks override system prompt
Cost monitoring	Track tokens/query and $/query in real-time; alert on anomalies
A/B testing	Run two retrievers in parallel for 5% of traffic; compare RAGAS metrics
Eval CI	RAGAS golden-set runs on every PR; fail builds on > 3% regression

Building a RAG App

After this lesson, you will be able to:

Before You Start

Your Reflection

Continue Learning

Discussion

#The Production RAG Architecture

#Step-by-Step Implementation

#Production Hardening Checklist

#The Cost Math: Unit Economics of a RAG App

#Common Production Failure Modes

#Hands-On Walkthrough: Building a Perplexity-Clone

#Key Takeaways

#Quick Check

Deep Dive