Re-ranking
Dense retrieval gets you 80% of the way. The last 20% — the difference between "usually right" and "reliably right" — is reranking. Cross-encoders cost 100× more per query and are worth every cent. Cohere Rerank, BGE-Reranker, ColBERTv2 — these are the models that turned weekend RAG demos into Perplexity-grade products. Every production RAG above hobby-scale ships with a reranker, and here's why.
Learning Objectives
After this lesson, you will be able to:
- Explain why a two-stage retrieve-then-rerank pipeline beats single-stage retrieval — first stage casts a wide net, second stage applies a smarter (slower) model
- Use cross-encoder rerankers (Cohere Rerank, BGE-Reranker, MS MARCO MonoT5) to lift retrieval quality 10-30% with a 100ms cost per query
- Apply ColBERT/ColBERTv2 late-interaction for the best of both worlds — reranker quality at retrieval speed
- Pick the right reranker tier — bi-encoder retrieval, cross-encoder reranking, late-interaction — based on latency and corpus size
Prerequisites
Before You Start
Try it: Watch a cross-encoder rerank top-100 → top-10Interactive
Open the /play/reranking-stage playground to step through how a cross-encoder re-scores a candidate set retrieved by bi-encoder. Notice how documents at rank 47 jump to rank 2 — that's the NDCG@10 lift you're paying 100× per query for.