Retrieval quality
How good a search or AI retrieval system is at finding the right information, returning what is genuinely relevant without missing important results or including irrelevant ones.
The Simple Version
How good a search or AI retrieval system is at finding the right information, returning what is genuinely relevant without missing important results or including irrelevant ones.
Detailed Explanation
Retrieval quality is assessed using established information retrieval metrics: precision@k (fraction of top-k results that are relevant), recall@k (fraction of relevant items returned in top k), Mean Reciprocal Rank (MRR, average of reciprocal rank of first relevant result), Normalised Discounted Cumulative Gain (NDCG, measures relevance ranking quality). In RAG systems, retrieval quality is measured additionally by context relevance (are retrieved passages relevant to the query?) and context faithfulness (does retrieved content actually support the generated answer?). Poor retrieval quality is the primary cause of RAG hallucination, the model generates from irrelevant or missing context.
Key Characteristics
- Assessed by precision, recall, MRR, and NDCG for ranking quality
- In RAG: context relevance and context faithfulness are key additional metrics
- Primary determinant of RAG output quality and hallucination rate
- Depends on embedding model quality, chunking strategy, and re-ranking
Why It Matters
AI teams building RAG systems must establish retrieval quality baselines and benchmarks before focusing on generation quality, low retrieval precision cannot be compensated by generation model capability.
Real-World Analogy
Like measuring the quality of a reference librarian, not just whether they found any relevant books, but whether the books they selected are the most relevant ones, and whether they missed any important titles.
Common Misconceptions
- High retrieval recall guarantees high RAG output quality, high recall means few relevant documents are missed, but without high precision, the model is overwhelmed with irrelevant context that degrades generation.
- Retrieval quality only matters for keyword queries, semantic and complex multi-part queries typically require more sophisticated evaluation to capture retrieval quality accurately.
Related Terms
Related Articles
- The Shift Toward Retrieval-Augmented Generation: Architecture Implications
- How AI Shopping Assistants Recommend Products