Skip to main content

Retrieval-Augmented Generation

An AI approach that looks up relevant information in real time before generating an answer, combining the flexibility of language models with the accuracy of up-to-date knowledge retrieval.

The Simple Version

An AI approach that looks up relevant information in real time before generating an answer, combining the flexibility of language models with the accuracy of up-to-date knowledge retrieval.

Detailed Explanation

RAG addresses two key LLM limitations: knowledge staleness (training data has a cutoff date) and hallucination (generating plausible but false claims). The RAG pipeline typically involves: query encoding (converting the user query to an embedding), retrieval (finding semantically similar document chunks in a vector database), context injection (providing retrieved chunks to the LLM as context), and generation (producing a response grounded in retrieved content). Advanced RAG variants include iterative retrieval, query rewriting, re-ranking, and hybrid sparse-dense retrieval. RAG quality depends on retrieval quality, poor retrieval introduces incorrect context that may degrade generation.

Key Characteristics

  • Combines language model generation with real-time document retrieval
  • Reduces hallucination by grounding responses in retrieved, verifiable content
  • Addresses LLM knowledge cutoff limitations with current retrieval sources
  • Generation quality is fundamentally dependent on retrieval quality

Why It Matters

RAG is the dominant architecture for enterprise AI applications requiring accuracy, citability, and currency, replacing custom fine-tuning for many knowledge-intensive tasks because it is cheaper to update retrieved knowledge than to retrain models.

Real-World Analogy

Like an open-book exam versus a closed-book exam. RAG allows the AI to consult relevant reference material before answering, producing more accurate responses than relying on memorised knowledge alone.

Common Misconceptions

  • RAG eliminates hallucination. RAG significantly reduces hallucination but cannot eliminate it if retrieved content is poor quality or if the model generates claims beyond what was retrieved.
  • RAG is only suitable for text retrieval. RAG architectures can be extended to retrieve structured data, images, and other modalities alongside text.

Related Terms

Related Articles

Sources & Further Reading