What Is Retrieval-Augmented Generation and Dense Passage Retrieval?

What Is Retrieval-Augmented Generation and Dense Passage Retrieval?

Retrieval-Augmented Generation (RAG) is the pattern where an LLM answers using context retrieved at query time from an external corpus, rather than relying on parametric knowledge from training. Dense Passage Retrieval (DPR) — Karpukhin et al. (2020) — is the dense-embedding retriever that became the canonical RAG retrieval method. DPR uses two BERT-style encoders (query and passage) that project into a shared vector space; retrieval is nearest-neighbor search over passage embeddings. RAG + DPR is the textbook pipeline: embed query, retrieve top-k with DPR, feed as context, generate. Modern 2026 stacks add hybrid retrieval, reranking, and chunk-level evaluation.

Why It Matters in Production LLM and Agent Systems

RAG is how production LLMs handle knowledge they were never trained on — internal docs, current product catalogs, regulatory filings, customer history. Without retrieval, the model has to either confabulate or refuse. The retrieval quality directly determines the answer quality: a perfect generator with a wrong retrieved passage produces a confidently wrong answer; a great retriever with an under-grounded generator produces a fluent but unfaithful answer.

DPR-style dense retrieval became the default because it captures paraphrase (“waterproof rating” vs “water resistance spec”) that lexical retrievers like BM25 miss. The pain shows up when teams ship dense retrieval without measurement. ML engineers see top-1 hit rate at 60% and have no signal on whether the failures are encoder coverage gaps or chunking artifacts. Compliance leads cannot demonstrate the system grounds its answers because no eval pins each output to a retrieved source. Product managers debug “the bot is wrong about pricing” complaints and find the retrieval index has six-week-old chunks for a SKU that updated last night.

In 2026 RAG stacks the surface widens. Agentic RAG variants (corrective RAG, self-RAG, modular RAG) issue multiple retrievals per query and reason about retrieved context across steps. Multi-vector and parent-document retrievers replace single-vector DPR in many systems. Reranking layers on top of DPR are now standard. Each stage is a place where retrieval quality can collapse and produce a wrong final answer; each stage needs its own evaluator.

How FutureAGI Handles RAG and DPR Evaluation

FutureAGI does not run the retriever or the LLM — those live in your stack (LangChain, LlamaIndex, Haystack, or hand-rolled). FutureAGI sits across both surfaces and answers the questions a RAG team needs: did the retriever fetch the right context, and did the generator actually use it?

Concretely, a team running DPR retrieval over a product catalog builds a held-out Dataset of representative queries with ground-truth passages and ground-truth answers. Dataset.add_evaluation() runs ContextRelevance (does the retrieved chunk match the query?), ChunkAttribution (which retrieved chunks did the answer actually use?), ContextRecall (did retrieval surface the chunks that answer the query?), and Groundedness (did the answer stay anchored to retrieved context?). The four evaluators isolate whether failures are encoder-coverage gaps, reranker mistakes, or grounding errors.

RegressionEval reruns the cohort against every retriever candidate (DPR vs hybrid vs reranked DPR) so the team can pick the configuration that maximizes downstream answer quality, not just retrieval recall. In production, traceAI captures retrieval spans with the retrieved chunks attached as span attributes; Groundedness runs on every sampled trace and writes a score back to the span. An eval-fail-rate-by-cohort dashboard surfaces RAG quality drifts on specific query types, store regions, or document categories before users complain. FutureAGI’s approach is that RAG quality is a stack of measurable stages, not a single end-to-end vibe check.

How to Measure or Detect It

RAG + DPR pipeline quality is measured per stage:

• fi.evals.ContextRelevance: scores whether retrieved chunks match the query; the canonical retriever-quality metric.
• fi.evals.ContextRecall: scores whether retrieval surfaced the chunks needed to answer; complements precision-style metrics.
• fi.evals.ChunkAttribution: which retrieved chunks the generated answer actually used; reveals over-retrieval and unused-chunk waste.
• fi.evals.Groundedness: scores whether the answer anchors to retrieved context, regardless of whether the answer is correct.
• Top-k retrieval recall: standard IR metric; track top-1, top-5, top-10 against ground-truth.
• Hybrid-vs-DPR delta: head-to-head retrieval quality against a hybrid (BM25 + DPR) baseline; informs whether to keep DPR alone.

from fi.evals import Groundedness, ContextRelevance

g = Groundedness()
cr = ContextRelevance()

result = cr.evaluate(
    input="Is the Alpine-3 jacket waterproof?",
    context="Alpine-3 specs: 10K mm waterproof rating, 2L shell fabric."
)
print(result.score, result.reason)

Common Mistakes

• Trusting top-1 retrieval recall as a system metric. A retriever with 90% top-1 recall can still produce 60% answer accuracy if the generator does not actually use the right chunk.
• Skipping reranking on DPR output. Pure dense retrieval tends to over-cluster on stylistic similarity; a reranker materially improves precision.
• No chunk-attribution telemetry. Without knowing which chunks the answer used, debugging a wrong answer is guessing.
• Stale embeddings on a moving corpus. A retail catalog or policy doc that updates daily needs a re-embedding pipeline; otherwise retrieval is silently wrong.
• Single embedding model across everything. Specialized domains (medical, legal, code) often beat generic encoders by 10+ points; benchmark on your distribution.