RAG (Retrieval-Augmented Generation)
Fetching relevant documents at query time and injecting them into the LLM prompt to ground answers in real data.
Last updated: April 26, 2026
Definition
RAG is the pattern that makes generic LLMs answer questions about your specific data. The flow: (1) chunk your documents and embed each chunk into a vector; (2) at query time, embed the user question and find the most similar chunks; (3) pass those chunks as context to the LLM. The LLM grounds its answer in retrieved text instead of training-data memory. This solves three problems: hallucination (retrieved text is real), freshness (data updates without retraining), and source citation (retrieved chunks become footnotes).
Code Example
# Two-line RAG, conceptually
chunks = vector_store.search(query=question, top_k=5)
answer = llm.complete(
f"Answer using only this context: {chunks}\n\nQuestion: {question}"
)Conceptually two lines. Production adds reranking, citation extraction, and fallbacks.
When To Use
Default pattern when an LLM needs to answer questions about data it was not trained on. Cheaper and faster than fine-tuning. Easier to update.
Common Questions
What is the difference between RAG and fine-tuning?
RAG retrieves data at query time. Fine-tuning bakes data into model weights via additional training. RAG is cheaper, faster to update, and easier to attribute. Fine-tuning is better for changing behavior (style, format) than for adding knowledge.
Related Terms
Building with RAG (Retrieval-Augmented Generation)?
I've shipped this pattern in real production systems. If you want a second pair of eyes on your architecture, that's what I do.