RAG (Retrieval-Augmented Generation)
Fetching relevant documents at query time and injecting them into the LLM prompt to ground answers in real data.
Last updated: April 26, 2026
What Is RAG (Retrieval-Augmented Generation)?
RAG is the pattern that makes generic LLMs answer questions about your specific data. The flow: (1) chunk your documents and embed each chunk into a vector; (2) at query time, embed the user question and find the most similar chunks; (3) pass those chunks as context to the LLM. The LLM grounds its answer in retrieved text instead of training-data memory. This solves three problems: hallucination (retrieved text is real), freshness (data updates without retraining), and source citation (retrieved chunks become footnotes).
Code Example
# Two-line RAG, conceptually
chunks = vector_store.search(query=question, top_k=5)
answer = llm.complete(
f"Answer using only this context: {chunks}\n\nQuestion: {question}"
)Conceptually two lines. Production adds reranking, citation extraction, and fallbacks.
When Should You Use RAG (Retrieval-Augmented Generation)?
Default pattern when an LLM needs to answer questions about data it was not trained on. Cheaper and faster than fine-tuning. Easier to update.
Common Questions
What is the difference between RAG and fine-tuning?
RAG retrieves data at query time. Fine-tuning bakes data into model weights via additional training. RAG is cheaper, faster to update, and easier to attribute. Fine-tuning is better for changing behavior (style, format) than for adding knowledge.
Related Terms
Building with RAG (Retrieval-Augmented Generation)?
I've shipped this pattern in real production systems. If you want a second pair of eyes on your architecture, that's what I do.