RAG Basics

Two students sit the same exam. One studied the textbook a year ago and answers from memory. The other is allowed to bring three pages of notes, but only three, and must pick them right before the exam starts. On questions about the topics in those three pages, the second student wins easily, and can point to exactly where each answer came from.

A model answering from training is the first student. A shop's help pages, a company handbook or last week's meeting notes were never in that memory. You could paste them all into every request, but a real document collection is far bigger than any context window. The fix is to be the second student: look up the few passages that matter, then answer from them.

#Retrieve, augment, generate

That is all RAG (retrieval-augmented generation) means:

1. Retrieve. Search your documents for the passages most related to the question.
2. Augment. Put those passages into the prompt, with instructions to answer from them.
3. Generate. The model writes the answer, ideally quoting or citing the passages.

The model is unchanged. RAG is a context engineering technique: an automatic way to select the right material for each question.

The search usually works on meaning rather than exact words, using embeddings. That matters because people rarely use the same words as your documents. Someone asks "when do I get my money back?" and the answer sits under the heading "Refund processing times".

Indexing is the slow part, so it happens ahead of time. Each document is cut into passages called chunks (how to cut them well is the whole of the next lesson), each chunk is turned into a vector, and the vectors are saved. At question time you only embed the question and compare.

#Retrieval by hand

A bookshop's help centre has five chunks. To trace it by hand, pretend embeddings have just three numbers, meaning roughly "about money", "about shipping" and "about accounts". Real embeddings have hundreds or thousands of dimensions, but the arithmetic is the same.

The question "When will I get my money back?" embeds to [5, 1, 0]: mostly money, a little shipping. Its length is √26 ≈ 5.10. Score each chunk with cosine similarity, from the Embeddings lesson, and keep the top 2.

q = embed(question)for each chunk c:  score[c] = cosine(q, vector[c])sort chunks by score, highest firstreturn the first k

#The code

Here is the retrieval step and the prompt it builds. The embed function is whatever embedding model you use; the vectors below are the toy ones from the trace.

const dot = (a, b) => a.reduce((sum, x, i) => sum + x * b[i], 0);
const cosine = (a, b) => dot(a, b) / Math.sqrt(dot(a, a) * dot(b, b));
 
function topK(queryVec, chunks, k) {
  return chunks
    .map((c) => ({ ...c, score: cosine(queryVec, c.vec) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, k);
}
 
function buildPrompt(question, passages) {
  const sources = passages.map((p, i) => `[${i + 1}] ${p.text}`).join("\n");
  return `Answer using only the sources below. Cite them like [1].
If they do not contain the answer, say "I don't know."

${sources}

Question: ${question}`;
}
 
const chunks = [
  { text: "Refunds reach your card 5 to 7 days after we receive the return.", vec: [4, 1, 0] },
  { text: "You can return items within 30 days of delivery.", vec: [3, 2, 0] },
  { text: "Standard delivery takes 2 to 4 working days.", vec: [0, 5, 1] },
  { text: "Reset your password from the sign-in page.", vec: [0, 0, 5] },
  { text: "Gift cards cannot be refunded.", vec: [2, 0, 1] },
];
const best = topK([5, 1, 0], chunks, 2);
console.log(buildPrompt("When will I get my money back?", best));

import math
 
def dot(a, b):
    return sum(x * y for x, y in zip(a, b))
 
def cosine(a, b):
    return dot(a, b) / math.sqrt(dot(a, a) * dot(b, b))
 
def top_k(query_vec, chunks, k):
    scored = [{**c, "score": cosine(query_vec, c["vec"])} for c in chunks]
    scored.sort(key=lambda c: c["score"], reverse=True)
    return scored[:k]
 
def build_prompt(question, passages):
    sources = "\n".join(f"[{i + 1}] {p['text']}" for i, p in enumerate(passages))
    return (
        "Answer using only the sources below. Cite them like [1].\n"
        'If they do not contain the answer, say "I don\'t know."\n\n'
        f"{sources}\n\nQuestion: {question}"
    )
 
chunks = [
    {"text": "Refunds reach your card 5 to 7 days after we receive the return.", "vec": [4, 1, 0]},
    {"text": "You can return items within 30 days of delivery.", "vec": [3, 2, 0]},
    {"text": "Standard delivery takes 2 to 4 working days.", "vec": [0, 5, 1]},
    {"text": "Reset your password from the sign-in page.", "vec": [0, 0, 5]},
    {"text": "Gift cards cannot be refunded.", "vec": [2, 0, 1]},
]
best = top_k([5, 1, 0], chunks, 2)
print(build_prompt("When will I get my money back?", best))

The printed prompt, sent to a model, looks like this:

The instruction "say I don't know" is not decoration. Without it, when retrieval finds nothing useful, the model falls back on its training and produces a confident, generic answer that looks just as grounded as a real one.

#When RAG is the right tool

• Use it when the material is too big to send whole, changes often, or is private. Updating the documents updates the answers, with no retraining.
• Skip it when the material is small. If the whole handbook fits comfortably, sending all of it avoids retrieval mistakes entirely.
• Do not use fine-tuning instead for facts that change. Fine-tuning is good for teaching a style or format; it is a poor way to store facts you will need to correct next month.

#Common mistakes

#Try it yourself

#Check yourself

#What to practise next

Retrieval quality starts with how documents are cut up. Chunking and Retrieval covers chunk size, overlap, and the tricks that fix the most common misses.