Context Engineering

You are going away for a week and a neighbour will feed your cat. You could hand them a shoebox with every vet bill, the cat food receipts and the manual for the automatic feeder you stopped using last year. Everything they need is in there somewhere. Or you could write one page: how much food and when, the pill on Tuesdays, the vet's number, and "she hides under the bed when the doorbell rings, that's normal."

The one-page note works better, even though it contains less. Building the context for a model is the same job.

#From wording to the whole screen

Prompting is about how you phrase a request. Context engineering is about everything the model reads on a call: which information goes in, in what form, in what order, and what stays out. In a chat you do it by hand when you paste in the right paragraph. In an app, code does it on every request.

You already have the two facts that make this matter. The model only knows what is in its context, and the window is limited and works best when it is focused. So the question on every call is: what is the smallest set of information that lets the model do this task well?

#Four moves

Most ways to improve context come down to four moves.

1. Select. Include only what this task needs. Leave out the shipping rates when the question is about returns.
2. Compress. Send the relevant section instead of the whole manual. Replace a long thread with a short summary of what was decided. Strip boilerplate like headers, footers and legal text that repeats on every page.
3. Order and label. Put instructions where they cannot be missed. Wrap each piece of material in a clear label saying what it is and where it came from. For long material, restate the question after it, so it is near the end where the model reads it last.
4. Isolate. Split a big job into smaller calls, each with its own focused context. A call that summarizes ten reports one by one, then a call that compares the ten summaries, often beats one call with all ten reports at once.

Just as important is what to keep out: outdated versions of a document, two sources that disagree, material that is only loosely related, secrets the model does not need, and text from untrusted sources that might contain instructions (see Prompt Injection).

#Packing a context, step by step

A customer asks a shop's assistant: "Can I return headphones I've already opened?" The app has six candidate pieces of context. Each has a relevance score (in a real system this comes from retrieval, from fixed rules, or from a quick check by another model) and a size in tokens. The budget for this material is 250 tokens.

The method: throw out anything stale first, then take the most relevant pieces that still fit, and skip anything below a minimum relevance of 0.3.

drop stale piecessort the rest by score, highest firstfor each piece:  if score < 0.3: skip  if used + tokens > budget: skip  add it, used += tokens

Unlike trimming chat history, skipping a piece and carrying on is fine here: the pieces are independent.

function selectContext(pieces, budget, minScore = 0.3) {
  const candidates = pieces
    .filter((p) => !p.stale)
    .sort((a, b) => b.score - a.score);
  const chosen = [];
  let used = 0;
  for (const p of candidates) {
    if (p.score < minScore) continue;           // not relevant enough
    if (used + p.tokens > budget) continue;     // does not fit
    chosen.push(p.name);
    used += p.tokens;
  }
  return { chosen, used };
}
 
selectContext([
  { name: "policy", score: 0.9, tokens: 120 },
  { name: "product", score: 0.6, tokens: 80 },
  { name: "order", score: 0.8, tokens: 40 },
  { name: "FAQ", score: 0.4, tokens: 300 },
  { name: "old policy", score: 0.7, tokens: 120, stale: true },
  { name: "shipping", score: 0.2, tokens: 60 },
], 250); // { chosen: ["policy", "order", "product"], used: 240 }

def select_context(pieces, budget, min_score=0.3):
    candidates = sorted(
        (p for p in pieces if not p.get("stale")),
        key=lambda p: p["score"],
        reverse=True,
    )
    chosen, used = [], 0
    for p in candidates:
        if p["score"] < min_score:          # not relevant enough
            continue
        if used + p["tokens"] > budget:     # does not fit
            continue
        chosen.append(p["name"])
        used += p["tokens"]
    return chosen, used
 
select_context([
    {"name": "policy", "score": 0.9, "tokens": 120},
    {"name": "product", "score": 0.6, "tokens": 80},
    {"name": "order", "score": 0.8, "tokens": 40},
    {"name": "FAQ", "score": 0.4, "tokens": 300},
    {"name": "old policy", "score": 0.7, "tokens": 120, "stale": True},
    {"name": "shipping", "score": 0.2, "tokens": 60},
], 250)  # (['policy', 'order', 'product'], 240)

Sorting dominates the cost: O(n log n) for n pieces, tiny next to the model call. The hard part is getting good scores and honest "stale" flags.

#What the model receives

Here is the difference between dumping material and engineering it, on the same question.

The first reply mixed the two policies and missed the in-ear rule. In the second, every fact it needed was there, labelled, and nothing contradicted it.

#Common mistakes

#Try it yourself

#Check yourself

#What to practise next

The packing example assumed relevance scores already existed. RAG Basics shows how to compute them by searching your documents with embeddings, and Chunking and Retrieval shows how to cut documents into pieces worth retrieving.