What Is Context

A model only knows what it reads on each call. See what goes into that text, where it comes from, and why it decides the quality of every answer.

12 minLesson 1 of 8
On this page
  1. Two kinds of knowing
  2. What goes into the context
  3. Watch a chat being resent
  4. The same thing in code
  5. Context beats cleverness
  6. Common mistakes
  7. Try it yourself
  8. Check yourself
  9. What to practise next

You call a support line about a delivery. The person who answers has never spoken to you. Everything they know about your problem comes from the notes on their screen: your order, the last agent's comments, the company's refund rules. If the notes are good, the call takes two minutes. If the last agent wrote "customer called, sorted", you start from zero.

A language model is in that position on every request. It has broad general knowledge from training, but about you and this task it knows only what is on its screen right now. That screen is called the context.

Two kinds of knowing

A model has two sources of information, and they behave very differently.

  • Training. Everything it absorbed while it was built: language, facts about the world, how code usually looks. This is broad, but frozen at some date, and it contains nothing private: not your files, your company's rules or yesterday's news.
  • Context. The text sent to it on this call. This is narrow, but it is current and specific, and you control it.

When an answer is generic, out of date or wrong about your situation, the cause is usually the same: the fact it needed was not in the context, so it filled the gap from training. That is the pattern behind hallucinations and behind the generic answers in Prompting Basics.

What goes into the context

When you type a message into an AI app, your message is only one piece of what the model reads. The app assembles a larger block of text around it.

the context instructions (system prompt) tool descriptions retrieved documents saved notes / memory conversation so far your new message model reply training knowledge general, frozen
One request. The app stacks several sources into a single block of text; the model reads all of it, then writes the reply.
  • Instructions from whoever built the app: the role, the rules, the tone. See System Prompts.
  • Tool descriptions, if the model can search, run code or call other services. Covered in Tools and MCP.
  • Retrieved documents: pages the app looked up because they seemed relevant to your question. That is RAG.
  • Saved notes about you or the project, carried over from earlier sessions. That is memory.
  • The conversation so far: every earlier message and reply in this chat.
  • Your new message, plus anything you pasted or attached.

From the model's side there are no separate channels. It is all one long piece of text, read top to bottom, with markers saying who wrote what. Everything in this module is about choosing those pieces well.

Watch a chat being resent

The model keeps nothing between calls. A chat feels continuous because the app sends the whole conversation again every time. Step through two turns of a bakery chat, then see what happens in a fresh chat.

What the model reads on each callUse the arrows or ← → keys
request = [system]request += historyrequest += [new message]reply = model(request)history += [new message, reply]
  1. request
    0system: bakery orders
    history (kept by the app)
    turn
    1

    1Turn 1. The app starts every request with its instructions. The history list is empty.

  2. request
    0system: bakery orders
    1user: Lena, 12 rolls, Fri
    history (kept by the app)
    turn
    1

    2Nothing to copy from history, so the only other piece is the new message: Lena says she ordered 12 rolls for Friday.

  3. request
    0system: bakery orders
    1user: Lena, 12 rolls, Fri
    history (kept by the app)
    turn
    1
    reply
    noted

    3The model reads both pieces and writes a reply: "Noted: 12 rolls, Friday."

  4. request
    0system: bakery orders
    1user: Lena, 12 rolls, Fri
    history (kept by the app)
    0user: Lena, 12 rolls, Fri
    1model: noted
    turn
    1

    4The app saves the message and the reply in its history list. The model itself saves nothing.

  5. request
    0system: bakery orders
    1user: Lena, 12 rolls, Fri
    2model: noted
    history (kept by the app)
    0user: Lena, 12 rolls, Fri
    1model: noted
    turn
    2

    5Turn 2. Lena types "Can I make it 20?" The app rebuilds the request from scratch: instructions, then both saved messages.

  6. request
    0system: bakery orders
    1user: Lena, 12 rolls, Fri
    2model: noted
    3user: make it 20?
    history (kept by the app)
    0user: Lena, 12 rolls, Fri
    1model: noted
    turn
    2

    6Then the new message. "It" can only mean the rolls, because turn 1 is right there in the text the model reads.

  7. request
    0system: bakery orders
    1user: Lena, 12 rolls, Fri
    2model: noted
    3user: make it 20?
    history (kept by the app)
    0user: Lena, 12 rolls, Fri
    1model: noted
    turn
    2
    reply
    20 rolls

    7The model answers "20 rolls for Friday, done." It read four messages to write one short reply. The app then saves this turn too, and history grows to 4.

  8. request
    0system: bakery orders
    1user: make it 20?
    history (kept by the app)
    turn
    1

    8Now Lena opens a new chat and types the same thing. History is empty, so the request has only two pieces. "It" refers to nothing, and the model has to ask, or worse, guess.

1 / 8

Same model, same question, different context, different answer. The model did not "forget" Lena in the new chat. It never remembered her; it only ever read her.

The same thing in code

Here is the app's side of that tracer. The history list is the only memory there is, and it lives in the app.

// The app keeps the conversation. The model keeps nothing.
function buildRequest(system, history, newMessage) {
  return [
    { role: "system", content: system },
    ...history,
    { role: "user", content: newMessage },
  ];
}
 
const system = "You take orders for a small bakery.";
const history = [];
 
const first = buildRequest(system, history, "I'm Lena. 12 rolls for Friday.");
history.push(first.at(-1), { role: "assistant", content: "Noted: 12 rolls, Friday." });
 
const second = buildRequest(system, history, "Can I make it 20?");
console.log(second.length); // 4: system, two saved turns, new message
 
const freshChat = buildRequest(system, [], "Can I make it 20?");
console.log(freshChat.length); // 2: nothing says what "it" is
# The app keeps the conversation. The model keeps nothing.
def build_request(system, history, new_message):
    return (
        [{"role": "system", "content": system}]
        + history
        + [{"role": "user", "content": new_message}]
    )
 
system = "You take orders for a small bakery."
history = []
 
first = build_request(system, history, "I'm Lena. 12 rolls for Friday.")
history += [first[-1], {"role": "assistant", "content": "Noted: 12 rolls, Friday."}]
 
second = build_request(system, history, "Can I make it 20?")
print(len(second))  # 4: system, two saved turns, new message
 
fresh_chat = build_request(system, [], "Can I make it 20?")
print(len(fresh_chat))  # 2: nothing says what "it" is

Real apps send the list through an API instead of printing its length, but the shape is the same: a list of messages, rebuilt and resent on every call.

Context beats cleverness

Compare two ways of asking the same question. The wording barely changes. What changes is the information on the screen.

Missing contextIllustrative example. Real replies vary.
You
Is my flight affected by the strike?
Model
I don't have information about your specific flight. Strikes can affect airlines, airports or air traffic control. I'd recommend checking your airline's website or app for updates on your booking.
Context suppliedIllustrative example. Real replies vary.
You
Is my flight affected by the strike? Answer from the notice below only. My flight: leaves Thursday 07:40 from Terminal 2. <notice> Ground staff at Terminal 2 will strike from 06:00 to 10:00 on Thursday. Flights in that window may be delayed by up to 3 hours. Terminal 1 is not affected. </notice>
Model
Yes, probably. Your flight leaves at 07:40 from Terminal 2, which falls inside the 06:00 to 10:00 strike window. The notice says flights in that window may be delayed by up to 3 hours.

The first reply is not a bad model. It is an honest model with nothing to work with.

Common mistakes

Try it yourself

Try it: What would it need to know

Someone asks an assistant: "Should I accept this meeting invite?" List every piece of information the model would need in its context to give a useful answer, and where each piece could come from (the user, a calendar, a document, earlier conversation).

Show a sample answer

One reasonable list:

  • The invite itself: who sent it, topic, time, length. From the user pasting it, or a calendar tool.
  • The user's calendar that day: is there a clash? From a calendar tool.
  • The user's priorities: "I'm protecting mornings for focus work this month." From the user or saved notes.
  • Who the sender is to them: their manager, a vendor, a stranger. From the user or a contacts lookup.
  • Whether they are actually needed: the agenda, and whether a colleague is already going. From the invite or the user.

Without these, the only possible answer is generic advice about meetings. Notice that most of the useful facts are not in the question at all. That is normal: the question is usually the smallest part of a good context.

Check yourself

Quick check

In a long chat, the assistant correctly refers to something you said 10 messages ago. Why can it do that?

An assistant gives an outdated answer about your company's holiday policy. What is the most likely cause?

Which of these does the model not see unless the app adds it to the context?

What to practise next

Context is not unlimited. The Context Window shows how much fits, what happens when it overflows, and why more is not always better. Then Context Engineering turns all of this into a method.