What Is Context
A model only knows what it reads on each call. See what goes into that text, where it comes from, and why it decides the quality of every answer.
On this page
You call a support line about a delivery. The person who answers has never spoken to you. Everything they know about your problem comes from the notes on their screen: your order, the last agent's comments, the company's refund rules. If the notes are good, the call takes two minutes. If the last agent wrote "customer called, sorted", you start from zero.
A language model is in that position on every request. It has broad general knowledge from training, but about you and this task it knows only what is on its screen right now. That screen is called the context.
Two kinds of knowing
A model has two sources of information, and they behave very differently.
- Training. Everything it absorbed while it was built: language, facts about the world, how code usually looks. This is broad, but frozen at some date, and it contains nothing private: not your files, your company's rules or yesterday's news.
- Context. The text sent to it on this call. This is narrow, but it is current and specific, and you control it.
When an answer is generic, out of date or wrong about your situation, the cause is usually the same: the fact it needed was not in the context, so it filled the gap from training. That is the pattern behind hallucinations and behind the generic answers in Prompting Basics.
What goes into the context
When you type a message into an AI app, your message is only one piece of what the model reads. The app assembles a larger block of text around it.
- Instructions from whoever built the app: the role, the rules, the tone. See System Prompts.
- Tool descriptions, if the model can search, run code or call other services. Covered in Tools and MCP.
- Retrieved documents: pages the app looked up because they seemed relevant to your question. That is RAG.
- Saved notes about you or the project, carried over from earlier sessions. That is memory.
- The conversation so far: every earlier message and reply in this chat.
- Your new message, plus anything you pasted or attached.
From the model's side there are no separate channels. It is all one long piece of text, read top to bottom, with markers saying who wrote what. Everything in this module is about choosing those pieces well.
Watch a chat being resent
The model keeps nothing between calls. A chat feels continuous because the app sends the whole conversation again every time. Step through two turns of a bakery chat, then see what happens in a fresh chat.
request = [system]request += historyrequest += [new message]reply = model(request)history += [new message, reply]- request0system: bakery ordershistory (kept by the app)
- turn
- 1
1Turn 1. The app starts every request with its instructions. The history list is empty.
- request0system: bakery orders1user: Lena, 12 rolls, Frihistory (kept by the app)
- turn
- 1
2Nothing to copy from history, so the only other piece is the new message: Lena says she ordered 12 rolls for Friday.
- request0system: bakery orders1user: Lena, 12 rolls, Frihistory (kept by the app)
- turn
- 1
- reply
- noted
3The model reads both pieces and writes a reply: "Noted: 12 rolls, Friday."
- request0system: bakery orders1user: Lena, 12 rolls, Frihistory (kept by the app)0user: Lena, 12 rolls, Fri1model: noted
- turn
- 1
4The app saves the message and the reply in its history list. The model itself saves nothing.
- request0system: bakery orders1user: Lena, 12 rolls, Fri2model: notedhistory (kept by the app)0user: Lena, 12 rolls, Fri1model: noted
- turn
- 2
5Turn 2. Lena types "Can I make it 20?" The app rebuilds the request from scratch: instructions, then both saved messages.
- request0system: bakery orders1user: Lena, 12 rolls, Fri2model: noted3user: make it 20?history (kept by the app)0user: Lena, 12 rolls, Fri1model: noted
- turn
- 2
6Then the new message. "It" can only mean the rolls, because turn 1 is right there in the text the model reads.
- request0system: bakery orders1user: Lena, 12 rolls, Fri2model: noted3user: make it 20?history (kept by the app)0user: Lena, 12 rolls, Fri1model: noted
- turn
- 2
- reply
- 20 rolls
7The model answers "20 rolls for Friday, done." It read four messages to write one short reply. The app then saves this turn too, and history grows to 4.
- request0system: bakery orders1user: make it 20?history (kept by the app)
- turn
- 1
8Now Lena opens a new chat and types the same thing. History is empty, so the request has only two pieces. "It" refers to nothing, and the model has to ask, or worse, guess.
Same model, same question, different context, different answer. The model did not "forget" Lena in the new chat. It never remembered her; it only ever read her.
The same thing in code
Here is the app's side of that tracer. The history list is the only memory there is, and it lives in the app.
// The app keeps the conversation. The model keeps nothing.
function buildRequest(system, history, newMessage) {
return [
{ role: "system", content: system },
...history,
{ role: "user", content: newMessage },
];
}
const system = "You take orders for a small bakery.";
const history = [];
const first = buildRequest(system, history, "I'm Lena. 12 rolls for Friday.");
history.push(first.at(-1), { role: "assistant", content: "Noted: 12 rolls, Friday." });
const second = buildRequest(system, history, "Can I make it 20?");
console.log(second.length); // 4: system, two saved turns, new message
const freshChat = buildRequest(system, [], "Can I make it 20?");
console.log(freshChat.length); // 2: nothing says what "it" is# The app keeps the conversation. The model keeps nothing.
def build_request(system, history, new_message):
return (
[{"role": "system", "content": system}]
+ history
+ [{"role": "user", "content": new_message}]
)
system = "You take orders for a small bakery."
history = []
first = build_request(system, history, "I'm Lena. 12 rolls for Friday.")
history += [first[-1], {"role": "assistant", "content": "Noted: 12 rolls, Friday."}]
second = build_request(system, history, "Can I make it 20?")
print(len(second)) # 4: system, two saved turns, new message
fresh_chat = build_request(system, [], "Can I make it 20?")
print(len(fresh_chat)) # 2: nothing says what "it" isReal apps send the list through an API instead of printing its length, but the shape is the same: a list of messages, rebuilt and resent on every call.
Context beats cleverness
Compare two ways of asking the same question. The wording barely changes. What changes is the information on the screen.
The first reply is not a bad model. It is an honest model with nothing to work with.
Common mistakes
Try it yourself
Someone asks an assistant: "Should I accept this meeting invite?" List every piece of information the model would need in its context to give a useful answer, and where each piece could come from (the user, a calendar, a document, earlier conversation).
Show a sample answer
One reasonable list:
- The invite itself: who sent it, topic, time, length. From the user pasting it, or a calendar tool.
- The user's calendar that day: is there a clash? From a calendar tool.
- The user's priorities: "I'm protecting mornings for focus work this month." From the user or saved notes.
- Who the sender is to them: their manager, a vendor, a stranger. From the user or a contacts lookup.
- Whether they are actually needed: the agenda, and whether a colleague is already going. From the invite or the user.
Without these, the only possible answer is generic advice about meetings. Notice that most of the useful facts are not in the question at all. That is normal: the question is usually the smallest part of a good context.
Check yourself
In a long chat, the assistant correctly refers to something you said 10 messages ago. Why can it do that?
An assistant gives an outdated answer about your company's holiday policy. What is the most likely cause?
Which of these does the model not see unless the app adds it to the context?
What to practise next
Context is not unlimited. The Context Window shows how much fits, what happens when it overflows, and why more is not always better. Then Context Engineering turns all of this into a method.