Give the Model Room to Think

Someone asks you what 17 × 24 is and wants the answer in one second. You blurt out something close, maybe 398. Give yourself a pencil and ten seconds and you write 17 × 20 = 340, 17 × 4 = 68, 340 + 68 = 408. Same brain. The difference is that the paper held the in-between steps.

A language model has a similar limit, and the same fix. The only paper it has is the text it writes.

#The text is the model's scratch paper

A model produces its reply one token at a time. To choose each token, it runs the same fixed stack of calculations over everything before it: your prompt plus whatever it has already written. That is a limited amount of work per token. Unless a tool gives it a hidden scratch area (more on that below), it has no notebook where it can work things out first and then reply.

That has a direct consequence. If the first words of the reply must be the answer, the answer is produced with no working on the page. If the reply starts with the working, each step becomes text, and every later token, including the final answer, can build on it.

#Watch it happen

Here is a small shopping problem: pens cost $9 for a pack of 12, or $1 each. What is the cheapest way to buy exactly 30 pens? Work it out yourself first. The tracer shows two illustrative replies, one asked for the price only and one asked to work it out first.

repeat:  look at the prompt and everything written so far  write the next piece

Reply A was not careless. It was asked to commit first, so it committed to the most obvious-looking number. Reply B was allowed to lay the options out, which turned a tricky question into an easy one: pick the smallest.

This does not make the model always right. Steps can still be wrong. But a wrong step written on the page is something you can see and check, which a wrong one-word answer is not.

#Separate the working from the answer

Working makes replies longer. If a person or a program only needs the result, ask for the two parts in clearly marked sections.

Wednesday fails: Ben is out. Checking each person against each slot is several small steps, and the prompt left no room for them.

The honest answer was "none", which the one-word prompt made very unlikely, because it implied one slot must work. The second prompt allowed for that case explicitly, a habit from Be Clear and Specific.

With the answer in tags, a program can ignore the working and pull out the result:

function extractAnswer(reply) {
  const match = reply.match(/<answer>([\s\S]*?)<\/answer>/);
  return match ? match[1].trim() : null; // null: the tag was missing
}
 
const reply = "<thinking>Mon 10:00 ... Out.</thinking>\n<answer>none</answer>";
console.log(extractAnswer(reply)); // "none"

import re
 
def extract_answer(reply):
    match = re.search(r"<answer>(.*?)</answer>", reply, re.DOTALL)
    return match.group(1).strip() if match else None  # None: the tag was missing
 
reply = "<thinking>Mon 10:00 ... Out.</thinking>\n<answer>none</answer>"
print(extract_answer(reply))  # none

Always handle the missing-tag case. A model usually follows the format, not always, and your code should notice instead of crashing.

#When to skip it

Room to think helps when there are steps: arithmetic, comparing options, checking rules, debugging, planning. It adds little to a lookup ("What is the capital of Peru?") or a short creative task, and it costs time and tokens on every request. For very large tasks, a different form of room helps more: split the job into several prompts, each doing one stage, and feed each result into the next.

#Try it yourself

#Common mistakes

#Check yourself

#What to practise next

The tags in this lesson were a first step toward replies a program can read. Structured Output takes that all the way to JSON you can parse and check.