Show Examples

A new teammate asks how your team writes commit messages. You could explain the rules: verb first, under 60 characters, ticket number in brackets at the end, no full stop. Or you could paste three good ones from last week. Most people get it faster from the three examples, and they also pick up rules you forgot to mention.

Models work the same way. When the thing you want is easier to recognise than to describe, such as a format, a tone or a labelling style, show it.

#Why examples work so well

A language model writes by continuing text in the most likely way, as What Is an LLM describes. If your prompt contains three pairs of input and output, and then a fourth input with the output left blank, the most likely continuation is an output that matches the pattern of the first three.

So one example carries a lot of information. It shows the format, the length, the vocabulary and how to judge a borderline case, without a single extra rule. Using examples this way is often called few-shot prompting; a prompt with no examples is zero-shot.

#Before and after

Say you want to tag app store reviews so you can count them in a spreadsheet. Here is the zero-shot version.

Every line is a reasonable answer, and none of them can go in a spreadsheet. The order of fields changes, topics are invented on the fly, one line has two topics, and one has a commentary. Now the same request with three examples.

The format is now identical on every line, and it picked up details nobody wrote as rules: lowercase, hyphens in multi-word topics, exactly one topic each. The examples also showed how to handle a mixed review.

Look at line 3, though. general was never in an example. With no topic that fit, the model made one up in the right format. Examples show the pattern; they do not set its limits. If only certain values are allowed, list them in the instruction too: "Topics: performance, bug, feature-request, other."

#Choosing good examples

The model learns from everything your examples have in common, including things you did not mean to teach. So pick them on purpose:

• Vary them. Different lengths, different topics, different answers. If all three example reviews are negative, the model leans negative.
• Include a tricky case. One example that shows the edge, like a mixed review or an input that fits no category, teaches more than three easy ones.
• Keep them clearly separate. Wrap each in tags like <example> so the model does not mistake an example for the real input.
• Use a few. Two to five is usually enough. Past that, each new example adds little and makes the prompt longer to read and to run.

#Keep examples as data

When you use the same prompt in a program, store the examples as data and build the prompt from them. Adding or swapping an example then means changing one list entry, and every example gets exactly the same formatting.

function buildFewShotPrompt(instruction, examples, newInput) {
  const parts = [instruction, ""];
  for (const ex of examples) {
    parts.push("<example>");
    parts.push(`Review: "${ex.input}"`);
    parts.push(`Tags: ${ex.output}`);
    parts.push("</example>");
  }
  parts.push("", `Review: "${newInput}"`, "Tags:");
  return parts.join("\n");
}
 
const examples = [
  { input: "Search is fast now, finally.", output: "topic=performance; sentiment=positive" },
  { input: "Can't attach photos anymore.", output: "topic=bug; sentiment=negative" },
];
console.log(buildFewShotPrompt("Tag the review. Copy the format exactly.",
  examples, "Battery drain is terrible."));

def build_few_shot_prompt(instruction, examples, new_input):
    parts = [instruction, ""]
    for ex in examples:
        parts.append("<example>")
        parts.append(f'Review: "{ex["input"]}"')
        parts.append(f'Tags: {ex["output"]}')
        parts.append("</example>")
    parts += ["", f'Review: "{new_input}"', "Tags:"]
    return "\n".join(parts)
 
examples = [
    {"input": "Search is fast now, finally.", "output": "topic=performance; sentiment=positive"},
    {"input": "Can't attach photos anymore.", "output": "topic=bug; sentiment=negative"},
]
print(build_few_shot_prompt("Tag the review. Copy the format exactly.",
                            examples, "Battery drain is terrible."))

The prompt ends with Tags: and nothing after it. That last line is the blank in the figure: the natural thing to write next is one line of tags in the same shape.

#Try it yourself

#Common mistakes

#Check yourself

#What to practise next

Examples fix the shape of an answer. When the answer depends on working something out, the shape is not enough: continue with Give the Model Room to Think. If you need replies a program can read, Structured Output builds on the formatting ideas here.