Guardrails

A developer asks a coding agent to clear out the project's temporary folder. The instructions say, in capitals, "BE CAREFUL. ONLY DELETE FILES IN tmp/." The agent builds the path from two pieces, one of them is empty, and the delete command runs on the project folder itself. A week of uncommitted work is gone.

The instruction was clear. The model even agreed with it. But an instruction is a request, and the delete tool accepted any path it was given. A guardrail would have been a delete tool that refuses any path outside tmp/, no matter who asks or why.

#The idea: limits the model cannot talk its way past

Everything a model writes, including tool requests, comes from predicting text. It can misread, be tricked by text in a tool result, or follow a plausible but wrong chain of reasoning. You cannot make that chance zero by writing better instructions.

So put the limits in code, outside the model, where they hold no matter what the model writes. Think of them in four layers, from the outside in:

1. Permissions. Which tools exist at all, and what the credentials behind them can reach.
2. Tool limits. What each tool accepts: which paths, which amounts, which recipients.
3. Human approval. Which actions wait for a person to say yes.
4. Untrusted input. How much you let text from tool results steer later actions.

#Permissions: give the least that works

The safest tool is the one the agent does not have. An agent that answers questions about orders needs to read orders. It does not need to delete customers, so that tool should not be on its list, and the database account behind it should be read-only. Then even an attack that fully fools the model has nothing to call.

The same goes for credentials. If the agent's API key can do everything your admin account can, every bug is an admin bug. Give the agent its own key, scoped to the few things its tools need, so you can also see in the logs which actions came from it.

#Tool limits: check the arguments

Allowing a tool is not all or nothing. Tool Use showed tools that reject bad input so the model can fix it. Guardrails use the same place to reject dangerous input:

• A file tool only accepts paths inside the project's tmp/ folder, after resolving .. and links.
• A refund tool accepts amounts up to the order's value, never more.
• An email tool only sends to addresses already on the customer's record.
• A code-running tool runs inside a sandbox with no network and no access to your files.

Add rate limits too: no more than 10 refunds an hour, say. A rate limit does not stop one bad action, but it stops a confused agent from repeating it a hundred times.

#Human approval for risky actions

Sort every tool by what happens if it is used wrongly:

Here is a policy gate doing that sorting for a support agent. Every tool request passes through it before anything runs.

rule = policy[call.tool]if no rule: blockif rule is read: allowif money and amount <= autoLimit: allowotherwise: ask a person

#The gate in code

The gate sits in the agent loop where the tool used to run directly. askHuman shows the request to a person and returns their yes or no.

const policy = {
  getOrder: { risk: "read" },
  refundOrder: { risk: "money", autoLimit: 50 },
  sendEmail: { risk: "external" },
};
 
function checkAction(call) {
  const rule = policy[call.tool];
  if (!rule) return { decision: "block", reason: `${call.tool} is not on the allowed list` };
  if (rule.risk === "read") return { decision: "allow" };
  if (rule.risk === "money" && call.input.amount <= rule.autoLimit) {
    return { decision: "allow" };
  }
  return { decision: "ask" };
}
 
function guardedRun(call, runTool, askHuman) {
  const { decision, reason } = checkAction(call);
  if (decision === "block") return { error: `Blocked: ${reason}.` };
  if (decision === "ask" && !askHuman(call)) {
    return { error: "A person declined this action. Do not retry it." };
  }
  return runTool(call);
}

policy = {
    "get_order": {"risk": "read"},
    "refund_order": {"risk": "money", "auto_limit": 50},
    "send_email": {"risk": "external"},
}
 
def check_action(call):
    rule = policy.get(call["tool"])
    if rule is None:
        return {"decision": "block", "reason": f"{call['tool']} is not on the allowed list"}
    if rule["risk"] == "read":
        return {"decision": "allow"}
    if rule["risk"] == "money" and call["input"]["amount"] <= rule["auto_limit"]:
        return {"decision": "allow"}
    return {"decision": "ask"}
 
def guarded_run(call, run_tool, ask_human):
    check = check_action(call)
    if check["decision"] == "block":
        return {"error": f"Blocked: {check['reason']}."}
    if check["decision"] == "ask" and not ask_human(call):
        return {"error": "A person declined this action. Do not retry it."}
    return run_tool(call)

Two details matter. Unknown tools are blocked by default: the policy lists what is allowed, not what is forbidden, so a new tool is denied until someone decides. And a block or a refusal goes back to the model as an error it can read, so it can tell the user instead of retrying.

#Text from tools is untrusted

The sendEmail request in the tracer did not come from the user. It came from text inside a customer message the agent had read with a tool. Prompt Injection covers how this works. For guardrails, the lesson is this: once an agent has read text you do not control, such as web pages, emails, documents or another system's output, treat its next actions as possibly steered by whoever wrote that text.

In practice, you tighten the gate after untrusted reads. An agent that has just read a web page should not send messages or touch private data without approval, even if those tools run freely at other times. The gate is where you keep apart the dangerous trio from that lesson: untrusted text, private data and a way to send data out.

#Try it yourself

#Common mistakes

#Check yourself

#What to practise next

Take an agent or AI feature you use and list its tools with the worst thing each could do. Then add those worst cases to your evals, and read Privacy and Security for what to keep out of AI tools in the first place.