Guardrails and Requirements
By the end of this module
- Separate an obligation written in a prompt from a mechanism that enforces one
- Place a check at the right hook point for what it needs to see and do
- Judge when a control should block and when it should only watch
Definition
Every team has a list of things their agent must never do. The list is usually real, usually agreed, and usually written in a prompt. That last part is the subject of this module.
A requirement is an obligation met by instruction. The agent is told, and nothing enforces it. A control is an intervention on the loop: code that runs at a defined point and can stop, change or record what passes. A guardrail is the policy itself, which may be implemented either way.
This is the binding distinction from unit three wearing different clothes. An obligation the model is asked to honour is discretionary. An obligation checked by code at a hook is edge-bound. Both appear in a design document as a bullet point beginning "must".
Three Things With One Name
| Requirement | Control | |
|---|---|---|
| How it is met | By telling the agent, in its instructions. | By code that runs whether the agent cooperates or not. |
| What enforces it | Nothing. | The runtime, at a defined hook point. |
| How it fails | Quietly, on a run that looks like every other run. | Loudly, and the run gets a guardrail exit. |
| Cost | Context budget, and nothing else. | Latency if it blocks, engineering either way. |
| What it is good for | Obligations no mechanism can judge: tone, pitching an explanation, knowing when to stop. | Obligations with a decidable answer: amounts, permissions, destinations, formats. |
| How you know it holds | Sampling, which tells you about the runs you sampled. | By construction, which tells you about the runs you have not seen. |
Requirements are not a failure of engineering. Plenty of obligations cannot be checked by code, because judging them needs the judgement the model is there to provide. Tone, helpfulness, not being condescending: no mechanism catches these, so they are instructed and sampled. The mistake is not having requirements. It is not knowing which of your obligations are only requirements.
The Hook Plane
A control does not sit in the flow. It attaches beside a stage, at one of eight points, and each point differs in what it can see and what it can still do about it. Click through them.
Position decides capability. A check that needs to see the model's complete answer cannot also stop the run before the model was called, and a check that wants to stop something cheaply has to run before the expense, when it knows least. Most arguments about guardrails are really arguments about this trade, held without naming it.
Drawing a control as a box in the flow is the commonest mistake in a hand-made agent diagram, and it is not cosmetic. A check drawn as a step looks like something the agent passes through, which invites implementing it as a tool the agent may call, which makes it discretionary, which means it is not a control at all.
Blocking and Watching
The second property of a control is whether it is in the way.
| Mode | Behaviour | When it is right |
|---|---|---|
| Blocking | The run waits for the check, and the check can stop it. | The thing being prevented is worse than the delay. Anything with side effects or money. |
| Streaming | Runs against output as it is produced, and can halt mid-reply. | The answer is streamed to a person and part of it has already been read. |
| Shadow | Sees everything, delays nothing, prevents nothing. | Measuring how often something happens before deciding whether to block it. |
The honest version of most guardrail conversations is this: blocking controls are the ones that work and the ones that cost. A shadow control that has been firing on 4% of runs for six months has prevented nothing. It has told you that 4% of runs need a blocking control, which is worth knowing and is not the same as having one.
A policy says the agent must never state a specific refund amount. It is written in the system prompt, and in six months of sampled transcripts it has never been violated. Is it a control?
Discussion
Take your team's list of things the agent must never do. Mark each one as instructed or enforced. Then, for every item that is only instructed, say what a violation would cost and who would be the first to know. The items where nobody would find out are the ones worth the meeting.