Decisions Without Text
By the end of this module
- Describe what a decision model returns and how it differs from a constrained LLM reply
- Read a calibrated probability correctly, including what it does not tell you
- Map probabilities onto an action, and say who chose the thresholds
Definition
Module three left a ticket being classified by a model that sometimes sent it to the wrong team. The first response on that list was to constrain the decision, and there is now a class of model built to do exactly that.
A decision model takes some state and a set of typed questions, and returns a probability for each answer the caller supplied. It generates no text. TypeSafe AI named the class System One models when it launched Jev in September 2026, after Kahneman's split between fast judgement and slow deliberation. AWS followed with Strands Decider 2B, which runs locally. The knowledge base covers both in more depth on Decision Models.
The Three Question Types
Below is the same ticket from module three, with three questions asked about it at once. Ask it a few times and watch what does not happen: there is no typing, no sentence, and no wait.
Ticket: My payouts have been failing for 3 days and I can't see why in the dashboard. Account: merchant, 14 months old, no prior tickets. System: payouts failed 14, 15 and 16 October. Bank mandate expired 13 October.
All three answers come back from one call, in one pass. There is nothing to parse and no way to get an answer that was not on the list.
Compare that with the classifier in module three. That model could reply "Billing, though it could reasonably be Technical", which is a sentence a parser has to cope with. Here the answer space was fixed by the caller, so probability exists only on the three options and the shape of the reply cannot vary. The format is guaranteed. What is in it is still a judgement.
Calibration
The number next to each option is the part worth slowing down on, because it means something narrower than people assume.
A calibrated 0.90 means: across many cases where this model said 0.90, it was right about nine times out of ten. It is a claim about a population of decisions, not about this one. It does not mean the answer is 90% correct, and it does not mean the model understood the ticket. Strands Decider reports an expected calibration error of 0.052, which is what makes a threshold worth setting at all.
This is the property an ordinary model does not offer. Token probabilities, where a provider exposes them at all, are usually overconfident, and a model asked to rate its own confidence is writing a plausible number rather than reporting one.
Gating a Tool Call
A decision that costs a tenth of a second can sit on every step, where an LLM judge would add seconds to the thing it guards. The clearest use is checking an agent's work before it acts.
The scenario below is from the Strands Decider repository. An agent has a weather tool and has been told to be eager. Asked "What is the weather?" with no location, it guesses a city. Move the three probabilities and watch which rule fires.
get_weather(city: "San Francisco")
The user asked "What is the weather?" and named no place. The agent was told to be eager, so it filled the argument in by itself.
- Hard to undo at 0.70 or above, and arguments grounded below 0.50Deny
- Hard to undo at 0.70 or aboveConfirm
- arrow_rightArguments grounded below 0.50Guide
- Premature at 0.60 or aboveGuide
- OtherwiseProceed
The model gets its turn back and asks which city was meant. Denying here would have failed the turn over something a question fixes.
The questions, the thresholds and the order of the rules were all chosen by hand. Nothing in the model supplies them, and a different team would pick differently.
Two things in that panel matter more than the model. The first is that an ungrounded argument returns Guide rather than Deny: the model gets its turn back with feedback and asks which city was meant, instead of the turn failing over something a question fixes. The second is that every threshold and the order of the rules were written by a person. The model supplies numbers. The policy is yours, and it is where the actual decisions about risk live.
Limits
A typed output guarantees the shape of the answer. It guarantees nothing about whether the answer is right, and vendor claims about hallucination refer only to the first.
| Limit | What it means in practice |
|---|---|
| No rationale | The output is a number. Where an audit needs the reasoning behind a decision, a bare confidence does not meet the requirement. |
| Closed answer space | If the right answer is missing from the options, the model picks the nearest wrong one, sometimes with high confidence. |
| Shallow reasoning | One pass is weaker than step-by-step work on multi-hop questions, arithmetic and date ordering. Decider gets about half of the hard benchmark tier right. |
| Calibration scope | Confidence is fitted on short classification tasks. On long documents or unfamiliar rubrics it is less trustworthy than the headline figure suggests. |
| Adversarial input | State written to steer the decision moves the probabilities like any other input, so a model used as a guardrail is itself an attack surface. |
| Version drift | Thresholds tuned against one version need rechecking after an upgrade, because the format is fixed but the numbers are not. |
A decision model returns 0.91 that a refund request is within policy. What have you actually been told?
Discussion
Find a decision in your team's work that happens often, has an answer space you could write down in advance, and currently either waits on a person or on a slow model call. That is the shape that fits. Then ask the harder question: who would own the thresholds?