Choosing between schema-constrained LLM outputs and Jev's Choice primitive depends on whether an application just needs a typed answer or also needs a probability distribution over possible answers. JSON mode only guarantees valid JSON; schema-constrained structured outputs additionally restrict the response to a declared shape with an enum like billing, technical, or other. Jev's Choice primitive returns a selected option plus a probability for every option, and a derived confidence score. The piece walks through a support-ticket routing example, including a policy function that only auto-routes when probability exceeds 0.9, and argues teams should measure automatic coverage alongside accuracy rather than comparing raw confidence values between systems.
Table of contents
JSON mode and structured outputs solve different problemsA correct shape can contain a wrong decisionWhat Jev adds to the contractA generated confidence field is a separate design choicePut the action policy in ordinary codeWhen I would keep the existing LLMWhen I would evaluate a decision modelCompare coverage as well as accuracyMigration can start in the backgroundQuestions this post answers
What's the difference between JSON mode and schema-constrained structured outputs in LLM APIs?
JSON mode only guarantees a model produces valid, parseable JSON, with no guarantee about which properties or values appear. Schema-constrained structured outputs additionally restrict the response to a declared shape, such as an enum of allowed department values, within the provider's supported schema features. A schema still doesn't define the business meaning behind categories, so prompts must explain what each category means. Teams deciding between JSON mode and schema validation for LLM routing can compare approaches like this on daily.dev.
How does Jev's Choice primitive calculate a confidence score?
Jev's Choice confidence is calculated from the probabilities it returns for every offered option, measuring how far the top option's probability sits above an even split, adjusted for the number of options available. This differs from an arbitrary numeric confidence field added to a generic LLM schema, which only constrains the output shape without defining what the number measures against real mistakes. Anyone evaluating probability-based routing models against plain LLM confidence fields can dig into comparisons like this on daily.dev.
How should I measure whether an automatic routing model is good enough to deploy?
Measure automatic coverage (how many cases the system handles without review) alongside accuracy, since reporting correctness alone can hide a system that only handles easy messages. Compare two approaches at the same acceptable error rate or the same coverage level, keep a held-out test set separate from the examples used to tune prompts and thresholds, and include validation and retry time in end-to-end latency measurements. Developers designing evaluation criteria for automated decision systems can find workflows like this on daily.dev.
Share this post