Every check passed. The discount was still wrong: building AI agents that make the right call

A typed contract can force an AI agent to answer qualify, do_not_qualify, or review. It can’t tell you what “qualify” means. I learned the difference building an auto insurance MGA. Our telematics discount depends on questions like how many weeks of trips count as enough driving data, and what a hard-braking event actually is. Telematics engineers define it as a deceleration rate. Claims teams define it by how well it predicts a loss. Both definitions are correct, and an agent sitting between them can pass every check and still price the discount wrong. This talk follows one discount decision through the 4 layers we use. Ontology sets what words mean. Policy draws the decision boundary, with thresholds tied to consequence and reversibility, because a 0.62 and a 0.97 shouldn’t trigger the same action. Typed decisioning limits what the agent can return. The harness decides what happens next and catches what slips through. I’ll also cover the failures I worry about most: an agent that’s accurate on average and unfair to one segment, and a review queue where humans are still in the loop but no longer deciding. Everything here comes from systems running in production.
Who is this presentation for?
Engineers and data scientists shipping AI agents, and the leaders who own what those agents decide. You’ll get the most from it if your agents make calls about customers in insurance, lending, healthcare, or hiring.
Prerequisite knowledge:
Some hands-on exposure to LLM-based agents, including how they call tools and return structured outputs. It also helps to know what a model confidence score is. No insurance background needed; I’ll walk through the telematics example from scratch.
What you'll learn?
1) A 2-minute test for whether your guardrails check meaning or just format 2) Why a typed contract is narrower than a definition, and what has to sit beneath it 3) How to catch 2 correct definitions that disagree at a system boundary, using an owned, versioned translation contract 4) How to set thresholds by consequence and reversibility rather than a single confidence cutoff 5) Why fairness testing and accuracy testing catch different bugs 6) How to tell when your human-in-the-loop has quietly stopped deciding