The claims & coverage AI benchmark
The claims track measures how language models do the work of a claims handler: read the policy and the claim file, decide whether the loss is covered, identify the clauses that control the answer, and calculate what's payable. Every case is grounded in real documents and scored pass@1 against the recorded outcome.
What the claims track tests
A coverage decision is a chain of reading. The handler establishes what happened from the claim file, finds the insuring agreement that might respond, then works through the exclusions, conditions, endorsements, and limits that modify it — and only then lands on covered or not covered, and for how much. The track asks a model to follow that chain over the actual documents and produce the determination, not a summary of the issues.
That's what makes it a true claims AI benchmark rather than a comprehension quiz: the model has to apply the policy as written, including the clauses that cut against coverage, and the answer is checked against what the file actually resolved to.
Why claims and coverage are hard for AI
Coverage turns on the interaction between the insuring agreement and everything that modifies it — the place models most often go wrong.
See how a claim moves from first notice to settlement in the claims lifecycle.
- Exclusions decide cases — cover granted by the agreement can be taken away by a later clause.
- Endorsements override the form — the model must apply the version that controls.
- Facts come from a messy file — spread across reports, correspondence, and forms.
- The number has to follow — the right deductible, limit, and payable amount on top.
Example case types
Does the exclusion apply?
Determine whether a property loss is covered given an exclusion that may or may not apply to the described cause.
Sub-limit & deductible
Apply a sub-limit and deductible to reach the amount payable on an otherwise covered claim.
Modified wording
Decide coverage where an endorsement materially changes the base policy wording.
The controlling clause
Identify the single clause that resolves a disputed liability claim.
How it's scored
Models run pass@1 — one attempt, no retries. A coverage determination is scored against the recorded covered/not-covered outcome; a payable amount against the recorded figure within a defined tolerance. Wording and rationale don't earn points; the determination does. The full rules are in the methodology. Claims is one of three families in the InsureBench insurance AI benchmark, alongside underwriting and actuarial work.