Insurance and actuarial
Testing whether AI gets the actuarial number right
Much of reserving and ratemaking has an answer you can compute once the method is fixed. We build tasks around those answers, run AI models through them and score each response against a key calculated in code. It is the same approach we use to test AI tools for credit teams.
What we write tasks for
Core P&C methods with checkable results
Each area below is one where the task can state its inputs and method precisely enough that only one answer is right.
Reserving
- Chain-ladder indications from paid or reported triangles
- Bornhuetter-Ferguson IBNR from an a priori loss ratio
- Carried reserves tested for adequacy against the indication
Loss development
- Age-to-age factors, volume-weighted averages and tail factors
- Cumulative factors to ultimate
- Triangles given as incremental amounts that must be accumulated first
Trend and on-level ratemaking
- Earned premium brought to current rate level by the parallelogram method
- Losses trended between average accident dates
- Loss ratio method indications, tested against a proposed change
Credibility
- Classical (limited fluctuation) credibility and the square-root rule
- Weighting an indication with its complement
Increased limits
- Limited expected values from a ground-up claim sample
- Increased limits factors against a basic limit, compared with a filed factor
Experience rating
- Experience modifications from actual and expected losses under stated plan rules
Built so far: a starter set of 5 tasks covering chain-ladder reserving, Bornhuetter-Ferguson IBNR, a rate indication with on-level premium and loss trend, credibility weighting and an increased limits factor. Two of the five have nothing wrong, and the right answer is to say so. Experience rating tasks are not built yet. Every insurer, triangle and claim in the tasks is invented.
How the tasks work
The method is stated, so the answer is not a matter of opinion
- Method notes in every package.
Each task comes with numbered notes that fix the method: which average of age-to-age factors to select, the tail factor, the credibility standard, the adequacy threshold. Reasonable actuaries can choose differently in practice. Within the task, the notes settle it.
- Answer keys computed in code.
Expected answers come from our calculation engine, written in ordinary code with exact decimal arithmetic, before any AI model sees the task. No language model writes or grades an answer.
- An independent re-check.
A second program, written separately and sharing no code with the engine, recomputes every key. Today the two agree on all five tasks.
- Built to catch real mistakes.
In the chain-ladder task, the paid triangle is given as incremental amounts. A model that computes development factors without accumulating the triangle first gets the wrong reserve and the wrong conclusion.
- The same four-part score.
Did the model reach the right conclusion, such as adequate or deficient? Did it get the number within the stated tolerance? Did it escalate when the notes require it? Did it cite the method note that governs? Each task runs three times by default, so we also see how consistent the answers are.
Worked example
A credibility-weighted class indication
One of the five tasks. The insurer is fictional.
The task
Ashgrove Fire & Casualty is pricing commercial property class 0532. The model must compute the credibility-weighted indication, decide whether the pricing memo's selection departs from it by more than 2.0 points, decide whether to escalate, and cite the governing note.
- Claims in the experience
- 400
- Full-credibility standard
- 1,082 claims
- Observed indication
- +12.0%
- Complement (statewide, all classes)
- +4.0%
Pricing memo: “Class 0532 experience indicates +12.0%; we select +12.0%.”
The package also carries the notes on on-level premium, trend, rate adequacy and increased limits, which do not govern this question.
The answer key
- Credibility:
Z = √(400 / 1,082) = 0.608 - Weighted indication:
0.608 × 12.0% + 0.392 × 4.0% = 8.86% - The memo selects +12.0%, which gives the class full weight. That is 3.1 points above the indication, over the 2.0-point limit.
- Problem: yes. Escalate: yes.
- Governing note:
Note P4(credibility). Tolerance on the indication is ±0.05 points.
Who this is for
Anyone who needs to know how far to trust the output
Insurers and MGAs
Choosing or already using an AI tool for reserving or pricing support? Score it on fixed tasks with known answers before relying on it.
Tool builders
Find out where your model or pipeline goes wrong on actuarial work, and re-test on the same suite version after each change.
Societies and researchers
Need a benchmark for your field, built to a written specification? See custom benchmarks.
What these tests are not: the tasks are synthetic and state their method, so they measure calculation and method on stated terms, not the judgment calls actuaries make in selecting assumptions. A scorecard is not an actuarial opinion, a certification or a regulatory approval. On larger projects we build review of the task designs by credentialed actuaries into the plan.
Contact
Talk to us about actuarial AI testing
Tell us which methods or lines of business you care about and which models or tools you want tested. We will reply with a proposed scope.