AI has moved quickly into healthcare. Health plans, provider groups, and the teams that support them are all exploring what it can do, and the first experience is usually impressive: type a question in plain English and get a number back in seconds.
The next question is whether that number is right, and whether a CFO or actuary can act on it. In July, I wrote that AI is only as good as the data underneath it. That still holds. But after a year and a half of building AI for actuarial work, I've found clean data is only the first requirement. Here's what else it takes, and what I'd ask before trusting any AI with a contract's numbers.
Why everyone wants AI to do the analysis
Most organizations managing risk contracts already have plenty of data and plenty of reports. What they don't have is time or insights they can act on. People spend hours finding the right report, applying filters, and pulling numbers, then realize they're looking at the wrong cut of the data and start over. The effort goes into finding information instead of deciding what to do with it.
There’s a second problem that gets less attention: the expertise these questions need is usually spread across teams. Analytics teams often don’t have a clinical background. Clinical teams often don’t work with the data directly. And the contract knowledge sits with actuarial and finance. A question like “Which members are driving avoidable cost?” needs all three, so it either gets passed between teams or never gets asked.
One provider organization we work with recently ran an analysis they haven’t been able to get done in six years. Their analytics team didn’t have the clinical background, and its clinical team didn’t have the analytics.
That’s the real promise of AI in this field. It can cut out many of the manual steps in analysis and reporting and bring clinical, analytical, and actuarial knowledge into the same analysis, so people get to a decision faster. But it only helps if the answers hold up, and actuarial questions are harder than they look.
Simple questions aren't simple
Here’s a simplified example that shows the problem. Take PMPM, or cost per member per month. It sounds easy to calculate: claims divided by member months. Then the questions start:
- Paid dollars or allowed dollars?
- Medical claims only, or medical and pharmacy?
- PMPM for the whole population, or for a group of patients with a certain condition?
For a condition group, the number is really per patient per month (PPPM), even though most people still call it PMPM, so you have to work out what the person asking actually means.
- Do they want the group's own cost, or the group's impact on the total?
- What's the measurement period, and how much claims runout is in it?
- Is it being compared to last year, to a benchmark, or to the rest of the book?
None of that is usually in the question. Someone has to make a judgment call on each one, and an AI that doesn't know to ask will pick an answer without telling you.
Risk contracts multiply the problem. Every contract has its own attribution rules, exclusions, stop-loss thresholds, and carve-outs, and each one has to be interpreted. Unless those choices all tie back to a single financial source of truth, two people asking the same question can get very different answers.
That's why the foundation decides almost everything. The contract terms and financial results have to sit at the center, with claims, eligibility, risk adjustment, benchmarks, and every analysis tying back to them. Without that, AI produces answers that sound plausible, and those are the hardest errors to catch.
Why errors compound in multi-step work
The newest AI tools, often called agents, can take on a whole analysis instead of a single question. They break a task into steps, run each one, and decide what to do next based on what they find. That's a meaningful step forward, and it raises the stakes on reliability. Every step involves the same kind of judgment calls above, and the agent makes them one after another.
If each step in an analysis is right 95% of the time, a 10-step analysis gets every step right only about 60% of the time. In this work, a 60% correct answer is 100% wrong. A small error early on carries through everything after it. A wrong member month count in step two shows up in every number that follows.
There's a less obvious problem too. An agent can take a reasonable-looking turn early, drill down in a different direction than the question needed, and come back with an answer that's directionally right but useless for the decision in front of you.
So speed is the wrong thing to judge an AI on in this work. What matters is how dependable each step is, and how you'd know if one went wrong.
What to look for in AI built for actuarial work
Whether you're evaluating a vendor or building your own, these are the five things I'd look for.
- Actuarial methodology. Does the system know how an actuary would approach a utilization versus unit cost question? The definitions and methods should be built into the system so the model doesn't have to guess.
- Tools built for the work. Writing a fresh database query for every step adds risk. Tools designed for specific actuarial tasks, like benchmark comparisons and contract term lookups, make each step more dependable.
- Data tied to the contract. Answers should connect back to the contract's terms and financial results, along with the claims behind them.
- Tests written by actuaries. Ask who wrote the test questions the system is measured against, and whether every change has to pass them before it ships. General AI benchmarks are useful, but they can't tell you whether a system gets a risk contract right. A good test set also lets a team move to a better model when one comes along without breaking what already works.
- A way to check the work. In finance and actuarial work, people need to trace an answer before they'll use it. You should be able to see the plan before it runs, check progress along the way, and trace every number back to its data source, how the data was processed, and the query behind it.
What AI still can't do
Even with all of that in place, AI still gets things wrong. On BIRD, a widely used public benchmark for turning plain-English questions into database queries, the best system as of September 2026 gets about 83% of questions right. People score about 93% according to the public BIRD leaderboard. That benchmark uses general databases. Risk contracts add their own rules on top. That's why the tests and the ability to check the work matter so much.
AI also can't fix data it doesn't have. If a file is late or incomplete, the answer inherits the gap. And judgment still belongs to people: whether a variance is a data problem or a real trend, and whether a number is ready to put in front of a CFO. Used well, AI gives actuaries and finance teams more time to make those calls.
How we're approaching this at Arbital
We started building AI into Arbital about a year and a half ago. The first version turned plain-English questions into SQL on a handful of datasets, and you had to frame the question extremely well to get something plausible back. When we launched Arbital AI in October 2025, answers traced back to the data, but a complex question still needed someone fluent in both the data and the actuarial concepts to frame it and ask the follow-ups.
Last week we launched agents in Arbital AI. They can plan and run a multi-step analysis the way an actuary would, breaking the task into steps, choosing the right analytical tools, and adjusting as results come in. Getting there meant working on everything above:
- Deeper actuarial reasoning and context
- Tools built for actuarial work
- Several foundation models each assigned to the job it does best
- More data tied to each contract, on a platform that has run more than 1,500 risk contract reconciliations
- A growing set of test questions with answers written by our actuaries
(My colleagues Ankur Jain and Alexandru Panait will go into how it's built in an upcoming post.)
We also built a library of Workflows and Workspaces, tested by our actuaries, so teams don't start from a blank page. A Workspace is a shared page where an analysis lives: the findings, the exhibits, and the numbers behind them, all editable by the team. Any number in it can be a starting point. Click into a figure and Arbital AI opens a conversation right there, showing the members, claims, and drivers behind it.
Here’s why that matters. In traditional reporting tools, you can only drill down along paths someone built in advance. A new question usually means a request to the analytics team and a new report or dashboard, which can take days or weeks. In a Workspace, the follow-up question gets answered where you found it, and nobody has to build anything new first.
One provider network with about 260,000 lives has four of its own experts building cost analyses with Arbital AI's agents, each with one-on-one coaching from our actuaries. They're looking at leakage in their clinically integrated network, cost of care in a home-based program, skilled nursing facility strategy, and which independent practices to target.
That's what extending actuarial expertise looks like in practice. Our actuaries stay at the center. They define the methodology, set how the AI reasons through each kind of problem, and write the tests it has to pass. Our product, engineering, and AI teams built the agent, the tools it calls, the error handling, the data pipelines, and the testing underneath, and this release is as much theirs as anyone's.
Where this goes
Most of the industry’s analytics still answer “what happened.” Arbital AI now shows what’s driving a contract’s results and what can be done next to bend the curve, before it's too late.
The next step is getting there without anyone having to ask. Today, someone still has to bring the task. We’re building toward AI that autonomously looks across a whole book of business, flags what deserves attention, and recommends where to act, with a dollar value on each option. That’s what we’re working on next.
Risk contracts keep getting more complex, and the decisions behind them can’t wait until the year closes. Our goal is for every team managing a contract to know where it stands and what’s driving it, with numbers an actuary would sign off on, while there’s still time to change the outcome.
