One of the first ideas I had while planning our AI verification prototype was to add a confidence score to every result.
The more I thought about it, the less I liked the idea.
Imagine an AI says:
"EBITDA = $12.3M (96% confidence)"
What does 96% actually tell the person reviewing a borrower package?
It doesn't explain:
where the number came from,
whether another document reports a different value,
whether the calculation follows the covenant definition,
or whether the evidence is complete.
A high confidence score can easily become another thing people trust without understanding.
So we removed it.
Instead, we're experimenting with something much simpler:
Every important financial claim should answer four questions:
Where did this value come from?
Can I open the source immediately?
Does another document disagree?
If it's calculated, can I reproduce the math?
Maybe confidence scores are useful in some applications.
For the kind of workflows we're exploring, I'd rather help someone verify an answer than persuade them to trust one.
I'm curious how others think about this.
If you're building AI products, do you expose confidence scores to users, or have you found better ways to communicate reliability?
[link] [comments]