Quick context: I've spent the last year+ building something in the "AI hallucination" space, specifically for finance, and I want honest feedback before I go further — not upvotes, actual criticism.
The problem I'm trying to solve:
AI copilots are increasingly drafting financial numbers — ratios, covenant checks, reconciliations, KPIs pulled from statements. The issue isn't that AI is bad at this, it's that it's confidently wrong sometimes, and in finance a confidently wrong number in a report or a covenant calculation isn't a minor bug, it's a real liability.
What I built:
A separate, deterministic verification layer (not another AI model) that sits behind the AI output. It:
- Extracts the actual source values from the underlying documents (PDFs, XLSX, DOCX)
- Independently recalculates the claimed number using exact rules/formulas, not vibes
- Compares the AI's claim against the recalculated value
- Flags mismatches with a full audit trail — what evidence was used, what rule was applied, where they diverged
So instead of "trust the AI's math," it's "here's proof the math is right, or here's exactly where it's wrong and why."
Where it stands right now:
- Working end-to-end on core financial ratios (net leverage, and a few others)
- Full evidence-to-conclusion traceability (nothing is asserted without a pointer back to source data)
- Not yet: broad rule coverage, tolerance-based matching (right now it's strict exact-match, which I know will cause false positives on rounding — actively working on this)
What I'm NOT asking for: Money, beta signups, "check out my landing page." I genuinely want this torn apart before I put more time into the wrong thing.
What I actually want to know:
- If you work in finance/accounting/audit/compliance — does "AI drafts it, a deterministic engine proves it" sound like something you'd actually want, or is this solving a problem nobody has?
- If you've built anything adjacent (fact-checking pipelines, agent guardrails, financial data extraction) — what broke when you tried something similar? What am I not seeing yet?
- Anyone dealt with the "AI + audit trail" requirement from a compliance angle — what would actually satisfy an auditor or regulator here, versus what sounds good but isn't enough?
Happy to answer anything about how it works under the hood. Not trying to be cagey, just trying to keep this post from turning into a spec doc.
[link] [comments]