Over the last few months, I've been building AutoFlow, not as another AI wrapper or workflow tool, but as a verification engine.Instead of asking:
"What does the model think?" we're asking:
Can the answer be mathematically, logically, and evidentially verified?
We're starting with finance because the cost of hallucinations is real.What we've built so far is:
Deterministic evidence extraction pipeline
Typed financial fact normalization
Cross-document reconciliation engine
C++20 verification core
Covenant calculation engine
Source-anchor tracking for every extracted fact
Complete audit trail explaining exactly why every conclusion was reached Synthetic financial benchmark suite designed for reproducible evaluation
Current implementation status:
✅ 11 JSON schemas validated
✅ Evidence extraction pipeline complete
✅ Deterministic fixtures and validation suite
✅ C++ verification engine
✅ 99/99 C++ unit tests passing
Early benchmark results:o
We're benchmarking frontier models on financial verification rather than generic Q&A.
The early runs are showing exactly what we expected:
Strong reasoning models still hallucinate under financial verification tasks. RAG alone is not enough—it retrieves evidence but doesn't verify calculations or resolve contradictions. Deterministic verification dramatically improves trust because every number can be traced back to evidence and independently checked.
We're now preparing large-scale benchmarks across OpenAI, Anthropic, Gemini, open-weight models, and other providers to measure where current AI systems succeed and fail.
The long-term vision
Finance is only the first step.
The goal is to build a Universal Trust Engine consisting of:
• Verification Engine
• Evidence Engine
• Adjudication Engine
An infrastructure layer that allows AI systems to prove their outputs instead of asking users to trust them.
Looking for people who enjoy hard engineering problems
If you're interested in:
C++ Systems programming Verification systems Distributed systems Retrieval and evidence graphs Formal methods AI evaluation Benchmarking Financial infrastructure
I'd love to connect.
We're accepted into the NVIDIA Inception startup program and are currently preparing the next generation of verification benchmarks.
If building infrastructure that makes AI more trustworthy sounds interesting, send me a message or leave a comment.
I'd especially love to hear from people who think current LLM evaluation is fundamentally broken
[link] [comments]