I expected the flashy AI claims to be the hard part to verify. It was mostly the opposite.
I screened 100 XPRIZE AI projects, looked closely at 14, and independently tested 5 using public code, data, live products, and official documents.
One product reported a 91.2% replication rate for behavioral-science effects. I reran the public responses and scoring code and got 91.2%.
But the number depended heavily on the aggregation rule:
- any one of five models: 91.2%
- best single model: 79.4%
- pooled responses: 73.5%
- majority of models: 67.6%
Their repo was transparent about the first rule being an optimistic ceiling. So the interesting part wasn't whether the number was fake. It wasn't. The question was what exactly the number meant.
Another business said AI pauses ad spend, rewrites strategy nightly, and writes product copy.
The published code showed:
- ad pausing: deterministic thresholds, no model call
- strategy: Claude
- product copy: Gemini
So "AI runs the business" was technically hiding three very different mechanisms.
I found the same pattern elsewhere. A legal-translation product cited the WMT25 benchmark correctly: 87 documents, 332 segments. But the official dataset contained literary, news, social-media, and speech text. Legal-domain segments: zero.
None of those findings made the original claims simply "true" or "false." Verification kept turning into a different question: What exactly was measured? What scope does the evidence support? Where does the model actually enter the system?
The odd part was that the technical claims were often reproducible. The claims I could not independently establish were things like revenue, customer count, and usage, because those records weren't public.
I wrote up the five cases here.
The thing I'm still thinking about is this: when an AI benchmark reproduces exactly, what else has to be true before you'd call the underlying claim independently verified?
[link] [comments]