Verify AI-generated financial report numbers before they ship
Last updated: July 26, 2026
To verify AI-generated financial report numbers, extract every figure, citation, and claim, ground each against independent sources, and get a per-claim verdict. Mephistopheles flags fabricated statistics and mis-attributed quotes for human review before you publish; it is not financial advice.
Fabricated figures in AI-drafted reports
AI-drafted financial and consulting reports can contain figures and quotations that are simply invented, dressed in the format of a rigorous document.
The clearest recent example is Deloitte Australia. A report commissioned by the Department of Employment and Workplace Relations, valued at roughly AU$440,000, was found to contain fabricated academic citations and an invented quotation attributed to a Federal Court judgment in the robodebt case Amato v Commonwealth. University of Sydney academic Dr Christopher Rudge identified the errors after publication. Deloitte issued a corrected version on October 6, 2025, disclosing it had used an Azure OpenAI GPT-4o-based tool chain, and agreed to refund the final contract installment (reported by The Register).
The lesson generalizes: a professionally formatted number is not a verified number.
How figure and claim verification works
Drag a report into /try, or paste a section into /chat. The steps:
- Figure and claim extraction. Each numeric figure, statistic, quotation, and factual assertion is pulled out as a discrete unit. See claim extraction.
- Grounding. Each unit is checked against independent sources rather than the model's own output. See grounding.
- Per-claim verdict. Every figure and claim returns supported, contradicted, disputed, or unverifiable, with the source used.
- Risk score. An overall risk score shows how much of the document needs a human pass.
Figures that reference internal data an external source cannot see will return unverifiable; that is correct behaviour, and it directs your review to the numbers only you can confirm.
Why a cited figure can still be wrong
AI often attaches a real report or filing to a number that report never stated, or quotes a figure from the wrong period. The citation exists; the support does not.
Mephistopheles checks whether the source actually contains the figure attributed to it, not just whether the source exists. That is the gap a link-checker leaves open. See citations vs claims.
De-identification and honest scope
For sensitive financials, run the de-identify-then-verify path so account names, client identifiers, and confidential details are stripped before any external grounding call. See de-identification.
Honest scope
Mephistopheles verifies whether stated claims and figures are supported by independent evidence. It is not financial, investment, accounting, or tax advice, and its verdicts are inputs for a qualified professional, not conclusions. De-identification reduces exposure but cannot guarantee removal of every re-identifying detail, and internal or proprietary figures will often return unverifiable rather than supported.
Frequently asked questions
Can it catch made-up numbers in an AI-generated financial report?
It catches many of them. Each figure and claim is extracted and grounded against independent sources, and anything it cannot confirm is flagged rather than passed. Internal or proprietary figures typically return unverifiable, which directs your review to the numbers only you can confirm. It is a detector, so verify every flag before publishing.
Is this financial advice?
No. Mephistopheles checks whether stated claims and figures are supported by independent evidence and returns verdicts for human review. It does not provide financial, investment, accounting, or tax advice, and its output is an input to your professional judgment, not a recommendation.
How do I verify sensitive financials without exposing them?
Use the de-identify-then-verify path, which strips account names, client identifiers, and confidential details before any external grounding request. That reduces exposure, though it cannot guarantee removal of every re-identifying signal, so apply your own confidentiality assessment for material information.
What if a figure references our internal data?
An external source cannot confirm data it cannot see, so those figures return unverifiable. That is the honest outcome: the tool will not fabricate support. Confirm internal figures against your own systems of record, and use the verifier to catch the externally-checkable claims around them.
Verify what your AI just told you.
Paste any AI answer and Mephistopheles checks each claim against independent sources — no sign-up to try.