AI fact checker for lawyers: verify every citation before you file
Last updated: July 26, 2026
An AI fact checker for lawyers extracts each claim and case citation from an AI-drafted brief, grounds every one against independent sources, and returns a per-claim verdict: supported, contradicted, disputed, or unverifiable. It flags fabricated or unsupported citations for review before you file, but does not replace a lawyer's own verification.
Why AI legal drafts get lawyers sanctioned
Generative AI invents case law that looks real. It produces correctly formatted citations, plausible reporter numbers, and quotations that read like a court wrote them, yet the case never existed.
The landmark example is Mata v. Avianca, Inc. in the Southern District of New York. Two attorneys submitted a brief containing cases ChatGPT had fabricated. On June 22, 2023, Judge P. Kevin Castel sanctioned them and their firm $5,000 under Rule 11, finding they had acted in bad faith (see the case summary).
It was not a one-off. Damien Charlotin, a research fellow at HEC Paris, maintains a public AI Hallucination Cases database that had grown well past 1,500 documented filings and rulings worldwide by mid-2026, spanning attorneys and self-represented litigants alike. The tracker is updated daily and exact counts move constantly, so treat any single number as a snapshot and check the database for the current figure.
What the ABA says you must do
You must independently verify AI output before relying on it. ABA Formal Opinion 512, issued July 29, 2024, ties generative AI use to existing duties, including competence (Model Rule 1.1) and confidentiality (Model Rule 1.6). Its core message is that a lawyer cannot outsource professional judgment to a tool (see the ABA announcement).
Mephistopheles helps you meet that verification duty faster. It does not discharge it. It surfaces what to check; the sign-off is still yours.
How the verification workflow runs
Paste an AI-drafted passage into /chat, or drag a full memo or brief into the /try document verifier. The steps:
- Claim and citation extraction. Each factual assertion and each case, statute, or docket citation is pulled out as a discrete unit.
- Grounding. Every unit is checked against independent sources rather than the model's own memory, using semantic retrieval plus web and structured lookups. See grounding.
- Per-claim verdict. Each returns supported, contradicted, disputed, or unverifiable, with the source the judge relied on.
- Risk score. An overall hallucination-risk score tells you how much of the draft needs a human pass.
On its own benchmark of roughly 290 deliberately hard claims across about 20 domains, Mephistopheles catches about 88% of factual errors and passes about 98% of true statements clean, a roughly 2% false-alarm rate. See how accurate it is. That means a fabricated citation can still slip through: the tool narrows the search, it does not guarantee the catch.
Why a real citation can still be unsupported
A citation resolving to a real case is not the same as that case backing your sentence. AI routinely cites a genuine opinion for a proposition the opinion never states, or attaches a real quotation to the wrong holding.
Mephistopheles checks both layers: does the authority exist, and does it actually support the claim attached to it? That second check, claim-support, is the one citation-checkers miss. Read the distinction on citations vs claims.
De-identification for privileged matters
For privileged or confidential drafts, run the de-identify-then-verify path. Identifiers are stripped from the text before any external grounding call, so client names and matter details are not sent out to be looked up. See de-identification.
Honest scope
De-identification reduces exposure; it does not eliminate it. Quasi-identifiers and unusual fact patterns can remain re-identifying, and this path is a risk-reduction measure, not a legal privilege guarantee. Mephistopheles flags claims for attorney review; it does not give legal advice and is not a substitute for your independent verification and professional judgment.
Frequently asked questions
Can an AI fact checker catch fabricated case citations before I file?
It catches many of them. Mephistopheles extracts each citation, checks whether the authority exists, and checks whether it supports the claim it is attached to. On its own benchmark it catches about 88% of factual errors, so it materially reduces fabricated-citation risk. It is a detector, not a guarantee, so you still verify anything it flags and confirm your own record.
Does this replace my duty to verify AI legal research under ABA Opinion 512?
No. ABA Formal Opinion 512 requires you to independently verify AI output, and that duty stays with you. Mephistopheles speeds up verification by pointing you to the claims and citations that need checking, but the professional judgment and sign-off are yours.
Is my client's privileged information sent to third parties?
Use the de-identify-then-verify path. Identifiers are stripped from the text before any external grounding request is made, so client and matter details are not transmitted to outside sources. This reduces exposure but does not eliminate residual re-identification risk from quasi-identifiers, and it is not a substitute for your own confidentiality assessment.
What does 'unverifiable' mean on a legal claim?
It means no authoritative independent source was found to confirm or contradict the claim, so the tool declines to guess. For unpublished orders, sealed matters, or very recent filings, unverifiable is common and correct. Treat it as a prompt to check the primary source yourself.
Verify what your AI just told you.
Paste any AI answer and Mephistopheles checks each claim against independent sources — no sign-up to try.