Verify AI research citations before they reach a manuscript
Last updated: July 26, 2026
To verify AI research citations, extract every reference and factual claim, resolve each identifier against CrossRef, PubMed, and arXiv-class registries, then check whether the source actually supports the sentence. Mephistopheles returns a per-claim verdict and marks niche claims unverifiable rather than guessing.
The fabricated-reference problem in AI research writing
AI language models invent references that look impeccable. Author lists, journal names, volume and page numbers, even DOIs, can all be plausible and all be fictitious.
The rates are well documented. In a 2023 Cureus study, Bhattacharyya and colleagues found that of 115 references ChatGPT-3.5 generated for short medical papers, 47% were entirely fabricated, 46% were authentic but inaccurate, and only 7% were both authentic and accurate (see the PubMed record). A separate 2023 Scientific Reports analysis by Walters and Wilder found that 55% of the citations ChatGPT-3.5 produced were fabricated, falling to 18% for GPT-4, with many of the remaining real citations still containing substantive errors. Figures vary by model, prompt, and field, so treat them as a range: a large share of AI references, especially from older models, cannot be trusted at face value.
How reference and claim verification works
Drop a draft section into the /try document verifier, or paste text into /chat. The pipeline:
- Extraction. Every reference and every factual claim is isolated. See claim extraction.
- Identifier resolution. DOIs, PMIDs, and arXiv IDs are resolved against registries such as CrossRef, PubMed, and arXiv to confirm the work exists and the identifier matches the title.
- Claim grounding. The sentence the reference supports is checked against the actual source, not just against the reference metadata. See grounding.
- Verdict. Each item returns supported, contradicted, disputed, or unverifiable.
CrossRef alone indexes over 165 million metadata records across tens of thousands of member organizations, which makes identifier resolution reliable for mainstream literature and honest about its edges elsewhere.
Why a real DOI is not a supported claim
A DOI resolving to a real paper does not mean that paper backs your sentence. Fabricated citations frequently attach a real, unrelated DOI, so the link works and the reference looks legitimate while supporting nothing you wrote.
Mephistopheles separates the two questions: does the reference exist, and does it support the claim? Reference resolution is the smaller check; claim support is the larger one. See citations vs claims for the full comparison.
Why niche claims come back unverifiable
For narrow or very recent findings, the honest answer is often unverifiable. If no authoritative independent source is indexed, Mephistopheles says so rather than manufacturing support.
Honest scope
A checker inherits the shared-belief ceiling of its sources: it can confirm what the literature already records, not adjudicate open scientific disputes or unpublished results. Preprints, paywalled corpora, and emerging findings often return unverifiable. That is the correct behaviour for a verifier, not a failure, but it means the tool assists peer judgment rather than replacing it.
Frequently asked questions
How do I verify AI-generated research citations?
Extract every reference and every claim, resolve each identifier against CrossRef, PubMed, and arXiv-class registries to confirm the work exists, then check whether the source actually supports the sentence it is cited for. Mephistopheles automates all three steps and returns a per-claim verdict, so you can find fabricated and mis-attributed references before they enter a manuscript.
Can it detect a fake DOI?
It resolves each DOI against registries and flags identifiers that fail to resolve or resolve to a title that disagrees with the citation. It also flags the trickier case where a real DOI is attached to the wrong claim. No resolver is perfect, so treat flags as a prompt to open the source yourself.
What happens with preprints or paywalled sources?
If the underlying content is not accessible to an independent source, the claim usually returns unverifiable rather than supported. Mephistopheles will not assert support it cannot ground, so for cutting-edge or paywalled work you should confirm against the primary source directly.
Does it check whether the paper actually says what the AI claims?
Yes, that is the point. Beyond confirming a reference exists, it grounds the specific claim against the source's content. A citation to a real paper that does not support your sentence is flagged rather than passed, which is what a reference-only checker misses.
Verify what your AI just told you.
Paste any AI answer and Mephistopheles checks each claim against independent sources — no sign-up to try.