Precision and recall
Last updated: July 26, 2026
Precision is how often a fact-checker is right when it flags something; recall is how many of all real errors it catches. They trade off: flag everything and recall rises but precision collapses. For a verifier, high precision matters most — flags you can't trust are worthless.
What are precision and recall?
Precision and recall are the two numbers that measure a detector. In fact-checking terms:
- Precision asks: when the checker flags a claim as wrong, how often is it actually wrong? High precision means few false alarms.
- Recall asks: of all the truly wrong claims, how many did the checker catch? High recall means few misses.
They are different failures. Low precision means the tool cries wolf (see false-alarm rate). Low recall means it lets errors through. You need both numbers to know how a checker behaves — a single 'accuracy' figure can hide a disaster in either.
The trade-off, and why precision wins for a verifier
Precision and recall pull against each other. Flag everything and you catch every error (perfect recall) but drown the user in false alarms (terrible precision). Flag only the certain cases and precision soars but you miss subtle errors (lower recall). The art is balancing them for the job.
For a fact-checker, precision is the one you cannot compromise. A flag is only useful if you act on it, and you only act on flags you trust. A high-recall, low-precision checker flags so much noise that you learn to ignore it — at which point its recall is irrelevant.
Mephistopheles prioritises precision: on its private benchmark of roughly 290 hard claims across about 20 domains, it passes about 98% of true statements clean (a false-alarm rate near 2%) while catching about 88% of factual errors, each with a contradicting source. Those are its own measured numbers on that set, not an external benchmark. See how accurate it is.
A worked example
Run a checker over 100 claims, 10 of which are truly false. Say it raises 12 flags and 9 of them are real errors. Precision is 9/12 = 75% (three flags were false alarms). Recall is 9/10 = 90% (it missed one real error). Those two numbers describe it far better than any single score — and they tell you where to improve: tighten precision by grounding harder before flagging, or lift recall by widening the sources searched.
Frequently asked questions
What's the difference between precision and recall?
Precision measures how often the checker is right when it flags something — few false alarms. Recall measures how many of all real errors it catches — few misses. A checker can be strong on one and weak on the other, so both must be reported. See false-alarm rate.
Why does precision matter more than recall for a fact-checker?
Because a flag is only useful if you trust it. A high-recall checker that flags many true statements trains you to ignore its output, making its catches worthless. High precision — reliable flags — is what keeps the tool usable, so it comes first.
Can you have both high precision and high recall?
To a degree, but they trade off — pushing one usually costs the other. The goal is a balance appropriate to the task, with precision prioritised for a verifier. Returning unverifiable instead of guessing is one way to protect precision without abandoning recall.
Verify what your AI just told you.
Paste any AI answer and Mephistopheles checks each claim against independent sources — no sign-up to try.