How Mephistopheles works

What does Mephistopheles actually do?

Mephistopheles is the accuracy layer for any AI. You give it an AI answer, it breaks that answer into individual factual claims, checks each claim against sources it fetches independently, and returns a verdict for every claim plus an overall hallucination-risk score.

The pipeline has four stages: claim extraction, independent grounding, a tiered judge, and verdict plus risk. The design principle running through all four is independence. Mephistopheles does not ask the model that wrote the answer whether it feels confident. It goes and gets its own evidence. That single choice is the honesty moat, and it is what separates verifying truth from measuring a model's self-assurance.

Step 1: How does claim extraction work?

Claim extraction turns prose into a list of discrete, checkable statements. A paragraph like "The treaty was signed in 1648 and ended a thirty-year conflict, which most historians consider the deadliest in the region until then" splits into separable claims: the date, the duration, and the comparative superlative.

This matters because claims are not equal. A verifiable date sits next to an untestable opinion ("most historians consider") inside one sentence. Extraction isolates the parts a source can actually confirm or contradict from the parts that are interpretation, prediction, or opinion, so the judge spends its effort where evidence exists. Each extracted claim is checked on its own, which is why a single AI answer can come back partly supported and partly contradicted.

Step 2: What does independent grounding mean?

Grounding is the core of Mephistopheles, and "independent" is the load-bearing word. For each claim, the engine gathers evidence from several source types that do not share a single point of failure:

• Semantic retrieval (RAG) over a curated reference corpus, matching claims to passages by meaning rather than keyword. See retrieval-augmented generation.
• Live web search, for facts that live outside any fixed corpus or that changed after a model's knowledge cutoff.
• Structured knowledge from Wikidata and Wikipedia, for entities, dates, and relationships with stable references.
• Tools, for claims a calculation or lookup can settle directly rather than by inference.

The point is not volume, it is independence from the source of the claim. Galileo's context-adherence metric, for example, checks whether an answer stuck to the RAG context you handed it, which is a real and useful groundedness signal but assumes the supplied context is correct. Mephistopheles goes out and finds its own evidence, so it can catch a claim that faithfully follows a wrong source. When it cannot find an authoritative source, it says unverifiable rather than manufacturing one.

Step 3: How does the tiered judge and diverse jury work?

Once evidence is gathered, a tiered LLM judge decides the verdict. Tiering means easy, clearly-sourced claims are settled quickly and cheaply, while ambiguous or high-stakes claims are escalated to stronger, more expensive judging. You do not pay frontier-model cost to confirm that Paris is the capital of France.

The judging uses a diverse jury design, and its limits are worth stating plainly. Using several independent judges and independent evidence sources de-correlates random error: if one judge or one source is noisily wrong, the others outvote it. What it does not fix is shared error. If every source and every model has absorbed the same false belief, agreement across them is not proof of truth. This is exactly why multiple models agreeing is not a grounded fact. Independence buys you protection from noise, not from a consensus that happens to be wrong.

Step 4: What do the verdict and risk score mean?

Every claim returns one of four verdicts. The overall hallucination-risk score aggregates them into a single read for the whole answer.

The distinction between contradicted and unverifiable is deliberate and central to the honesty of the product. "I found a source that says this is wrong" is a very different message from "I could not confirm this," and collapsing them would make the tool untrustworthy. For sensitive or PHI content, a de-identify-then-verify path strips identifiers before anything is sent to external grounding.

What are the limitations?

Independent grounding is powerful, and it is not magic. The honest boundaries:

• Shared-belief ceiling. If the whole reference world repeats the same error, Mephistopheles can inherit it. Grounding de-correlates random error, not shared error.
• Unverifiable classes. Private, hyper-local, brand-new, or genuinely contested facts often have no authoritative source and return unverifiable.
• Retrieval gaps. If the right source exists but is not retrieved, a real error can slip through. Coverage improves over time but is never total.
• Provider dependence. The engine leans on external search, knowledge bases, and model providers. Their outages, biases, and indexing gaps flow through to us.
• De-identification residual risk. The de-id path lowers exposure of sensitive data before external grounding but cannot promise zero residual risk. Do not send data you are not permitted to process.