The Mephistopheles verification API

Two ways to call it

There are two integration paths, and you can use either or both.

Drop-in generate-and-verify. Point your existing OpenAI-compatible client at the Mephistopheles base_url. Your code is unchanged; each completion comes back with verification attached, so you generate and check in one round trip.

Verify what you already have. If the answer already exists, from any model, a human, or a document, send it to POST /verify. You get back the extracted claims, a verdict and sources for each, and an overall risk score. This path does not care who or what produced the text.

Both run the same pipeline described in how it works: claim extraction, independent grounding, a tiered judge, and a per-claim verdict.

How do I do the OpenAI-compatible base_url swap?

Change one line. Keep your OpenAI client, models, and prompts; swap the base_url and key so requests route through Mephistopheles, which generates the completion and verifies it before returning.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_MEPHISTOPHELES_KEY",
    base_url="https://api.mephistopheles.ai/v1",
)

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "When was the Treaty of Westphalia signed?"}],
)

# Standard completion, plus a verification block:
print(resp.choices[0].message.content)
print(resp.verification.risk_score)      # overall hallucination-risk
print(resp.verification.claims)          # per-claim verdicts + sources

Because the interface is OpenAI-compatible, existing SDKs, retries, and streaming clients keep working. The only new thing you read is the verification object.

What does POST /verify return?

Send an answer, optionally with the question that produced it, and get structured verification back. The request:

POST https://api.mephistopheles.ai/v1/verify
Authorization: Bearer YOUR_MEPHISTOPHELES_KEY
Content-Type: application/json

{
  "question": "When was the Treaty of Westphalia signed?",
  "answer": "The Treaty of Westphalia was signed in 1658, ending the Thirty Years' War."
}

The response returns every extracted claim with its own verdict, the sources behind it, and an aggregate risk score:

{
  "risk_score": 0.71,
  "claims": [
    {
      "claim": "The Treaty of Westphalia was signed in 1658.",
      "verdict": "contradicted",
      "sources": [
        {"title": "Peace of Westphalia", "url": "https://en.wikipedia.org/wiki/Peace_of_Westphalia"}
      ],
      "note": "Sources give 1648, not 1658."
    },
    {
      "claim": "The Treaty of Westphalia ended the Thirty Years' War.",
      "verdict": "supported",
      "sources": [
        {"title": "Peace of Westphalia", "url": "https://en.wikipedia.org/wiki/Peace_of_Westphalia"}
      ]
    }
  ]
}

Every verdict is one of supported, contradicted, disputed, or unverifiable. The last one is a real answer, not an error: when no authoritative source exists, Mephistopheles says so instead of guessing. For sensitive or PHI payloads, a de-identify-then-verify path strips identifiers before external grounding.

How is this different from Cleanlab TLM and Galileo?

All three attach a trust signal to LLM output, and they credit different, genuine things. The difference that matters is where the signal comes from.

Cleanlab's Trustworthy Language Model (TLM) produces a 0-to-1 trustworthiness score from the model's own behavior, per its documentation, using self-reflection, consistency across multiple sampled responses, and uncertainty estimation. It is a strong confidence signal and needs no external sources, but it measures how sure the model is, not whether an independent source agrees.

Galileo centers on Context Adherence (groundedness): the metric checks whether each claim in an answer is supported by the retrieved RAG context you supplied. Per Galileo's materials, it can be computed either with ChainPoll, their chain-of-thought LLM method that also returns explanations, or with their purpose-built Luna small evaluation models, which are cheaper and lower-latency and which Galileo reports outperform ChainPoll. Either way, Context Adherence is excellent for catching a model that drifts from its sources, but it assumes the supplied context is correct.

Mephistopheles fetches its own independent external evidence for each claim, so it can flag a claim that is perfectly faithful to a wrong context or that a confident model fully believes. Independent grounding is the wedge. It is also why we publish an honest false-alarm rate rather than a bare confidence number.

These are honest, sourced descriptions of what each tool does well; check each vendor's current docs, since products evolve.

How is the API priced?

Billing is byte-metered: you pay for the volume of text verified, on tiered plans through Stripe. Verification cost scales with the work done, so short claims cost little and long documents cost more, with no per-seat lock-in for the API itself. See pricing for current tiers and byte allowances.

Because the judge is tiered, easy claims are settled cheaply and only ambiguous or high-stakes claims escalate to stronger judging, which keeps typical cost down without capping the ceiling on the hard cases where you most need a correct verdict.