Guides

Detect Hallucinations in Generative AI: 6 Methods That Actually Catch Them in Production (2026 Guide)

How to identify hallucinations in AI with six detection methods: Future AGI evals, ChainPoll judges, NLI, SelfCheckGPT, RAG faithfulness, and human review.

· Updated
· 8 min read
evaluations hallucination llms
Detect Hallucinations in Generative AI six methods
Table of Contents

TL;DR How to Identify Hallucinations in AI

Knowing how to identify hallucinations in AI output comes down to checking each answer against evidence the model cannot fake: the retrieved context, an entailment model, a second sample, or a human reviewer. In production that means a cheap inline check on every response, an LLM judge scoring the trace asynchronously, and human review for the low-scoring tail. The six LLM hallucination detection methods below map each check to the job it does best.

DecisionRecommendation
Best end-to-end stackFuture AGI detect_hallucination on turing_flash or turing_large, plus local faithfulness
Best for RAGFaithfulness plus chunk attribution and context precision
Best when no sourceSelfCheckGPT style multi-sample consistency or ChainPoll consensus
Cheapest deterministicNLI entailment (DeBERTa-v3-MNLI or similar) on response vs source
Highest precisionChainPoll style multi-sample LLM judge
Real-time patternNLI inline plus async LLM-judge attached to span via traceAI
Gateway guardrail layerAgent Command Center Hallucination Detection guardrail (Post stage)

Why Hallucinations Still Break Production LLM Apps

Newer frontier models still hallucinate. A September 2025 paper, Why Language Models Hallucinate, argues that hallucinations persist even in state-of-the-art systems because training and evaluation procedures reward guessing over acknowledging uncertainty.

The cost is real. In June 2023 a New York federal judge sanctioned two lawyers and their firm with a $5,000 penalty after they filed non-existent judicial opinions with fake quotes and citations created by ChatGPT. Detection in production is not optional.

Intrinsic vs Extrinsic Hallucinations

The Survey of Hallucination in Natural Language Generation splits the failure mode in two:

  • Intrinsic hallucination. The model contradicts the input, the retrieved context, or the system prompt. Example: a RAG bot summarizes a clause the source document explicitly negates.
  • Extrinsic hallucination. The model invents claims that cannot be verified against the source at all. Example: a summarization model adds a number that does not appear anywhere in the input.

The mitigations differ:

TypeBest detectionBest mitigation
IntrinsicFaithfulness, NLI entailment vs sourceTighter retrieval, refuse-if-not-grounded
ExtrinsicMulti-sample consistency, external fact lookupRetrieval augmentation, citations

Both should be measured in parallel. A model can pass a contradiction check and still add claims the source never made, so pair faithfulness or NLI with multi-sample consistency or an external fact lookup.

How to Spot AI Hallucinations Without Code

Before any tooling, you can catch many hallucinations by hand. Check every citation, link, case name, and number against a source you open yourself, because fabricated references read as confidently as real ones. The lawyers sanctioned in 2023 filed judicial opinions that did not exist, complete with fake quotes.

Then ask the same question again, or ask where a claim came from. SelfCheckGPT showed that hallucinated facts tend to diverge and contradict one another across resampled answers, while facts the model knows stay consistent. These checks suit a handful of answers; the six methods below automate them at production scale.

Method 1: Future AGI Hallucination and Faithfulness Evals

The fastest production-ready path. Future AGI’s agent-learning-kit SDK runs local hallucination metrics such as faithfulness with no API key, and cloud templates such as detect_hallucination on Turing judge models.

from fi.evals import evaluate  # cloud call: set FI_API_KEY and FI_SECRET_KEY

result = evaluate("detect_hallucination", input="When was the moon landing?",
                  output="Apollo 11 landed on the moon in July 1969.",
                  context="Apollo 11 landed on the Moon on July 20, 1969.", model="turing_flash")
print(result.passed, result.reason)

For a free local check with no API key, run faithfulness against the retrieved context:

result = evaluate("faithfulness", output=model_response, context=retrieved_chunks)
print(result.score)  # 0 to 1, no API key needed

For chunk-level diagnosis, run the chunk_attribution template:

attribution = evaluate("chunk_attribution", output=model_response, context=retrieved_chunks, model="turing_flash")

Latency profile on Future AGI cloud:

JudgeLatency
turing_flashabout 1 to 2 seconds
turing_smallabout 2 to 3 seconds
turing_largeabout 3 to 5 seconds

Pick turing_flash for online scoring and turing_large for nightly regression suites and high-stakes flows. The SDK, agent-learning-kit, is Apache-2.0 and installs with pip install agent-learning-kit.

Method 2: ChainPoll Style Multi-Sample LLM Judge

ChainPoll, introduced in the 2023 Galileo paper, asks an LLM judge, with a detailed chain-of-thought prompt, whether a completion hallucinates, repeats that check (typically 5 times), and scores the share of “yes” answers from 0 to 1. The intuition: a hallucinated claim splits the judge’s votes; a correct claim does not.

You can roll your own with the LLM-as-Judge engine in evaluate(): write the criteria once, call it five times, and average result.score. The example uses GPT-5.6 Sol, which OpenAI recommends in place of the deprecated GPT-5 snapshot; any LiteLLM model string works.

from fi.evals import evaluate

question = "What year did Apollo 11 land on the moon?"
answer = "Apollo 11 landed on the moon in 1969."
criteria = "Score 0 to 1. Is this answer to '{query}' supported by widely accepted facts? Return 0 if any part is fabricated, 1 if fully accurate."

raw_scores = []
for _ in range(5):
    result = evaluate(prompt=criteria, output=answer, query=question, engine="llm", model="gpt-5.6-sol")
    raw_scores.append(result.score)

final = sum(raw_scores) / len(raw_scores)

Strengths: high precision on factuality. Weaknesses: 5x judge cost, 5x latency unless you parallelize. Reserve for high-stakes flows or nightly batch.

Method 3: NLI-Based Contradiction Detection

NLI is cheap, deterministic, and suited to inline gating. Use a fine-tuned NLI model like MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli and score the entailment between the response and the source.

from transformers import pipeline

nli = pipeline(
    "text-classification",
    model="MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli",
)
result = nli({"text": source_text, "text_pair": response})
# label in {"entailment", "neutral", "contradiction"}

If contradiction probability exceeds a threshold (often 0.5), flag for review or trigger a heavier LLM-judge check. The trade-off: NLI struggles with long-form summarization where claims are decomposed across sentences. Pair NLI with sentence-level decomposition for better recall. If you already run agent-learning-kit, pip install "agent-learning-kit[nli]" switches its local hallucination metrics to DeBERTa-based NLI scoring with no code changes.

Method 4: SelfCheckGPT Style Consistency Probing

When you have no external source to compare against, sample the model multiple times and measure inter-sample agreement. The 2023 SelfCheckGPT paper showed that inconsistency across stochastic samples correlates strongly with hallucination.

from openai import OpenAI

client = OpenAI()

def selfcheck(question, n=5):
    samples = []
    for _ in range(n):
        resp = client.chat.completions.create(
            model="gpt-5.6-sol",
            messages=[{"role": "user", "content": question}],
        )
        samples.append(resp.choices[0].message.content)
    return samples

Score agreement with an NLI model or with an LLM-judge consensus prompt. The SelfCheckGPT repository recommends its NLI variant, SelfCheckNLI, which scores each sentence by contradiction probability from a DeBERTa-v3-large MNLI model.

Use case: free-form generation without retrieval. Limitations: n times inference cost, struggles when the model is consistently wrong.

Method 5: RAG-Specific Faithfulness, Chunk Attribution, Context Precision

If your system is RAG, these four metrics are the workhorses. Every claim in the output should map to a retrieved chunk. Every retrieved chunk should be relevant. Every relevant chunk should actually be used.

MetricWhat it answers
FaithfulnessDoes the output stay grounded in the retrieved context?
Chunk attributionWhich chunks contributed to the output?
Context precisionOf the retrieved chunks, how many were actually relevant?
Context recallDid retrieval surface all the chunks needed to answer?

In fi.evals, faithfulness, context_precision and context_recall run as free local metrics, while chunk_attribution is a cloud template on a Turing model. The RAG metrics reference covers context_precision and context_recall, which also need the user question as input; keep local and cloud metrics in separate calls. See RAG evaluation metrics for the full playbook. The flow:

from fi.evals import evaluate

# Local, no API key; question, answer, chunks, reference_answer come from your eval set
faithfulness = evaluate("faithfulness", output=answer, context=chunks)
precision = evaluate("context_precision", input=question, output=answer, context=chunks, expected_output=reference_answer)

# Cloud template: separate call
attribution = evaluate("chunk_attribution", output=answer, context=chunks, model="turing_flash")

Future AGI traceAI showing a RAG query trace with RAG_ContextRelevance and RAG_AnswerRelevance failing and RAG_Groundedness and RAG_Faithfulness passing on a dark dashboard

Method 6: Human-in-the-Loop With Active Learning

Human review has the highest precision and the lowest scale. Sample low-confidence outputs (NLI contradiction, low judge score, high refusal rate) into a review queue. Subject-matter experts label them. Labels feed back into:

  1. Prompt updates. Common failure patterns become explicit instructions.
  2. Judge calibration. Disagreement between human and judge surfaces judge bias.
  3. Eval datasets. Labeled failures become regression cases you rerun after every prompt or model change.
from fi.evals import evaluate

for case in labeled_failures:
    result = evaluate("detect_hallucination", input=case["question"], output=my_agent(case["question"]),
                      context=case["context"], model="turing_flash")
    print(case["question"], result.passed)

Pair human review with active learning sampling: prioritize outputs the judge is least sure about, not random samples. This maximizes information per labeled example.

A Production Architecture for Hallucination Detection

A production setup runs a cheap NLI check inline on every response and scores the traced span asynchronously with detect_hallucination. Low scores go to alerting and a human review queue, and the labels feed the prompt, the judge and the regression suite.

User request
  -> Gateway (Agent Command Center)
       - PII guard, prompt-injection guard, rate limit
  -> LLM call (traceAI instrumented span)
  -> Inline NLI check on response vs retrieved context
       - if contradiction > 0.5: refuse or fallback
  -> Return response to user
  -> Async detect_hallucination eval task on the span
       - low scores feed alerting + dataset
  -> Sample low-score spans into human review queue
  -> Labels feed prompt + judge + regression suite

Production hallucination detection architecture with a gateway, traced LLM call and inline NLI check on the request path, an async detect_hallucination eval on the span, and a human review queue whose labels feed the prompt, judge and regression suite.

Instrumentation with traceAI is one block:

import os
from fi_instrumentation import register, FITracer
from fi_instrumentation.fi_types import ProjectType
from traceai_openai import OpenAIInstrumentor

os.environ["FI_API_KEY"] = "your_fi_api_key"
os.environ["FI_SECRET_KEY"] = "your_fi_secret_key"

trace_provider = register(
    project_type=ProjectType.OBSERVE,
    project_name="prod-hallucination-detection",
)
tracer = FITracer(trace_provider.get_tracer(__name__))
OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

Every span is now evaluable. For deeper real-time eval patterns see Real-time LLM evaluation setup.

When To Pick Which Method

Use casePrimaryBackup
RAG chatbotFaithfulness plus chunk attributionNLI inline
Free-form summarizationChainPoll style judgeSelfCheckGPT
Long-tail Q and Adetect_hallucination on turing_largeHuman review
High-stakes medical or legalChainPoll plus human reviewNLI inline gate
Real-time chatbotNLI inline plus async fi.evalsturing_flash judge
Multimodal outputcaption_hallucination templateVision grounding model

Common Pitfalls

Treating one metric as ground truth. No single metric catches both intrinsic and extrinsic hallucinations. Use at least two.

Ignoring judge bias. Different judge models disagree systematically. Always pin a judge model version per evaluator and re-validate after every judge upgrade.

Skipping calibration. A judge score of 0.7 means nothing until you have calibrated it against human labels in your domain. Run 100 human labels per evaluator at deployment.

Single-shot prompts. ChainPoll style consensus consistently beats single-shot. If precision matters, vote.

Forgetting retrieval upstream. If your retriever drops the relevant chunk, no detector downstream can save you. Measure context recall first.

No regression suite. Real-time eval finds today’s problems. Regression suites built from labeled failures prevent tomorrow’s. Run both.

Where Hallucination Detection Goes Next

  • Calibrated abstention. Models that emit confidence and refuse when low. A 2024 survey of abstention in LLMs frames refusal as a way to mitigate hallucinations.
  • Tool-grounded detection. Models that call a retrieval tool to fact-check their own draft before returning it.
  • Multimodal detection. Future AGI’s cloud eval templates include caption_hallucination for image captions, and Detect Hallucination scores text, audio, image, and chat outputs.
  • Regulator-ready logs. EU AI Act obligations and similar frameworks push toward documented risk-management evidence for high-risk systems. Hallucination metric histories from fi.evals are practical compliance artifacts to retain.

Catch Hallucinations Before Your Users Do

pip install agent-learning-kit traceAI-openai
export FI_API_KEY=...
export FI_SECRET_KEY=...
from fi.evals import evaluate

result = evaluate("detect_hallucination", input="What was the boiling point of water at sea level?",
                  output="Water boils at 100 degrees Celsius at sea level.",
                  context="At sea level, pure water boils at 100 degrees Celsius.", model="turing_flash")
print(result.passed, result.reason)

For the dashboard view, log in at app.futureagi.com. For routing, gateway, guardrails, and BYOK key management visit the Agent Command Center. Docs live at docs.futureagi.com.

The fake-citation brief described earlier reached a judge because nothing checked the model’s output against a source. An inline NLI gate stops the worst answers before users see them, and detect_hallucination on every traced span plus a review queue catch the rest.

Related reading:

Book a call with Future AGI to wire hallucination detection into your stack.

Sources

Frequently Asked Questions

What is a hallucination check in AI?

A hallucination check in AI tests whether a model's answer is supported by its source, its retrieved context, or verifiable facts before anyone relies on it. Checks look for intrinsic hallucinations, which contradict the source, and extrinsic ones, which the source cannot verify.

How can I detect hallucinations in an LLM?

To detect hallucinations in an LLM, compare each answer against evidence such as the retrieved context, an NLI entailment model, resampled answers, or an LLM judge. Faithfulness scoring suits RAG, SelfCheckGPT-style sampling suits answers with no source, and human review covers high-stakes flows.

How can I test for hallucinations in AI?

To test for hallucinations in AI, run your model on prompts with known answers or source documents and score every output with a hallucination eval. Add each failure that reviewers label to that set, then rerun it after every prompt or model change so regressions show up before release.

How do you spot AI hallucinations without code?

You can identify hallucinations in AI without code by checking every citation, link, and number the model gives against a source you open yourself. Then rerun the prompt or ask a follow-up, because hallucinated details tend to change between answers while known facts stay consistent.

Is there a free AI hallucination detector?

Yes, Future AGI's agent-learning-kit SDK is Apache-2.0 and runs local hallucination metrics such as faithfulness and hallucination_score with no API key. Install its nli extra for DeBERTa-based entailment scoring, or call the cloud detect_hallucination template on a Turing model when you want a judge that explains each verdict.
Related Articles
View all