Why it matters
Hallucination is confident fiction: fake policies, contacts, features. It erodes trust faster than silence because users assume verification. RAG reduces risk; it does not remove it.
Tail questions with weak retrieval are hallucination hotspots. Coverage and abstention matter more than model size there.
Hallucinations cluster on long-tail intents where coverage is thin and retrieval returns weak matches above threshold. Fixing hallucination without fixing gaps is whack-a-mole.
Internal copilots hallucinate too. Employees waste hours on invented procedures that sound plausible in Slack tone.
How it works
Classify hallucinations in postmortems: empty retrieval, wrong retrieval, ignored retrieval, parametric fill, policy gaps.
Track rates by intent over time. Spikes precede complaints if watched.
Lower confidence should trigger abstention or handoff, not longer answers.
Instrument low-retrieval sessions explicitly. Averages hide improvisation.
Publish hallucination rate by intent monthly alongside citation rate. Leadership funds docs when metrics move, not when someone feels nervous.
Train support to recognize hallucination signatures: contact details, guarantees, precise numbers without citations. Signatures speed triage.
Example
Nintendo chatbot hallucinated a phone support line not listed anywhere in the Help Center. Root cause: empty retrieval on a contact intent and no refusal rule. Fix added escalation link and abstention tests.
Common mistakes
- 1Assuming RAG eliminates hallucination without abstention tests
- 2Blaming temperature while retrieval returned zero chunks
- 3No logging when models answer without citations on factual intents
- 4Ignoring hallucinations in internal copilots that later face customers
Test the answer before your customer does.
Run answer tests and evaluations against your governed knowledge before you ship.
See answer evaluations