Why it matters
A knowledge gap is a question your AI is expected to handle but cannot support with governed evidence. Gaps invite improvisation, escalations, and hallucinations. They are product signals, not model defects.
Gaps appear in failed tests, thumbs-down clusters, and repeat tickets. Ignoring them teaches users the bot is unreliable on topics they care about.
Closing gaps is documentation and policy work with verification, not prompt engineering alone.
Launches create predictable gap spikes. Budget gap closure before launch, not after deflection craters.
Letting models improvise on known gaps trains users the bot is unreliable on topics they care about most.
How it works
Mine gaps from eval failures, support taxonomy, and feedback tagged by intent. Prioritize by risk times volume.
Assign doc owners and due dates. Create or update Help Center and policy content with verification before indexing.
Add regression tests in the same release as gap closure. Content without tests invites repeat gaps.
Track gap burndown publicly for content leadership. Visibility drives staffing decisions.
Map gaps per application. Chatbot gaps and agent gaps differ even on the same topic.
Close gaps with verified content and tests in one release. Content alone without tests reopens gaps quietly.
Publish gap burndown metrics for content leadership staffing decisions.
Example
Nintendo flagged a gap on student discount eligibility with no approved article. Docs added a Help Center section, verified it, and added tests before enabling chatbot answers.
Common mistakes
- 1Letting models improvise on known gaps
- 2Gap lists without risk prioritization
- 3Closing gaps without regression tests
Test the answer before your customer does.
Run answer tests and evaluations against your governed knowledge before you ship.
See answer evaluations