Why it matters
Stale AI answers repeat old facts because underlying knowledge or sources changed and nobody updated the loop. They look like model bugs but are ops bugs.
How it works
Trace stale answers to source versions and collection freshness. Fix docs, re-index, rerun tests, and publish through release management.
Example
Nintendo customers received 30-day refund answers months after the Refund Policy moved to 14 days because an old FAQ chunk remained indexed. Monitoring plus regression tests now catch that class of drift before deploy.
Common mistakes
- 1Blaming the model without checking source version IDs
- 2Fixing the web page but not the vector index
- 3No customer comms when widespread stale answers discovered
Your source changed. Your AI should know.
Track source freshness, detect changes, and review impact before answers drift.
Explore Source Maintenance