Why it matters
When a customer screenshot shows a wrong refund answer, observability tells you whether retrieval, policy, or generation failed. Without retrieval traces, teams tweak prompts randomly.
How it works
Log query text, filters applied, candidate chunks with scores, injected context, model output, citations, and policy decisions. Dashboards aggregate miss rates by intent and collection.
Example
Nintendo support lead inspects a failed billing answer and sees retrieval ranked an archived Help Center promo third, but policy correctly excluded it. The model hallucinated filler because context was thin. Fix: expand coverage in the Billing FAQ, not tweak temperature.
Common mistakes
- 1Logging only user-visible text without retrieval candidates
- 2PII-heavy logs without redaction policy
- 3No retention aligned with compliance needs
Definitions are useful. Governed knowledge is better.
Wiki helps you turn these concepts into a real system for the knowledge behind your AI.
See Wiki in action