The AI Knowledge Base Checklist: 25 Things to Verify Before You Publish

Start from the smallest verified surface that can answer common questions. Expand collections only after review and Access Simulator checks. Require citations on answers that affect customer commitments. Pair published docs with the same sources AI cites so customers can verify responses in the help center.

Public AI fails loudly when it cites internal collections, drafts, or conflicted policies. The public safety and docs publishing groups in the checklist exist because customers cannot see your internal intent. They only see the answer. If your Access Simulator passes but answer tests show citations to internal URLs, you still fail launch readiness even when retrieval technically allowed the document.

Support and ops copilots benefit from internal troubleshooting collections. That access still requires lifecycle status, stale blocking, and document-level denies on sensitive subsets. Separate applications for sales, support, and admin contexts so policy changes in one lane do not silently affect another.

Internal users notice wrong answers too. They may have more context to spot errors, but they also act on copilot output in tickets and customer emails. Freshness and conflict checks matter internally because a wrong escalation policy hurts real accounts. Deny rules on compensation, legal strategy, and pre-disclosure security notes remain appropriate even when the audience is employees.

Some articles exist in public and extended internal versions. Do not point both audiences at one document with hidden sections. Split collections or use publishing rules so the public variant is the only one in public allowlists. Internal-only paragraphs belong in internal collections, not in comments hoping retrieval ignores them.

A common mistake is maintaining one master doc with a note that says internal only at the top. Retrieval and summarization do not reliably respect those markers. Duplicate structure with explicit public and internal variants, link them as related documents, and keep permissions aligned with the variant each AI application should use.

Run the checklist once per AI application, not once per company. Your public support bot might score twenty-four of twenty-five while your docs assistant scores nineteen because publishing health lags. That is useful information. It tells you which launch to delay and which collection needs a review queue this week.

Before any public AI ships: named owners on every collection it uses; verified status on high-risk docs; public allowlists with internal collections blocked; Access Simulator pass for your top leakage probes; answer tests on pricing, refunds, and security questions; defined refusal when no source exists; review dates on high-change collections in Knowledge Health.

These seven are the minimum bar because they cover accountability, eligibility, and behavior. Missing any one leaves a class of incident on the table. No owner means stale content persists. No allowlist means accidental breadth. No answer tests means you learn from customers instead of from Answer Playground before launch.

Expand saved test cases from production questions. Track knowledge gaps from no-source replies. Enable regression runs after doc edits. Align Docs Sites publishing health with AI collections. Add document-level denies where imports surfaced edge cases.

Launch week traffic reveals questions your test suite missed. Treat those as inputs, not as one-off fixes. Each repeated no-source question becomes a checklist failure in the monitoring group until a verified doc exists and a test case guards it. Regressions after doc edits catch the case where someone improved a source but broke citation paths or collection membership.

Quarterly audits where every checklist group gets a score. Conflicts resolved within SLA. Policy health green for each application. Publishing drift caught before support tickets spike. That state is reachable without custom engineering if governance lives in one workspace instead of spreadsheets.

Mature teams also tie checklist results to release gates. A product area does not enable AI answers for a new feature until the Feature collection passes ownership, freshness, permissions, and answer tests for that feature's top questions. Docs publishing waits on verified status. That rhythm slows reckless launches and speeds confident ones because stakeholders see green checks instead of debating whether we tested it.

The first run establishes baseline gaps. The second run, thirty days later, should show fewer failures in monitoring and freshness because review queues became habit. By the third quarterly audit, the checklist is mostly confirmation: owners still assigned, regressions still run, Access Simulator still part of permission changes. That is the goal. The list is not a launch artifact. It is how you prove knowledge stayed governed while the product kept shipping.