Building a searchable business knowledge base.

Goal: Build a small knowledge base from one topic’s documents, give it answering rules, and test it with 10 questions before anyone else uses it. Who it is for: Office managers, HR and health and safety leads, and team leaders who answer the same internal questions every week. Time: 45 minutes, plus 10 to 30 current documents on one topic and an AI tool with a project or notebook feature.

The question “where is the policy on that?” costs a small business more than it looks. A new starter asks the office manager, who searches a shared drive, finds two versions, and answers from memory. An AI knowledge base can answer in seconds with the document and section named. It can also answer confidently from the wrong version. The difference is almost entirely in the preparation.

What makes a knowledge base answer well

A knowledge base answers well when its documents are current, singular and owned, its rules force it to cite and to admit gaps, and it has been tested on questions with known answers. The AI model matters less than any of these.

As of October 2026 the common starting points are Projects in ChatGPT and Claude, Copilot Notebooks, and Gemini Notebook (formerly NotebookLM). Feature names and file limits change, so check yours.

Exercise: build and test a 10-question knowledge base

1. Pick one topic and gather the documents. Choose something staff ask about often, such as workshop safety and staff policies. Collect 10 to 30 documents.
2. Clean before you load. Keep one current version of each. Add an owner and review date to the first page if missing. Convert scanned pages to text. Remove anything not every user may see.
3. Write 10 test questions first. Use real questions staff have asked. Write the correct answer and source for each. Include two questions the documents do not answer.
4. Create the project or notebook. Load the documents and paste in answering rules like those in the worked example.
5. Run the test and score it. Ask all 10 questions in fresh chats. Mark each: correct with the right source, wrong, correctly said “not covered”, or invented an answer.
6. Fix documents first, then rules. Most failures trace back to an outdated or ambiguous document. Fix it, reload, and rerun the failed questions.
7. Name an owner and a review date. Someone must update the knowledge base when a policy changes.

Worked example: a workshop safety and policy assistant

Illustrative example: a 30-person Nelson steel fabrication workshop loads its health and safety manual, induction checklist, PPE procedure, vehicle policy and leave policy. The business and documents are fictional.

You answer staff questions for Tasman Fabrication Ltd using ONLY the
attached documents.

Rules:
1. Every answer names the document, version and section it came from.
2. If the documents do not answer the question, say "The documents don't
   cover this. Ask the office manager." Do not use general knowledge.
3. If two documents disagree, quote both and say which has the later
   review date.
4. For any safety procedure, quote the documented steps exactly. Never
   add, remove or reorder a safety step.
5. Do not give legal, medical or employment advice beyond what the
   documents say.

Test question: “I had a near miss on the press brake. What do I have to do?”

Example output (illustrative, abridged):

The Health and Safety Manual (v4, section 6.2 "Near misses") says:
1. Make the machine safe and tell your supervisor before the end of
   the shift.
2. Record it in the incident register within 24 hours.
3. Your supervisor reviews it at the next toolbox meeting.
Source review date: March 2026. Owner: Operations Manager.

Test question: “Can I bring my dog to work?” Correct behaviour: “The documents don’t cover this. Ask the office manager.” If it answers from general knowledge instead, tighten rule 2 and rerun.

Where knowledge bases go wrong

• Old versions win. Two leave policies, two answers. The control is cleaning before loading and a named owner who removes superseded files.
• Over-sharing. A simple project shows everything to everyone with access, so a question can surface content from a file the asker could never open. Keep sensitive documents out until you have per-user permissions, which our SharePoint and Google Drive guides discuss.
• Missed answers. The answer exists but is not found, usually in very long or poorly structured documents. Clear headings help. At larger scale you need retrieval designed and measured, covered in understanding retrieval augmented generation and our RAG development work.
• Paraphrased safety steps. Health and safety duties under the Health and Safety at Work Act 2015 sit with the business, not the tool. Quote procedures exactly and link the source.
• Treated as authority. Staff stop checking the source. Keep the citation rule so the document stays the authority.

PainPal with CHASNZ shows the same principle in a product: an AI workmate that answers from approved guidance rather than open-ended general knowledge. For a private search system across many documents, see building private AI search.

Self-check

• The knowledge base covers one topic with 10 to 30 current documents.
• Every document has one version, an owner and a review date.
• Nothing in it is restricted to fewer people than will use it.
• The rules require a cited source and a “not covered” answer when needed.
• You ran 10 known questions, including two it should not answer, and recorded results.
• Failures were fixed in the documents first and the questions rerun.
• A named person owns updates and reviews.

Next lesson: Using AI to prepare management reports.

Written by the Levy Street team in Wellington. Last updated 19 October 2026.