AI document assistant: which procedure applies?

An AI assistant for internal documents can find an answer in your procedures if it shows the passage supporting that answer, the applicable version and its effective date. It must also limit its search to documents the person is allowed to read. I suggest six cases to check before giving your team access, including cases where the right response is to stop.

'Do we still need the end-of-session questionnaire?' The procedure exists, but a colleague kept the old version. A well-written answer doesn't resolve that problem: you need to know which document applies. That is the starting point for this article's fictional scenario.

How does an assistant find an answer in your documents?

A document assistant searches a set of files for passages, then uses them to write its answer. This method is often called retrieval-augmented generation, or RAG. For the team, a useful result is simple: a short answer, a relevant excerpt and a link that opens the right source.

France's data protection authority, the CNIL, describes this approach as a way to enrich answers with external data that is easier to update than the model itself. It also points out that a model can produce an answer that sounds plausible but is wrong. CNIL questions and answers, published July 18, 2024 (in French).

• The answer: what your coworker needs to understand or do.
• The evidence: the exact passage, with its title, version, effective date and section.
• The limitation: what is missing or still needs confirmation from the person responsible.

A link to a document doesn't prove that the document supports the answer. Open the cited passage: does it cover the same file, situation and applicable rule? Reading amounts automatically addresses a different need, covered in my guide to reading supplier invoices.

---

Which version of the procedure should apply?

An answer that guides an action must rely on the approved version that applies to the situation. I ask for that rule to be made explicit before connecting the assistant. A file's modification date is not enough: a draft saved yesterday can sit alongside a procedure that is still in force.

Here is the small set of documents used to prepare the checks. All titles, rules and dates in this scenario are fictional. They describe neither a training-sector requirement nor a client procedure.

The five documents in the scenario

An old procedure can explain a change without becoming the rule to follow. To answer the questionnaire question, the assistant should cite B, version 2, section 3, effective November 1. If asked why the list changed, it can use A to explain the difference, clearly marking it as obsolete.

---

How do you check permissions before documents are read?

The application must identify the person and apply their permissions before retrieving excerpts for their question and sending them to the model. An instruction such as 'don't reveal the management file' doesn't replace that check. The role must come from the signed-in account, not from a sentence typed into the conversation.

OWASP, an organization that publishes IT security guidance, recommends preserving permissions on excerpts and checking them at retrieval time, before the content reaches the model. Microsoft also documents filters that exclude unauthorized documents in Azure AI Search. These mechanisms need to be configured and checked. OWASP recommendations and document access controls described by Microsoft.

'The AI system must take access rights and the need-to-know principle into account in the answers it gives.'
ANSSI, France's cybersecurity agency, guide dated April 29, 2024, page 13. Translated from French. Read the guide (in French).

An answer that hides the code doesn't prove that the restricted document was excluded. To check, ask for a record of the selected documents and the content actually sent to the model. The displayed title, link and excerpt must also respect permissions: a citation can disclose information.

---

What should you test before giving your team access?

Ask the same questions using two test accounts, one with an 'administration' role and the other with a 'management' role. For each case, compare the expected result with the selected documents, the answer and the citations. This table describes acceptance checks to request before signing off on a delivery. It does not report any software's performance.

Six cases and their expected results

What does the local check in this example show?

On November 11, 2026, a small local program with no AI reproduced the planned document selections for these six situations, using French questions and documents. The roles were simulated. For the 'administration' role, file E was not opened. This illustrates a check performed before reading these fictional files; neither user authentication nor AI-generated answers were evaluated.

Rewording a question also revealed a limitation. A French question meaning 'What needs to be saved to finish the file after the session?' found no source, even though B contains the answer. This shows why acceptance checks need different phrasings. The six cases above still need to be checked in the chosen solution, using real accounts and its final answers.

A contradiction between sources calls for a decision from the person responsible. The assistant shouldn't name the administration manager simply because that answer seems more plausible. It should enable the person who maintains the procedure to correct C or D.

Keep the date, solution and settings with the observed results. Run the questions again after changes to sources or permissions, especially after access is removed. Passing this small scenario does not measure overall reliability. To include these checks in a request for pricing, see my guide to automation quotes.

---

What happens to documents and questions?

Check model training, conversation retention and deletion of copies used for search separately. A commitment not to train a model on your data doesn't mean nothing is stored. The rules depend on your subscription, settings and connected services.

For example, Microsoft says Copilot stores questions and answers, including their citations, in activity history. Retention policies can apply even though the data is not used to train foundation models. Microsoft documentation on stored data.

• Where does the data go? Ask for details of hosting, model processing and the providers involved.
• Who can see the history? Check access for users, administrators and anyone providing support or services.
• What does deletion remove? Check the file, its search excerpts and saved answers against the chosen retention rules.

Removing the original file isn't enough if its excerpts remain searchable. OWASP recommends propagating deletion to derived copies and responses cached for reuse. Logs must be handled according to the applicable retention requirements. OWASP recommendations on deletion.

---

What questions come up before a first trial?

Start by checking the search features already available in your tools and the quality of your procedures. An assistant becomes useful when the team needs an answer that brings together the right passages, with visible evidence and limitations.

---

What should you remember before getting started?

Choose one department's procedures and questions its manager can verify. Prepare the expected answers, identify the applicable version and test two access profiles. If a regular search already opens the right passage without difficulty, it may be enough.

Three decisions before connecting the assistant

To plan an internal tool connected to your documents, you can tell me about your procedures and your team's questions. For an initial conversation, I only need the department, an example question and the roles allowed access. Keep confidential files in your usual workspaces.