From AI hype to everyday impact: Smart questions better answers

What’s the use case?

Companies are often required to fill out questionnaires about their products and services. Yet the answers needed to complete the questionnaires are usually spread across many documents and in the minds of department experts.

What if AI could bring all that knowledge together? The vision: to build an app that can answer questions about a company, its products, and services, using sets of documents as reference.

At its core, this approach applies Retrieval Augmented Generation (RAG), a method that allows AI to combine the precision of factual retrieval with the fluency of natural language generation. The goal is to create a RAG-based system that works reliably – with built-in checks and approvals during the creation process.

What does the workflow look like?

Rather than simply uploading the required documents and starting a chat building process, we recommend taking a more controlled and structured approach.

In theory, one could create an agent in MS Copilot Studio, upload documents and a prompt, and deploy the app directly. From our experience, this will produce a basic demo, but not a dependable and real solution.

The quality of the results depends heavily on the complexity of the source documents, and the ability of the model to comprehend the content. A more deliberate setup ensures that AI becomes a reliable partner that reflects the depth, accuracy, and trust your organization stands for.

The approach is this:

1. Building the foundation

Start by creating an intermediate agent in Copilot Studio. All documents belonging to a specific department or topic area (for example, data privacy or application documentation) can be uploaded into its knowledge base with clear, unique names, e.g. A.01-xxx, A.02-xxx.

Next, a prompt needs to instruct the agent to:

• read a single document starting with the prefix entered by the user, e.g. A-01.
• generate a list of questions and answers derived from that single document
• display the findings in the chat window
• convert the questions and answers into a format that we could download per document, CSV or JSON format.

This procedure needs to be repeated for each document. The result: a set of structured question–answer documents that capture the essence of the organization’s knowledge.

While it might be tempting to create another agent to compile these results or to process all documents simultaneously, our experience shows that such approaches led to lower-quality output.
Instead, we recommend compiling the extracted questions and answers into one source document (an Excel spreadsheet for example) and to repeat this process for all documents belonging to a single department and expertise group. Then repeat this process to generate another source document with information stemming from another department, until all of the resource documents from all departments are collected.

2. Human review and validation

Before any AI-generated content becomes “official,” it must pass through expert hands. Each source document or spreadsheet should therefore be shared with the responsible persons to review the generated content and correct any errors or inconsistencies. Once reviewed, the sheets can become trusted input material and the basis for the actual agent. This step adds an important layer of human quality assurance to the workflow as the results combine the efficiency of AI with the knowledge of human experts.

3. Building the actual “Questionnaire Expert”

With verified material in place, a new agent can be built. The validated source spreadsheets can be uploaded to its knowledge base and marked as ‘Official’ to indicate higher trust and priority for these documents. Original source documents can also be included as references but unofficial ones.

The prompt for this agent should then define clear rules:

• Answers should be provided from the official documents first.
• If no answer was found, search in an adjacent official document for a related question.
• If still unresolved, search in the source documentation.

This hierarchy introduces structure and trust into the model’s reasoning — elevating it beyond a basic RAG setup into a reliable, explainable assistant.

4. Deployment

Finally, the agent could be deployed as a MS Teams app to everyone who needs to fill in questionnaires.