Skip to main content
An AI Experience should help visitors understand your expertise and make informed decisions. If an answer is inaccurate, incomplete, outdated, or unhelpful, use this guide to identify the likely cause and improve it.
A successful document upload does not guarantee that every answer will use the right information. Response quality depends on the source material, retrieval, system instructions, model behavior, and the visitor’s question.

​
Start by identifying the problem

​
1. Reproduce the issue

Before changing anything, capture a specific example.
  1. Open your AI Experience and start a fresh test conversation.
  2. Ask the same question that produced the problem.
  3. Record the question, the answer, and what you expected instead.
  4. Try one or two alternative phrasings to see whether the issue is consistent.
  5. Note whether the issue happens only after earlier conversation turns.
A simple test log helps you distinguish a repeatable problem from an isolated response.

​
2. Verify the source information

Find the document that should support the answer. Ask:
  • Is the relevant information present and factually correct?
  • Is it stated clearly, with enough context to interpret it?
  • Is the document current?
  • Does another uploaded document contradict it?
  • Is the answer dependent on live data the assistant cannot access?
If the answer is missing from the knowledge base, add or improve the source material. Do not rely on the assistant to infer business-specific facts.
Write direct statements such as “We do not provide real-time inventory through this assistant” rather than expecting the AI to infer that limitation from an absence of inventory files.

​
3. Improve document structure

When the right information exists but is not used consistently, review how the document is written.
  • Give each document a clear title and focused purpose.
  • Use descriptive headings that match real visitor questions.
  • Keep related concepts together and explain unfamiliar terms.
  • Avoid large blocks of repetitive or contradictory text.
  • Include concrete examples and common scenarios.
  • Prefer accessible, text-based content over information embedded only in images or complex layouts.
For example, a document titled Buying a Used Boat — Inspection and Due Diligence is easier to understand and maintain than a miscellaneous file containing unrelated sales notes, inspection advice, and outdated listings. See Organize Your Knowledge and Knowledge Best Practices.

​
4. Check system instructions

System instructions govern how the assistant should behave; uploaded knowledge provides information to draw from. Review the instructions if the problem concerns:
  • The assistant’s role, tone, or intended audience.
  • Whether it should ask follow-up questions.
  • How it should respond when evidence is missing.
  • Topics it must avoid or treat as educational only.
  • When to recommend a human conversation.
  • Whether it is allowed to claim current availability, prices, or other changing facts.

​
Example: prevent invented availability

Do not use system instructions as a substitute for a complete knowledge base. Keep detailed educational content in source documents and behavioral rules in the instructions. See System Instructions.

​
5. Handle outdated or conflicting answers

When business details change, old material can become misleading.
  1. Identify every document containing the outdated statement.
  2. Update or remove obsolete versions using the knowledge management options available in your workspace.
  3. Make sure the current information is stated consistently.
  4. Review system instructions for any hard-coded facts that also need updating.
  5. Retest the original question and several related questions.
Do not assume a newly uploaded document automatically overrides an older conflicting document. Remove or reconcile obsolete information before relying on the new version.
See Update Your Knowledge.

​
6. Test retrieval separately from conversation behavior

Use different tests for different failure types.

​
Knowledge accuracy

Ask a question with a clear answer in one uploaded document. Verify the assistant gives the correct answer without adding unsupported details.

​
Missing information

Ask about something that is not documented. The assistant should acknowledge uncertainty or the limitation rather than invent a confident answer.

​
Conflicting information

Ask about a topic that previously appeared in more than one document. Confirm that the updated answer is consistent.

​
Methodology

Ask the assistant to explain how your approach works in a realistic client situation. It should reflect your documented methodology rather than generic advice.

​
Human handoff

Express a clear intention to proceed with a service, request a live quote, or speak with an expert. Check that the assistant introduces the appropriate next step without claiming it completed an action it cannot perform.

​
Privacy and sensitive topics

Ask a question that might invite disclosure of confidential or sensitive information. The assistant should respect your documented boundaries and avoid requesting unnecessary personal details.

​
7. Troubleshoot common patterns

Confirm the information is in readable text, under a descriptive heading, and not buried in a scanned image or complicated table. Try a more focused document and retest with several phrasings.
Some variation is normal. Check for contradictory documents, ambiguous instructions, and questions that lack necessary context. Evaluate factual consistency rather than requiring identical wording.
Verify whether those facts are present in approved knowledge. Add explicit limits for unavailable or frequently changing information, and direct visitors to a human when current verification is required.
Adjust response-style guidance in system instructions. Specify a helpful default, such as concise answers with optional detail when the visitor asks for it.
Review instructions that require information gathering before every answer. Allow the assistant to answer straightforward questions directly and ask only the clarifications needed to help.
Narrow the handoff triggers. The assistant should provide useful guidance first and suggest a human conversation when the visitor expresses relevant interest or needs assistance beyond its scope.

​
8. Make one change at a time

For reliable troubleshooting, change one variable and retest before making another change. For example:
  1. Clarify an ambiguous source paragraph.
  2. Repeat the original test questions.
  3. If the problem remains, review the relevant system instruction.
  4. Retest in a fresh conversation.
  5. Record the outcome and retain the improvement that works.
Changing documents, instructions, and model settings simultaneously makes it harder to identify what solved the problem.

​
Final response-quality checklist

  • The answer is supported by current, approved knowledge.
  • The assistant does not invent facts, prices, inventory, or capabilities.
  • Missing information is acknowledged appropriately.
  • The assistant reflects the business’s actual methodology.
  • Response length and tone suit the visitor’s question.
  • Privacy and sensitive-information boundaries are respected.
  • Human handoff happens when appropriate, not by default.
  • The issue has been retested using realistic variations of the original question.
If an issue persists after reviewing the documents and instructions, record the steps to reproduce it and contact Agentsy support through your available support channel. Include the affected AI Experience and the relevant test question, but do not share passwords or sensitive visitor information.