Skip to main content

​
Knowledge retrieval problems

Your AI Experience can use uploaded knowledge to answer questions about your expertise, services, and methodology. If an answer overlooks a document, uses the wrong information, or makes an unsupported claim, use this guide to isolate the cause before changing your configuration.
Knowledge retrieval and response generation are related but different. A file may be available to the assistant without the relevant passage being selected for a particular question. Even when the right passage is found, the assistant can still misunderstand it or fail to follow its instructions.

​
Start with a reproducible example

Before making changes, record:
  • The exact question you asked.
  • The answer you received and what was wrong or missing.
  • The correct information and the document where it appears.
  • Whether the issue happens consistently or only sometimes.
  • Any recent changes to files, instructions, or assistant settings.
Then ask the same question in a new test conversation. This helps distinguish a repeatable retrieval problem from one influenced by earlier conversation context.

​
Diagnose the symptom

​
Step 1: Verify the knowledge source

Open the knowledge management area for the relevant AI Experience and confirm that the intended document is available. Check its contents, title, and any status information shown in your account. Make sure that:
  • The document contains the correct and current information.
  • Important passages are readable text, not only screenshots or images.
  • The uploaded version is the version you intended to use.
  • The content belongs to the AI Experience you are testing.
  • Any file-processing status shown in the interface does not indicate an unresolved problem.
Do not assume a document is ready for use simply because you selected it for upload. Check any processing or availability indicators provided by your account before testing.
If the source is a scanned PDF, image-heavy presentation, or complex table, consider preparing a text-first version with clear headings and explicit statements. Preserve the original meaning and verify that no important details were lost.

​
Step 2: Make information easier to find

Retrieval works best when each document has a clear purpose and relevant passages are easy to identify. A useful document structure is:
For example, instead of placing all insurance information in one long file, separate general education, life insurance, health insurance, and fixed annuities into focused documents. Within each, use descriptive headings such as Term vs. Permanent Life Insurance rather than Other Information.
Write important facts as complete, self-contained statements. A passage such as “A used boat should be inspected by a qualified marine surveyor before purchase” is easier to interpret than a short bullet reading “Inspection: recommended.”
Avoid excessive duplication. If several documents give different answers to the same question, the assistant may have difficulty identifying which one is authoritative.

​
Step 3: Check for conflicting or outdated information

Look for conflicting statements across uploaded files, particularly around:
  • Prices, service packages, and promotional offers.
  • Geographic availability and eligibility.
  • Legal, medical, or financial disclaimers.
  • Booking processes and contact information.
  • Product features and operating procedures.
If an older policy has been replaced, update or remove the superseded content where your knowledge management controls allow it. Keep a clear source of truth for frequently changing facts.
Uploaded documents are not the same as a live connection to your inventory, calendar, pricing system, or other external database. Unless a supported real-time integration is explicitly configured, do not expect the assistant to know current values from those systems.

​
Step 4: Review system instructions

Your system instructions should tell the assistant how to use its knowledge and what to do when information is missing. Review whether they clearly establish that the assistant should:
  1. Prioritize relevant, available knowledge over unsupported assumptions.
  2. Acknowledge uncertainty when the documents do not support an answer.
  3. Avoid inventing availability, prices, credentials, policies, or guarantees.
  4. Ask a focused clarification question when needed.
  5. Recommend an appropriate human handoff for requests that require current or individualized information.
Do not try to solve every retrieval issue by adding more instructions. If the underlying file is missing, contradictory, or poorly structured, improve the source first. See System Instructions and Testing Your Assistant for related guidance.

​
Step 5: Test questions systematically

Use a small test set that covers both supported and unsupported requests. Run each question in a fresh conversation and record whether the answer is accurate, complete, and appropriately cautious. Test natural language variations rather than only exact phrases copied from the document.

​
Step 6: Improve one variable at a time

When you find a repeatable issue, make the smallest useful change:
  1. Missing source: Add or update the relevant knowledge document.
  2. Poorly organized source: Rewrite headings and separate unrelated topics.
  3. Conflicting sources: Remove or reconcile outdated information.
  4. Unclear behavior: Refine the relevant system instruction.
  5. Overly broad question: Improve the assistant’s clarification behavior.
Retest the original question and at least two related questions after each change. This makes it easier to understand which adjustment helped.

​
Example: Boat advisory without live inventory

Suppose a visitor asks, “Which 30-foot boats do you have available right now?” If the assistant lists specific boats without access to current inventory, the problem is not simply missing knowledge. The assistant has crossed from educational guidance into unsupported real-time claims. A better response would explain that it can help the visitor compare boat types, define requirements, and prepare for a purchase, while directing current inventory requests to a human broker. To correct this behavior, check both the knowledge documents and the system instructions. Ensure neither implies access to live listings unless such access has actually been configured.

​
When to contact support

If the issue persists after checking your files, instructions, and test questions, gather:
  • Your AI Experience URL or identifier.
  • The affected document name.
  • The exact test question and unexpected answer.
  • The expected answer and supporting passage.
  • The approximate time of the test.
  • Any error messages or relevant screenshots.
  • The troubleshooting steps already attempted.
Do not send passwords, API keys, payment card details, or sensitive visitor information in a support request. Remove or redact personal information from screenshots and conversation excerpts. For further help, see Report a Bug or Contact Support.

​
Knowledge reliability checklist

  • The correct document is uploaded to the intended AI Experience.
  • The content is readable and current.
  • Titles and headings clearly describe each topic.
  • Conflicting or outdated documents have been reviewed.
  • System instructions address uncertainty and unsupported claims.
  • Direct, paraphrased, and boundary-case questions have been tested.
  • Changes have been retested in fresh conversations.
A reliable AI Experience does more than retrieve facts: it uses relevant knowledge accurately, explains limitations, and knows when a human should take over.