Agents

The agent did not use my document

Work through the five things that stop a knowledge source reaching a turn, in the order they fail most often.

The agent answered without using a file you know is in the library. Five things can cause that. Check them in this order, because that is roughly how often each one turns out to be the answer.

What you need

  • The agent, open on its Knowledge tab.
  • The source you expected it to use, and the question you asked.

Steps

Step 1: Check retrieval is on for this agent

Retrieval starts off on a new agent. Until you turn it on, the agent ignores every source you selected, however many there are.

On the agent's Knowledge tab, look at the banner. Retrieval is on means the agent pulls passages each turn. Retrieval is off means it does not, and the portal spells out the consequence: This agent ignores all selected knowledge, so nothing below is used.

If it is off, open Retrieval Settings, turn on Enable retrieval and click Save.

Step 2: Check the source is selected for this agent

Being in the library is not enough. The agent uses only what it selected.

Still on the Knowledge tab, look under Folders and Individual sources. If the source is not listed, use Add New and Choose From Library to add it, or add the folder that contains it.

There is a trap here. A source that lives inside a folder cannot be selected on its own, and moving a source into a folder removes it from every agent's individual picks. If a file stopped working the same day somebody tidied the library, this is why. Select the folder instead.

Step 3: Check the status is Ready

Only sources at Ready are used. Anything at Uploading, Pending, Crawling or Indexing is not there yet, and anything at Failed never will be.

Read the status on the row. For a selected folder, the row reads All ready only when every source inside it is usable, and otherwise shows how many are still indexing.

If it says Failed, read the reason on the row and fix the source. A source stuck for over an hour fails on its own with Indexing timed out. Try again.

Step 4: Check the relevance threshold

A ready, selected source is still filtered at the turn. Retrieval scores every passage against the caller's question and keeps only those at or above Relevance threshold, up to Top documents of them.

The default threshold is 0.7, which is deliberately strict. A question phrased differently from the document scores below it and nothing comes back, which looks exactly like the document is missing.

Open Retrieval Settings and try the two dials:

Symptom What to change
The agent never uses a source that clearly covers the question Lower Relevance threshold in steps of 0.05
The agent finds the right source but misses part of a long answer Raise Top documents
The agent quotes irrelevant material Raise Relevance threshold

Top documents defaults to 5 and the slider runs from 1 to 20. The agents API accepts up to 50.

Step 5: Check the content is really in the file

The last cause is the document itself. Retrieval can only match text that was extracted from it.

Look at the Chunks column. A source that is Ready with a very low chunk count gave up almost no text. The usual culprits are a scanned PDF with no text layer, a slide deck where the words are inside images, and a web page that renders its content with JavaScript, leaving an empty shell to fetch.

Download the file and check you can select the text in it. If you cannot select it, it could not be extracted either. Re-export it with a real text layer, or paste the relevant text in as a text source.

Get it right

Move the threshold in small steps and re-test after each one. Dropping it to the floor makes every source match every question, so the passages that reach the turn stop being the relevant ones.

Recommended

  • 0.7, then 0.65, then 0.60, asking the same question after each change.
  • Raise Top documents from 5 to 8 when the answer is right but incomplete.

Less effective

  • Setting Relevance threshold to 0 and Top documents to 20 to be safe.
  • Changing both dials at once, so nothing tells you which change helped.

Worked example

A caller asks "how long do I have to send something back" and the agent says it does not know, although the returns policy is Ready and selected.

You lower Relevance threshold from 0.7 to 0.65, save, wait a few seconds, and start a new conversation. Same answer. You ask instead "what is the return window", which uses the document's own words, and the agent answers correctly: the gap is vocabulary, not configuration. You paste a short source holding the caller's phrasing, it indexes in seconds, and the original question is answered on the next conversation.

What happens next

Adding, removing or replacing an agent's knowledge applies to the next conversation at once. Changing Enable retrieval, Top documents or Relevance threshold, and a source turning Ready, can take up to a minute to reach a new conversation. A conversation already running keeps the settings it started with.

So after any change here, wait a few seconds, then start a new conversation to test it. Re-asking inside the conversation you already have will not pick it up.

Limits

Limit Value Applies to
Top documents Default 5, slider 1 to 20, API 1 to 50 Per agent
Relevance threshold Default 0.7, range 0 to 1, slider steps of 0.05 Per agent
Usable status Ready only Every source
When a change applies The next conversation, up to a minute for retrieval settings Per agent

Troubleshooting

It worked yesterday and not today

Check the source still exists in the library. Deleting a source removes it from every agent that used it, and the agent side shows nothing afterwards. Check the folder too, since deleting a folder deletes its sources.

The agent uses one source but ignores another

Both are competing for the same Top documents slots. If one document consistently scores higher, the other never makes the cut. Raise Top documents, or split the losing document into a smaller, more focused source.

The agent invents an answer instead of saying it does not know

Retrieval returning nothing does not stop the model answering. That behaviour comes from the agent's Personality, not from knowledge. Add an instruction there telling it to say it does not know when it has no source. See How the prompt is assembled.

Everything is Ready and selected but the agent still misses it

Ask the question using words that actually appear in the document, as a test. If that works, the gap is vocabulary rather than configuration: lower the threshold, or add the caller's phrasing to the document.

FAQ

Does the agent tell me which source it used?

Not on the knowledge screens. Compare the answer against the documents you selected.

Does turning retrieval off delete anything?

No. It only stops the agent using its selection. Every source and every pick is left as it is.

Is retrieval on by default for a new agent?

No. A new agent starts with retrieval off, Top documents at 5 and Relevance threshold at 0.7.

Does adding knowledge to one agent affect another?

Only if they share a source or a folder, and only because the library is shared. Selections are per agent.

Is there a limit on how many sources one agent can select?

No limit on the selection. The practical limit is Top documents, since only that many passages reach any one turn.