> For the full page index, fetch https://echonova.ai/llms.txt

# The retrieval bench

Check whether an agent would reach an imported API operation for something a caller might say, and understand what the bench deliberately does not do.

The bench answers one question: would this agent reach this API operation if a caller
said that? It runs the same search a live turn runs and shows you what came back and
how strongly. It is a retrieval test, not an execution test, and the difference
decides how you read the result.

## What you need

- An agent, opened on its **Tools** tab. The bench is scoped to one agent, so the
  workspace **Tool Library** page does not offer it.
- An imported OpenAPI tool whose status is **Ready**, selected on that agent.
  Nothing else is searched. A database connection has its own bench, described in
  [Review and approve database questions](/docs/agents/database-tools-templates).

## Steps

### Step 1: Open the tool from the agent

On the agent's **Tools** tab, open the tool. Its detail view has an **Overview**
tab, an **Endpoints** tab and, once the tool is **Ready**, a **Tests** tab.

### Step 2: Open the Tests tab

The **Tests** tab holds the bench, scoped to this tool. Once a run has been recorded,
a line above the box shows "✓ Matched" or "× No match" with the query you ran.

### Step 3: Type what a caller might say

The box takes plain speech, not an endpoint name. The placeholder is "e.g. when is
the next flight?" and the hint above it explains the point: "Type what a caller
might say. This runs the same semantic search the agent runs each time, so you can
detect an incorrect match before launch."

Four suggestion chips sit under the box: "when is the next flight", "cancel my
booking", "what is my baggage allowance" and "change my seat". Pressing one fills
the box and runs immediately.

### Step 4: Run it

Press **Run** beside the box, or press Enter in the box. The button reads
**Testing…** while the search is in flight.

## Get it right

The bench compares your sentence with the operation's description, so type the
sentence a caller would actually say. Typing the operation's own name proves nothing:
it matches itself.

**Recommended**

```text
my parcel hasn't turned up and I ordered it last Tuesday
```

**Less effective**

```text
getOrderShipmentStatus
```

## What happens next

EchoNova takes the tools selected on this agent, keeps the imported OpenAPI ones
that are **Ready**, narrows that to the tool you opened, and returns the operations
that best match your text. What comes back is a ranked list, not a call.

Under **Results** each row shows a signal-bar glyph, the HTTP method, the operation's
path, the tool it belongs to, and a score as a percentage. The score is a similarity
rounded to a whole number and clamped between 0 and 100. It measures the distance
between your wording and the operation's description. It is not a probability, and
it is not comparable across different queries.

Nothing above the threshold is a result in its own right, and the bench says so:
"There is no match above the specified minimum threshold. The agent will ask a
clarifying question instead of calling an endpoint."

## What the bench does not do

- It does not call your API. No request reaches the operation's URL, and your server
  logs will show nothing.
- It sends no arguments. You give it an utterance, not a request body, and no
  parameters are filled.
- It does not use the agent's language model. Matching compares text, and the model
  you chose for the agent is not involved.
- It does not create a conversation. No session is opened and no turn is recorded,
  so nothing appears under the agent's conversations.
- It does not consume credits. Credits are charged for turns in a live conversation,
  and a bench run is not a turn.

So a green result means the agent would reach the operation, not that the operation
works. Auth, base URL, request shape and your endpoint's own behaviour are all
untested here. Verify those separately, then talk to the agent in the playground to
see the two halves work together.

The same limit applies in reverse to the tool kinds that are never searched. A
webhook, integration or system tool is in front of the agent on every turn, so there
is no retrieval decision to test and the bench does not score it.

## What the run records

A run's outcome is kept per tool: whether it matched, the best score, the exact query
and the time. Only the tools that run searched are updated, so a scoped run leaves
the other tools' results alone.

You see it in two places afterwards.

| Where                                                 | What it shows                                                                            |
| ----------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| The **Last tested** column of the agent's tools table | How long ago it ran, or "Never"; hover shows the query and the top score as a percentage |
| The **Tests** tab of the tool                         | "✓ Matched" or "× No match", with the query                                              |

The result survives a reload, and it belongs to the agent that ran it. What is
searched depends on what that agent has selected, so the same tool can surface for
one agent and not another.

## Limits

| Limit               | Value                                                               | Applies to        |
| ------------------- | ------------------------------------------------------------------- | ----------------- |
| Query length        | 500 characters                                                      | Every run         |
| Tools searched      | Imported OpenAPI tools with status **Ready**, selected on the agent | Every run         |
| Scope in the portal | One tool, the one you opened                                        | The Tests tab     |
| Hits returned       | Up to 3, each scoring 0.5 or higher (a live turn keeps up to 5)     | Every run         |
| Run timeout         | 30000 ms                                                            | Every run         |
| Results kept        | One per tool, replaced by the next run that searches it             | The agent         |
| Permission          | Workspace membership, nothing further                               | Running the bench |

The number of hits and the score they must reach are not adjustable from the portal
today.

## Troubleshooting

### "Could not run the bench."

The run failed rather than returning nothing. Retry it once. If it keeps failing,
wait a moment and try again: a tool that has only just finished importing may not be
searchable yet.

### The result is empty for every query, including the presets

Nothing was searched. Check two things in order: the tool's status is **Ready**
rather than **Importing** or **Failed**, and the tool is selected on this agent
rather than only present in the workspace library. A run with nothing to search
returns an empty result rather than an error, which reads exactly like a bad match.

### There is no Tests tab

The bench is offered for imported OpenAPI tools that are **Ready**, opened from an
agent. A webhook, integration or system tool has no retrieval step to test. A
database connection has its own bench instead, and only once its questions have been
drafted.

### The bench matches but the agent still does not call the operation

Retrieval is one of two gates. The operation surfaced, so the wording is fine; what
is left is the agent's decision and the operation's own state. Check that the
operation is enabled, and that its description says when it should be used.

### The score looks low but the agent behaves correctly

Treat the percentage as a ranking aid rather than a pass mark. What matters is
whether the operation you expected is in the list and above the ones you did not
expect.

## FAQ

### Does a bench run cost anything?

No. It records no turn and consumes no credits.

### Does the bench hit my endpoint?

No. It searches the descriptions of your operations, not the operations themselves.

### Why is my webhook tool not in the results?

It is never searched. Webhook, integration and system tools are in front of the
agent on every turn, so they are not ranked and carry no score.

### Can I test how the agent chooses between two tools?

Not from the Tests tab, which always scopes to one tool. Talk to the agent in the
playground instead, where every tool it has competes as it would in a real
conversation.

### Does the recorded result change when I edit the tool?

No. It holds the outcome of the last run and its query. Run it again after an edit
to refresh it.

### Do bench results follow the tool to another agent?

No. The result belongs to the agent that ran it, because what is searched depends on
that agent's selected tools.

## Related

- [OpenAPI tools](/docs/agents/openapi-tools): import the API this bench searches
- [OpenAPI tool operations](/docs/agents/openapi-tool-operations): fix the description a poor score points at
- [Review and approve database questions](/docs/agents/database-tools-templates): the same test for a database
- [Test your agent](/docs/agents/test-your-agent): the other half, in a real conversation
- [Tools overview](/docs/agents/tools-overview): which tool kinds are searched and which are not