What is agentic RAG?
Agentic RAG lets the model decide whether to retrieve, what to retrieve, and whether to retrieve again after reading what came back. Better on questions whose evidence is not knowable up front, and slower, costlier and much harder to evaluate.
Agentic RAG is retrieval-augmented generation in which retrieval is a decision the model takes, rather than a step the pipeline performs on its behalf. Classic RAG is a fixed sequence: embed the question, fetch the nearest passages, paste them in, generate. It runs identically every time, and it retrieves exactly once. Agentic RAG hands the model the control flow — it decides whether to retrieve at all, what to ask for, which source to ask, and, having read what came back, whether to go again.
What the model gets to decide
Whether to retrieve. Not every question has evidence behind it. A one-shot pipeline searches anyway and puts three irrelevant passages in front of the model, which is worse than an empty context.
What to retrieve. The question as typed is often a poor query. Rewriting it, or breaking it into sub-questions that each have a findable answer, is the cheapest improvement in retrieval.
Where to retrieve from. Given a ticket search, a CRM lookup and read-only SQL over a warehouse, choosing is part of the work — and an agent chooses worse the more tools it has.
Whether what came back is enough. This is the step that defines the category. The model reads the result, judges it thin or off-target, and issues a different query. Multi-hop questions need it structurally: who approved the BigCo discount takes the deal record first, then a name, then that person — and the second query cannot be written until the first has returned.
The questions it is for
The honest test is whether the evidence a question needs is knowable before you look.
What does our refund policy say about damaged goods is knowable: one query, one passage, done, and a loop only adds latency. Which of our top twenty customers has nobody spoken to since August is not knowable, because the second half of the question depends on the answer to the first. No amount of embedding quality lets a single fetch stand in for a sequence.
What it costs
- Latency: several sequential round trips rather than one, which makes a four-minute answer a different product from a four-second one.
- Variable cost, harder to manage than high cost. The number of retrievals per answer is no longer fixed, so neither is the bill.
- Non-determinism. The same question asked twice may take different routes and read different records.
- Evaluation, where most of the real difficulty lands. Recall at k assumes one retrieval over one index; when the loop decides for itself how many times to retrieve, there is no k. What has to be assessed instead is the trajectory — was the rewrite faithful, was the tool the right one, did it stop too early, did it keep going after it had enough. It is the main reason agentic systems that demo well plateau in production.
- Compounding errors. A bad rewrite on the first step sends every later step after the wrong thing, fluently.
- A filling context window. By the fifth retrieval, the material from the first is competing for attention with the question.
A loop over a weak index does not buy a better corpus: it rediscovers the same wrong passages, more slowly and at greater expense.
Why this converges on tools rather than an index
A model that retrieves once can be served by a pre-built index. A model that retrieves repeatedly needs something it can interrogate with parameters — fetch this ticket, now look up that email address in the CRM, now total the orders against this account id — and that is not search over a corpus. It is calls against live systems, which is to say tool calling, which is why agentic RAG and MCP turned into the same conversation.
It also reopens two things a nightly index quietly settled: freshness, because a tool call reads what is true now rather than what was true at the last crawl, and permission, because a call made on somebody's behalf can return only what they are allowed to see, where an index had to approximate that with filters.
Where it stops
It cannot find what is not connected, and a loop with nothing to find will spend its whole budget establishing that.
It cannot settle identity by trying harder. If the ticket, the invoice and the CRM record for one customer share no key, more retrievals produce more unjoined fragments. That is the other half of the problem, and the half GraphRAG addresses: the loop decides what to fetch, the graph decides what is the same thing.
And it is a poor fit wherever the answer has to be reproducible — a figure in a board pack, a regulated disclosure. "It usually finds it" is not a property anybody can sign off.
Where Memtro fits
Agent mode is this pattern with the budget made explicit: the model plans, works through tool steps (thirty by default), delegates independent sub-tasks to workers running in parallel, pulls the identifiers out of one record and looks them up in the other connected systems, and stops when more calls would not change the answer. Two of its habits exist because of the costs above: every answer carries the records it drew on, and a run ends by naming each system it consulted, including the ones that had nothing to say.