What is agent memory?
Agent memory is what an AI agent keeps between turns and sessions beyond its context window: facts, preferences, history. The kinds of memory, how it is built, the hard problems, and why a shared memory differs from a personal one.
A language model has no memory of its own. Each request arrives with a context window, the model answers from what is in it, and when the request ends nothing is kept. Agent memory is everything a system adds around the model so that the next conversation can know what the last one learned.
The term covers several distinct things, and conversations about it go wrong when they are mixed up.
Kinds of memory
Working memory is the context window itself: the conversation so far, the documents pasted in, the results of tools called this turn. It is large but temporary, and it is paid for on every request, which is why pasting the same context into every prompt costs so much.
Short-term memory survives a session: a summary of a long conversation kept so it can continue past the window, or the state of a task an agent is part-way through.
Long-term memory survives everything. Facts about people, companies and projects, decisions and their reasons, preferences, how things are done here. It has to be stored somewhere outside the model and brought back into the window when it is relevant.
Within long-term memory, people borrow three more words from psychology. Episodic memory is what happened: this call, that email. Semantic memory is what is known: the renewal is in September, Priya owns billing. Procedural memory is how to do things: the steps for a refund.
Personal and shared
A memory can belong to one person, to a team or to a whole organisation, and the difference is not a detail. A personal assistant remembers that you prefer short answers and that your daughter's birthday is in May. A company memory remembers that the BigCo contract renews in September and that "settlement" means something particular in your business. Putting the second kind in the first kind of store makes it invisible to colleagues; putting the first kind in the second is a privacy problem. A memory system for a company needs both scopes and a rule for which facts go where.
How it is built
Most systems follow the same loop. After an exchange, an extractor decides whether anything durable was said and turns it into facts. The facts are stored, in a vector index, a database, a graph, or all three. On the next request, a retriever finds the facts relevant to the question and injects them into the context window. Over time facts are updated: a newer fact supersedes an older one rather than sitting beside it, near-duplicates are merged, and what is no longer true is marked as such or removed.
The storage choice shapes what the memory can answer. Vector similarity finds text that resembles the question. A knowledge graph knows that two records are the same customer and what is attached to them. Good systems use both.
The hard problems
- Deciding what is durable. Most of a conversation is not worth keeping. An extractor that keeps everything produces noise; one that keeps too little forgets the thing that mattered.
- Change. Facts go stale. A memory that cannot supersede ends up holding two renewal dates and picking one at random.
- Contradictions. Two systems, or two people, say different things. A memory that merges them silently is worse than none.
- Provenance. An answer drawn from memory should be able to say where each fact came from, or it cannot be checked.
- Privacy and deletion. A request to forget a person has to reach every store the memory touches, and the receipt has to be believable.
- Evaluation. Memory is easy to demonstrate and hard to measure; the useful test is a question whose answer you already know.
Why it matters
Without memory an agent starts every conversation from zero and the person re-explains the company each time. With it, answers improve with use, the context window carries only what is relevant, and tools can be used in the light of what is already known, which is the difference between an assistant and a search box.
Where Memtro fits
Memtro is an organisation-wide memory with a personal scope inside it: a knowledge graph of entities, facts and relations with validity over time, filled from chat, connected systems and imports, and read by any client over the API or MCP. The memory page explains what it holds and the documentation how it works.