"Does the agent remember?" is the wrong question. Memory is not a property an agent has — it is a set of design decisions you make, and getting them wrong causes most of the behaviour people describe as an agent "going off the rails".
Three kinds, three lifetimes
Working memory — the context window during a single run. The goal, the steps so far, tool results. It is everything the model can actually see, and it is finite.
Session memory — persists across a conversation but not forever. What the user said earlier, decisions already made.
Long-term memory — survives across sessions. Stored somewhere real: a database, a file, a vector store. Crucially, it is not automatically available — something must retrieve the relevant part and place it into working memory before the model can use it.
That last point is where intuition usually fails. A model with long-term memory does not "know" things. It knows what was retrieved and inserted this time.
Why long runs degrade
Working memory is finite, and agents fill it with raw material fast. Twelve steps of full API responses, error messages and page content will crowd out the goal.
Three symptoms follow:
- The goal fades. It was stated once, at the start, and is now buried under results.
- Early findings vanish. Something learned at step two is gone by step ten.
- Recent noise dominates. The last tool result gets disproportionate weight simply by being close.
Deciding what to keep
Design memory explicitly, the way you would design a data structure:
Always keep — the goal, restated every iteration; hard constraints; decisions already made and why; anything a later step must not contradict.
Summarise — completed sub-tasks, once they have produced a result. "Searched the archive, found three matching records, IDs 4, 9, 17" replaces three pages of output.
Discard — raw responses after extraction, failed attempts once the lesson is recorded, and anything already superseded.
Store outside — large artefacts. Write the document to a file and keep the path in context. A reference costs a few tokens; the content costs thousands.
Retrieval as memory
For long-term knowledge, the standard approach is retrieval: store material, convert it to embeddings, and at each step fetch the pieces relevant to the current question into context.
This is the same machinery as retrieval-augmented generation, used for a different purpose — not to answer a question, but to give an agent access to more than fits in front of it. The trade is that retrieval can miss. If the relevant memory is not fetched, the agent behaves exactly as if it never existed.
The diagnostic
When an agent contradicts itself, repeats work, or forgets a constraint, do not assume the model failed. Ask what was in the context at that moment. Almost always, the answer explains the behaviour completely.