External Memory
Memory stored outside the LLM context window in databases, vector stores, or files, retrieved on demand at inference time.
Last updated: April 26, 2026
Definition
External memory is everything the agent remembers but does not keep in the context window by default. It lives in a vector database (for semantic retrieval), a key-value store (for structured facts), or a document database (for raw conversations and decisions). At inference time, a retrieval step pulls only the relevant memories into the context window. The benefit is unbounded scale: the agent can remember millions of past interactions without paying for them on every call. The cost is retrieval complexity: you have to write the retrieval query that surfaces the right memory at the right moment.
When To Use
Required for any agent that needs to remember across sessions or across more data than fits in a single context window. Almost every production agent needs it.
Related Terms
Building with External Memory?
I've shipped this pattern in real production systems. If you want a second pair of eyes on your architecture, that's what I do.