Buffer Memory

Buffer memory keeps the conversation messages verbatim and sends them with every request. The pattern is simple: append each user message and assistant reply to a list, then pass the whole list to the model on the next turn. With the Anthropic Messages API the application resends the full history on every turn anyway, and buffer memory is that pattern with nothing summarized or removed. It works until the history no longer fits the context window. It is the right default for short conversations and prototypes.

The cost shows up before the hard limit. Every turn resends all earlier turns, so input tokens and latency grow, and Anthropic notes that accuracy and recall degrade as token count grows, a problem it calls context rot. The usual fix is a bounded buffer: keep the last N messages, or trim from the oldest end until a token budget fits. LangChain lists the same options for short term memory: trim, delete or summarize earlier messages. A trimmer should keep the system prompt and never split a tool call from its tool result.

class BufferMemory:
    def __init__(self, max_messages: int = 20):
        self.max_messages = max_messages
        self.messages: list[dict] = []

    def add(self, role: str, content: str) -> None:
        self.messages.append({"role": role, "content": content})
        # Once the cap is hit, drop the oldest messages first
        self.messages = self.messages[-self.max_messages:]


memory = BufferMemory(max_messages=4)
for i in range(3):
    memory.add("user", f"question {i}")
    memory.add("assistant", f"answer {i}")
print([m["content"] for m in memory.messages])

A bounded buffer where the oldest messages drop first. A production trimmer should also keep the system prompt and tool call pairs intact.

Use buffer memory for prototypes and any agent where conversations are bounded short. Put a message or token cap on it from the start, and switch to summary memory the moment conversation length is unpredictable. Avoid it for long running sessions, where cost and recall both get worse as the buffer grows.

• LangChain: Memory types
• LangChain: Short term memory
• Anthropic: Context windows

Related Terms