You can’t bolt memory onto a chat LLM

The standard way to give a chat agent memory is to hand it a tool, remember_this(fact), and a place to write. The model decides what’s worth saving, calls the tool, and later reads it back. It demos well. It does not work.

The problem is that the chat-facing model is the wrong thing to put in charge of memory, because memory is mostly upkeep and the chat model is lazy about upkeep. It will happily answer your question. It will not, mid-conversation, notice that a new fact contradicts something it saved three sessions ago, reconcile the two, re-index the affected entries, and prune the stale one. That work has no reward inside the conversation, so it doesn’t happen, and you’re left with a pile of append-only notes that slowly rots.

Real memory is a system, not a tool. When I built Willow, a consumer chat agent on top of an evolving knowledge graph, the chat was the least interesting part. The memory layer underneath runs its own background agents, each with one job: one indexes new information, one evolves the graph as relationships change, one does maintenance and contradiction resolution, one handles retrieval. They run beside the conversation, not inside it, on their own schedule, answerable to the state of the graph rather than to whatever the user just asked.

Separating them matters for the same reason you don’t ask the waiter to also cook, run the till, and do inventory. Each background agent can be slow, thorough, and single-minded in a way the chat model can’t afford to be when someone’s waiting for a reply. The chat is just the surface that exercises the memory: it reads from the graph and writes intents to it, and the upkeep happens out of band.

So when someone says their agent “has memory,” the question I ask is: what maintains it? If the answer is “the model saves things when it remembers to,” that’s not memory, it’s a notebook the model occasionally writes in. Memory you can trust over months is an architecture, a set of separate processes doing the unglamorous work of keeping the store true. It’s a lot more than one tool call.