Most agent posts here focus on tools and prompts, so this one takes the layer underneath: what actually happens to conversation state between turns. It is the part that decides whether your bot feels like a colleague or a stranger every Monday morning.
The Stateless Default
Every mainstream LLM API is stateless. You send the full conversation, the model answers, and then it forgets. So "memory" in most chatbots is just an ever growing message list resent on every request, and that breaks in three predictable ways.
Context windows overflow. Fifty turns can pass 30,000 tokens before you count the system prompt, tool definitions and retrieved documents. Once you exceed the limit something gets truncated, usually the oldest messages, which is exactly where the user explained their problem.
Cost scales badly, because turn N pays to reprocess turns 1 through N-1 all over again.
And nothing survives the session. Close the tab, come back tomorrow, start from zero.
Three Layers, Not One Store
Working context is the current task: the last few turns verbatim plus whatever the model needs right now. Short lived, high fidelity.
Episodic memory is what happened: summaries of past sessions with timestamps. Compressed, retrieved by relevance.
Semantic memory is what is true about the user: preferences, entities, account facts. Small, structured, updated in place rather than appended.
Most implementations collapse all three into one vector store, then wonder why retrieval returns a two month old aside instead of the preference the user stated last week.
Retention Is A Policy Decision
The write path is the easy half. The hard calls are what gets summarized versus kept verbatim, what expires and when, and what happens when a new fact contradicts an old one. Write those rules down before you pick a database, because they shape the schema more than the database does.
A practical default: summarize any session older than the current one, keep entities and preferences as structured rows you update in place, and give every stored fact a source and a timestamp so you can expire it later without guessing.
The longer version of this, including dialogue management patterns, summarization strategies and the framework landscape, is in our guide to conversational AI and chatbot memory.
What does your retention policy look like, or is it all still one big embeddings table?
