Most of the memory posts here are about getting storage working. This one is about what happens six months later, when it is working and the answers have quietly gotten worse.
If you have shipped an agent that stores what users tell it, you have probably watched retrieval quality peak somewhere around a few hundred memories and then slide. The code did not change. The store just got bigger. Four things are happening at once, and none of them show up as an error.
Contradiction
A user says in January they prefer Python. In March they say they moved to Rust. Both rows are in the store, both embed near a question about coding style, and cosine similarity has no concept of one statement superseding another. Your retriever returns whichever is textually closer to the query, which is a coin flip. Nothing logs a failure, the agent just sounds confused.
Staleness
API endpoints move, docs get rewritten, people change roles. A store that never forgets treats a two year old reference exactly like the one you wrote yesterday. Worse, older docs are often longer and more detailed, so similarity scoring can actively favor them. You end up with a system confidently describing an endpoint you deleted.
Cost
Every memory takes storage, an embedding, and a slot in the candidate set that gets scored at query time. If 30,000 of your 50,000 memories are stale or redundant, you are paying compute to make your results worse. That is the part that stings.
Retrieval Noise
Top N is a fixed budget. As the store grows, the ratio of signal to junk inside that budget drops. The right memory can still be in the store and still never reach the context window, because five near duplicates outranked it.
What Actually Fixes It
The fix is not a better embedding model, it is treating memory as something with a lifecycle rather than an append only log. Memories get created, promoted when they prove useful, consolidated when they say the same thing twice, and allowed to decay when nothing touches them. Retrieval scoring stops being pure similarity and starts including how recent a memory is, how often it has been used, how connected it is to other entities, and how well later evidence confirmed it. That last signal is what resolves the Python and Rust problem without a human in the loop.
That is the model Adaptive Recall is built on, using ACT-R activation scoring from cognitive science rather than similarity alone, with four retrieval strategies running in parallel and an explicit forget tool sitting next to store and recall. There is a free tier if you want to test the behavior against your own data.
The takeaway holds regardless of what you build on: if your memory system has no way to forget, it has no way to stay correct. Growth alone will degrade it.
What does your store do today when a user contradicts something they told you last quarter?
