Most of us reach for a vector database the first time an app needs memory, and it holds up right until it does not. This is the piece that changed how I score retrieval, so I am putting it here in case anyone else is hitting the same wall.
A retrieval system holding a few hundred records works fine on cosine similarity alone. Somewhere between one thousand and ten thousand records it stops working, and the symptom is specific: the results are still topically related, but they are the wrong ones. A note from six months ago outranks the thing you said yesterday because it happens to share more vocabulary with the query.
Similarity Has No Sense Of Time
Cosine similarity answers exactly one question: how close are these two vectors. It has no opinion about when a memory was written, how often it has proven useful, or whether something later contradicted it. Every record you add is one more candidate competing on the only axis the system can see, so precision degrades as a direct function of volume.
Human memory does not degrade that way, and the reason is that biological retrieval is scored on more than one axis.
Base-Level Activation
ACT-R is a cognitive architecture that John Anderson started at Carnegie Mellon in the 1970s, and its retrieval model has been calibrated against human experimental data for forty years. The core idea is base-level activation: every memory carries an activation value that rises each time the memory gets used and falls as time passes since the last use.
In practice that hands you two extra ranking signals almost for free, because you are already storing timestamps and an access count is cheap. Recency and frequency turn out to carry a lot of the signal that similarity alone is missing.
Spreading Activation
The second piece is spreading activation, where the context of the current query raises the activation of connected memories before ranking happens. If the conversation is about one specific customer, everything linked to that customer gets a boost, so a weakly worded but highly relevant record can beat a well worded irrelevant one.
This is where an entity graph pays for itself. You do not need a full knowledge graph to start, just links between records and the entities they mention.
Decay Is A Feature, Not Data Loss
The part that feels wrong to engineers is deliberately letting activation fall. We are trained to treat losing data as a bug. But a store that never forgets is a store where every stale fact competes forever with the current one, and the forgetting curve is what keeps the working set small enough to stay accurate. Records that keep getting retrieved resist decay on their own, which is the behaviour you actually want.
What It Costs To Add
Less than you would expect, which is the part worth knowing. You keep your embeddings and your vector index. You add a last-accessed timestamp, an access count, and a scoring function that combines the similarity score with an activation term. The equations are published and the parameters are already calibrated against decades of experimental data, so this lands as a scoring change rather than an architecture rewrite.
The full breakdown, including the base-level learning equation, how spreading activation traverses entity links, and how the decay curve behaves under spaced repetition, is written up at ACT-R cognitive architecture for AI.
If you are running retrieval over more than a few thousand records, the cheapest accuracy win available is usually not a better embedding model. It is giving your scorer a sense of time.