A cache is a promise with an expiration date.
That sounds obvious until a production system starts depending on cached state as if it were truth. Then the cache is no longer an optimization. It is a second source of truth with worse durability, weaker ownership, and fewer people watching it.
That is where cache bugs become painful.
The hard part of caching is not storing a value. The hard part is deciding what the value is allowed to mean.
A Cache Should Have a Narrow Job
The safest caches answer narrow questions:
Have we recently computed this result?
Have we recently seen this object absent?
Have we recently fetched this configuration?
Can we avoid doing the same expensive work again?
The more dangerous caches answer authority-shaped questions:
Does this object exist?
Is this user allowed?
Is this state current?
Has this workflow completed?
Sometimes a cache must sit near one of those questions. That is fine. But the design needs to say exactly how wrong the cache is allowed to be and what happens when it is wrong.
The Three Questions Every Cache Needs
Before adding a cache, I like to ask three questions.
First:
What is the source of truth?
If the team cannot answer this clearly, the cache will eventually become part of the data model by accident.
Second:
What is the failure mode when the cache is unavailable?
Fail-open and fail-closed are both valid choices, but they belong to different classes of systems. A performance cache in front of an idempotent delete path can usually fail open. An authorization decision probably cannot.
Third:
What is the failure mode when the cache is stale?
This is where many designs get uncomfortable. A stale cache entry might mean an extra database read, which is fine. It might mean stale search results for a minute, which may be acceptable. It might mean granting access after permission was revoked, which is not the same kind of risk.
TTL Is Not Invalidation
TTL is a recovery mechanism. It is not a complete correctness story.
A TTL says:
If we miss an invalidation, the system will eventually heal.
It does not say:
The system is correct until the TTL expires.
If stale data can cause a correctness bug, the system still needs an invalidation path, a version check, a source-of-truth read, or a design that makes stale data harmless.
TTL is useful because production systems are imperfect. It should not be the only thing standing between the user and a wrong answer.
Fail-Open vs Fail-Closed
The right behavior depends on what the cache protects.
For a performance cache:
cache hit -> use cached result
cache miss -> recompute or read from source
cache error -> bypass cache
This keeps correctness independent of cache availability.
For a safety or authorization cache, bypassing the cache may be wrong. If the cache is the only fast path to know whether an operation is allowed, fail-open can turn an outage into a security issue. In those systems, the right answer may be to fail closed, degrade the feature, or require a source-of-truth check.
The cache design should make that tradeoff explicit.
When a Cache Becomes a Lie
A cache becomes a lie when the rest of the system forgets that it is allowed to be wrong.
Common warning signs:
- no documented source of truth
- no owner for invalidation
- no metric for hit rate, miss rate, or stale reads
- business logic that only exists in cache-writing code
- cache entries with no version, generation, or TTL
- code that treats cache errors as successful reads
- tests that mock the cache but never test cache loss
The worst cache bugs are not caused by Redis being down. They are caused by Redis being trusted too much.
The Lesson
A good cache makes a system faster without becoming more important than the data it stores.
It is allowed to be missing. It is allowed to be cold. It is allowed to be flushed. It may be useful when present and boring when absent.
The moment a cache cannot be lost, rebuilt, bypassed, or corrected, it is no longer just a cache.
It is part of the system of record, whether the architecture diagram admits it or not.