diff --git a/_posts/2026-05-14-cache-invalidation-strategies.md b/_posts/2026-05-14-cache-invalidation-strategies.md new file mode 100644 index 0000000..901f54d --- /dev/null +++ b/_posts/2026-05-14-cache-invalidation-strategies.md @@ -0,0 +1,162 @@ +--- +title: "Cache Invalidation: The Hard Problem, Made Concrete" +description: "Every cache is a bet that stale data is acceptable for some window. Here is how TTL, event-driven purges, and write strategies trade correctness against hit rate, and where distributed invalidation quietly goes wrong." +date: 2026-05-14 12:00:00 +0000 +categories: [Distributed Systems, Caching] +tags: [caching, cache-invalidation, ttl, consistency, distributed-systems, performance] +image: + path: /assets/img/posts/cache-invalidation/hero.svg + alt: "A write path updating the source of truth while cache nodes serve stale data inside the staleness window, with TTL expiry and event-driven purge as two ways to close it" +--- + +There is an old joke that the two hard problems in computer science are cache invalidation, naming things, and off-by-one errors. The reason invalidation keeps its spot on that list is not that the code is hard to write. A delete is one line. The hard part is that **every cache is a bet that stale data is acceptable for some window**, and invalidation is the machinery that decides how long that window lasts and how reliably it closes. + +Once you frame it that way, the whole topic becomes a single tradeoff stated in different shapes: correctness versus hit rate. A cache that never serves anything stale is just a slow read of the source of truth. A cache that serves stale data forever is fast and wrong. Everything interesting lives in between, and the strategies below are different ways of choosing where in between you want to sit. + +## The Staleness Window Is the Real Unit + +Before picking a strategy, name the thing you are actually managing: the **staleness window**, the interval between when the truth changes and when every cache reflects it. You are never eliminating it. You are sizing it, and deciding what happens to a read that lands inside it. + +A cache is worth having only because reads dominate writes and the cost of a miss is high. The instant you cache, you accept that some reader will see an old value during the window. The engineering question is not "how do I avoid staleness" but "how small does the window need to be for this data, and how much load and complexity am I willing to pay to shrink it." A user's display name can be stale for minutes. A permission check usually cannot be stale at all. Those are not different caches; they are the same cache tuned to different window sizes. + +This is the same instinct I keep returning to with DNS, where a record's [TTL is a deliberate staleness budget](/2026/02/18/dns-the-silent-killer-of-distributed-systems.html). A cache TTL is the same idea wearing different clothes. + +## TTL Expiry: Simple, Bounded, and Wasteful on Purpose + +The simplest invalidation strategy is to not invalidate at all. You attach a time-to-live to each entry and let it expire. After the TTL elapses, the next read misses, fetches fresh data, and repopulates. + +
+ +What is attractive here is that **staleness is bounded and the system is self-healing**. You never have to find every cache that holds a copy of a value, because each copy expires on its own. There is no invalidation path to keep reliable, no message to deliver. If your purge infrastructure is broken, TTL still eventually converges. That property, convergence without coordination, is worth more than it looks. + +The cost is the tradeoff between freshness and load, and it pulls in both directions. A short TTL means fresher data but more misses, which means more load on the source of truth: at the extreme, a one-second TTL on a hot key is barely a cache at all. A long TTL means a cheap origin but a wide staleness window. You are tuning a single knob that trades the thing you built the cache to protect (the origin) against the thing the cache promised (correctness). + +There is also a failure mode TTL introduces on its own: synchronized expiry. If you populate ten thousand keys at the same instant with the same TTL, they all expire at the same instant, and the next moment is a stampede of misses hammering the origin together. The fix is **jittered TTLs**: add a small random spread so expirations smear across time instead of detonating at once. + + + +## Event-Driven Invalidation: Fresh, but You Own the Path + +The other end of the spectrum is to invalidate the moment the truth changes. On a write, you actively remove or update the cached copy. Done right, the staleness window shrinks to the latency of the invalidation, often milliseconds, and you keep a high hit rate because entries live as long as they stay valid instead of expiring on a timer. + +The catch is in the words "active" and "path." TTL needs nothing to be delivered. Event-driven invalidation needs a message to travel from the writer to every cache holding a copy, and **that path is now a part of your system you have to keep reliable**. If the purge is lost, dropped, or reordered, the entry stays stale with no timer to save you. Freshness has been traded for a dependency on delivery. + +This is exactly the kind of silent failure mode I described in [why DNS load balancing is not enough](/2026/02/24/when-dns-load-balancing-is-not-enough.html): the happy path looks instant, and the gap only shows up when the invalidation that was supposed to fire quietly did not. + +In practice nobody picks one end. The pattern that survives production is **short TTL plus purge as a backstop**: event-driven invalidation for fast convergence on the common case, and a modest TTL underneath so that any missed purge self-corrects within a bounded window. The purge gives you freshness; the TTL gives you a floor under your worst case when the purge fails. + +## Write Strategies Decide Who Sees Stale Data + +Where the cache sits relative to the write determines the staleness story before any invalidation logic runs. The four classic strategies are not interchangeable; each one moves the window to a different place. + + + +**Write-through** keeps the cache and source of truth in lockstep on every write, so the cache never trails the database. The cost is write latency, you pay for two writes, and you cache data that may never be read again. **Write-back** writes the cache first and flushes to the database asynchronously: writes are fast, but now the *database* is the stale copy, and a crash before the flush loses data outright. **Write-around** skips the cache on writes and goes straight to the database, which is great for write-heavy data that is rarely re-read, but it leaves any existing cached copy stale until it expires or is evicted. **Cache-aside** is the workhorse: the application reads through the cache and, on a write, deletes the entry so the next read repopulates it. It is simple and robust, but it has a sharp edge in the distributed case that the next section is entirely about. + +The point is that "what does a reader see during a write" has a different answer for each of these before you have invalidated anything. Choosing a write strategy is choosing where you are willing to let staleness live. + +## Where It Breaks: The Distributed Problems + +A single-process cache with a lock makes invalidation look easy. Distribute it and three problems appear that no amount of careful single-node reasoning prepares you for. + +**Invalidation messages get lost or reordered.** Your purge travels over a network, often a pub/sub channel fanning out to many cache nodes. Networks drop and reorder. If a purge is lost, that node stays stale until its TTL backstop fires (you do have one, right). Worse, if two writes to the same key produce two purges that arrive out of order on a system that updates rather than deletes, a node can end up holding the older value. This is why **purge-by-delete is safer than purge-by-update** in a distributed cache: a delete is idempotent and order-insensitive, while an out-of-order update can resurrect stale data. + +**Multi-region coherence.** Replicate caches across regions and the invalidation now has to cross a WAN with real latency and partition risk. A write in one region and a read in another are separated by the inter-region propagation delay, which is your staleness window whether you wanted one or not. You are usually forced to choose between paying cross-region invalidation latency on every write or accepting a per-region staleness window. There is no configuration that gives you both global freshness and cheap local reads. + +**The race between a write and a concurrent cache fill.** This is the subtle one, and it bites cache-aside specifically. Consider the interleaving: + + + +The reader loaded the old value before the write, then wrote it into the cache *after* the writer's delete. The cache is now confidently serving stale data with nothing queued to fix it. This is not a hypothetical; it is a real and well-documented hazard of cache-aside under concurrency. Mitigations include short TTLs so the wrong value cannot persist, per-key versioning so a stale fill can be detected and rejected, or a brief delete-after-delay so a late fill gets wiped. None of them are free, which is the recurring theme of this whole topic. The same write-versus-concurrent-read hazard shows up in [how reverse proxies handle concurrent requests](/2026/03/09/concurrent-requests-reverse-proxy.html): the moment two operations on the same state interleave, ordering stops being a detail and becomes the design. + +## The Pattern That Survives Production + +After enough incidents, the strategy that holds up is rarely the clever one. It is a small stack of unglamorous defaults. + + + +The durable recipe combines a few of these. Use **versioned keys** where you can, because changing the key sidesteps the stale-set race entirely: an old fill lands on a key no one will read again, and it ages out quietly. Use **short TTL plus purge as a backstop** so that fast convergence and self-healing both hold, and a lost invalidation costs you a bounded window rather than permanent wrongness. Use **jittered TTLs** so expiry never synchronizes into a stampede. And size every window deliberately, because, just like a [DNS TTL](/2026/02/18/dns-the-silent-killer-of-distributed-systems.html), it is a promise about how stale your readers may be, not an implementation detail. + +None of this makes invalidation easy. It makes it honest. You stop pretending the cache is always correct and start stating, explicitly, how wrong it is allowed to be and for how long. That is the whole job. + +A cache does not give you correctness for free. It lets you trade a precise, bounded amount of staleness for speed. Invalidation is just the part of the system where you write that trade down and make it true.
+ +--- + +*This builds on my earlier pieces on [It's Always DNS](/2026/02/18/dns-the-silent-killer-of-distributed-systems.html), [when DNS load balancing is not enough](/2026/02/24/when-dns-load-balancing-is-not-enough.html), and [how reverse proxies handle concurrent requests](/2026/03/09/concurrent-requests-reverse-proxy.html).* + +*Tuning staleness windows or chasing a stale-cache bug across regions? I am on [LinkedIn](https://www.linkedin.com/in/singhsanjay12) or reachable by [email](mailto:hello@singh-sanjay.com).* diff --git a/assets/img/posts/cache-invalidation/hero.svg b/assets/img/posts/cache-invalidation/hero.svg new file mode 100644 index 0000000..c7cd507 --- /dev/null +++ b/assets/img/posts/cache-invalidation/hero.svg @@ -0,0 +1,74 @@ +