Hello! Welcome to the third lesson in our "Scalability and Performance Principles" module.
In our last lesson, we focused on how to distribute load across multiple servers using various load balancing strategies. That's a crucial step for handling high traffic. Today, we'll tackle the other side of the coin: how to reduce the amount of load on your backend services in the first place. The answer, in many cases, is intelligent caching.
This lesson is designed to help you master the learning outcome: Explain caching strategies (write-through, write-back, write-around) and cache invalidation approaches. As a senior engineer, you're expected not only to know these patterns but to confidently articulate the trade-offs between data consistency, performance, and operational complexity, which is precisely what we'll focus on.
1. Why Cache? A Quick Refresher
You're already familiar with the concept of caching from your extensive backend experience. At its core, caching is about trading a bit of storage (usually faster, more expensive memory) and complexity for significant gains in speed and a reduction in load on slower systems like databases.
Let's start with a short clip to set the stage and quantify the performance difference we're talking about.
Caching in System Design Interviews w/ Meta Staff Engineer
This video from 'Hello Interview' features a Meta Staff Engineer explaining the fundamental 'why' of caching by comparing memory and disk access speeds. It's a great way to frame our discussion.
Watch from 00:50 to 02:03. Focus on the order-of-magnitude difference in speed between memory and disk, and how that motivates the use of caching.
That 10,000x speed difference between RAM and SSD access is the fundamental reason caching is one of the most powerful tools in system design. The main challenge, then, is not if you should cache, but how.
2. Core Caching Patterns and Write Policies
The "how" of caching is defined by the caching patterns, which dictate how your application interacts with the cache and the primary data store (like a database). The learning outcome specifically asks about write-through, write-back, and write-around, which are all write policies. We'll examine these alongside the most common read policy, cache-aside.
Let's dive into the most important patterns and their trade-offs.
Caching in System Design Interviews w/ Meta Staff Engineer
The same video provides an excellent breakdown of the most common caching patterns you'll encounter in interviews.
Watch from 08:41 to 14:16. Pay close attention to the descriptions and trade-offs of cache-aside, write-through, and write-back (also called write-behind). The video correctly points out that cache-aside is the most common pattern and your likely default in an interview.
The video gave us a great overview. Now, let's solidify our understanding of these write policies and add write-around to the mix using a concise article.
In-Depth Guide to Cache Invalidation Strategies
This guide from Design Gurus clearly defines the key write policies. We'll use it to reinforce what we just watched and to specifically cover the 'write-around' strategy.
Read the sections titled 'Write-through cache', 'Write-around cache', and 'Write-back cache (or lazy-write)'. As you read, compare these definitions to what you saw in the video and focus on the specific use cases and trade-offs mentioned for each.
To bring all this together, let's visualize the data flow for these patterns.
Here’s a summary of the key trade-offs in a way you can articulate in an interview:
| Strategy | How it Works | Pros | Cons | Best for... |
|---|---|---|---|---|
| Cache-Aside | App is responsible for reading from DB on cache miss and writing to cache. | Simple to implement with standard caches (Redis, Memcached). Resilient to cache failure. | Cache can hold stale data. A cache miss incurs three steps (cache read, DB read, cache write), increasing latency. | The default choice for read-heavy workloads. Most general-purpose applications. |
| Write-Through | App writes to cache; cache writes to DB synchronously. | Data in cache is always consistent with DB. High data durability. | High write latency (must write to two places). Can pollute cache with data that's never read. | Systems where data consistency is paramount and read-heavy workloads justify the write penalty (e.g., product inventory, bank account balances). |
| Write-Back | App writes to cache; cache writes to DB asynchronously later (in batches). | Very low write latency, high write throughput. Resilient to DB downtime. | High risk of data loss if the cache fails before a write to the DB occurs. More complex. | Write-heavy systems where top performance is critical and some data loss is acceptable (e.g., high-frequency counters, analytics data collection). |
| Write-Around | App writes directly to DB, bypassing the cache. | Prevents cache pollution from write-only or rarely-read data. | Reads for recently written data will always be a cache miss, resulting in higher read latency. | Write-heavy workloads where data is not read immediately after being written (e.g., logging, bulk data ingestion). |
3. Cache Invalidation: The Hard Problem
"There are only two hard things in Computer Science: cache invalidation and naming things." This famous quote is an exaggeration, but it highlights a critical challenge. Once you put data in a cache, how do you ensure it's removed or updated when the source data changes? Stale data in the cache can lead to everything from minor UI bugs to serious business logic failures.
This is the problem of cache invalidation.
The "When": Triggers for Invalidation
Broadly, there are two triggers for invalidating a cache entry:
- Time-Based (TTL - Time To Live): You set an expiration time on each cache entry (e.g., 5 minutes). The cache automatically evicts the entry after the time is up. This is simple and guarantees that stale data won't live forever, but it doesn't ensure immediate consistency.
- Explicit Invalidation (On Write): When your application writes a change to the database, it also sends a command to the cache to explicitly delete (or update) the corresponding entry. This provides much stronger consistency.
In the Hello Interview video, the discussion on "Cache Consistency" (starting at 20:30) covers exactly this trade-off: invalidating on write for freshness vs. using a short TTL when some staleness is acceptable.
The "How": Methods of Invalidation
When you decide to invalidate data, there are several specific actions you can take. Let's explore the most common methods.
In-Depth Guide to Cache Invalidation Strategies
The Design Gurus article we looked at earlier also has a fantastic section on specific invalidation methods. This directly addresses the second half of our learning outcome.
Read the section 'Cache Invalidations Methods', covering 'Purge', 'Refresh', 'Ban', 'Time-to-live (TTL) expiration', and 'Stale-while-revalidate'. Focus on the distinct action each method performs.
This image provides a great visual summary of the methods you just read about.
Summary of Invalidation Methods:
- Purge/Delete: The most common approach for explicit invalidation. When data is updated in the DB, you send a
DELETEcommand to the cache for that key. The next read will be a cache miss, forcing a fetch of the fresh data. - Refresh/Update: Instead of deleting the key, you could update the cache with the new value. This is essentially what a write-through strategy does.
- TTL (Time-To-Live): A passive approach where you rely on automatic expiration. Simple, but data can be stale until the TTL expires.
- Stale-While-Revalidate: A sophisticated strategy that prioritizes low latency. It serves stale data to the user instantly while triggering an asynchronous background fetch to update the cache. This is great for user experience when eventual consistency is acceptable.
4. Structuring Your Answer in an Interview
Knowing the concepts is half the battle; articulating them within a structured interview framework is the other half. When you propose adding a cache, you need to justify it and explain your design choices clearly.
Caching in System Design Interviews w/ Meta Staff Engineer
Let's return to the 'Hello Interview' video for the final, most important part: a step-by-step guide on how to discuss caching in a system design interview.
Watch from 25:37 to 29:44. This is a goldmine. Internalize the 5-step process he outlines for introducing and explaining a caching layer.
Here is the framework from the video, which you should aim to follow:
- Identify the bottleneck: Don't just add a cache. Justify it. "Our database is experiencing high read load from profile lookups, increasing p99 latency to 200ms. We can add a cache to serve these profiles from memory, which is orders of magnitude faster."
- Decide what to cache: Be specific. "We'll cache user profile objects. The cache key will be
user:<user_id>. We will not cache transient, user-specific data like session information in this global cache." - Choose your cache architecture (write policy): This is where you apply today's knowledge. "I'll start with a cache-aside strategy. The application will manage fetching from the database on a cache miss. This is the simplest and most common approach, and it keeps the cache from being polluted with unread data."
- Set an eviction policy: You need to manage the cache size. "We'll use LRU (Least Recently Used) as the eviction policy, which is a good general-purpose default. We'll also set a TTL of 10 minutes on profile data as a safeguard against stale data."
- Address the downsides: Show senior-level thinking by discussing trade-offs and potential problems. "The main downside is data consistency. When a user updates their profile, the cache will be stale. To mitigate this, our write service will explicitly invalidate the cache entry (e.g.,
DELETE user:<user_id>) immediately after committing the change to the database. This ensures the next read will fetch fresh data."
Conclusion
Today, we've dissected the core strategies that govern how data moves into, out of, and between your cache and your database. Mastering these patterns is fundamental to designing high-performance systems.
Key Takeaways:
- Cache-Aside is your default, general-purpose caching pattern for read-heavy workloads.
- Write-Through prioritizes strong consistency over write performance.
- Write-Back prioritizes high write throughput but risks data loss on failure.
- Write-Around is for write-heavy workloads where data isn't immediately read back.
- Cache invalidation is critical for managing data freshness. You can do this explicitly (on-write deletion) or passively (using TTLs), with patterns like stale-while-revalidate offering a sophisticated hybrid approach.
- Always use the 5-step framework (Bottleneck, What, Architecture, Eviction, Downsides) to structure your caching discussion in an interview.
In our next lesson, we'll zoom out from these fundamental patterns and look at how caches are composed into larger architectures. We'll cover multi-tier caching, discussing the distinct roles and behaviors of CDNs, application-level caches, and database caches.