Caching Strategies and Invalidation Techniques

Why and Where Do We Cache?

Before we get into specific strategies, let's solidify the "why" and "where" of caching. A cache is a high-speed data storage layer that stores a subset of data, typically transient in nature, so that future requests for that data are served up faster than is possible by accessing the data’s primary storage location.

This speed difference is profound, but where you place the cache in your system architecture is just as important. Caching isn't a single concept; it's a layered approach.

For most backend interviews, the focus will be on the external, distributed cache (like Redis or Memcached) sitting between your application servers and your database. This is where the most interesting architectural trade-offs lie.

Core Caching Read/Write Strategies

This is the heart of our lesson. A caching strategy defines the rules for how your application interacts with the cache and the database. The choice of strategy directly impacts your system's performance and data consistency.

The diagram below illustrates the five most common patterns we'll discuss.

We'll use a detailed article as our main reference for this section. It provides an excellent breakdown of each strategy with clear pros and cons.

Let's consolidate what you've just read into an interview-focused summary.

1. Cache-Aside (or Lazy Loading)

• How it works:
  1. Read: The application first checks the cache. If it's a miss, the application reads from the database, then writes the result into the cache before returning it.
  2. Write: The application writes directly to the database. To handle invalidation, it usually deletes the corresponding key in the cache.
• Trade-offs:
  • Pros: Most common and versatile pattern. Resilient to cache failures (the app can still operate by hitting the DB). The cache only stores data that is actually requested.
  • Cons: The first read for any piece of data is always a miss (a "cold start"). There's a slight window for data inconsistency between the database write and the cache invalidation.
• Interview Answer: "This is my default, general-purpose strategy. It gives the application layer full control and is simple to implement with standard tools like Redis."

2. Read-Through

• How it works:
  1. Read: The application only ever talks to the cache. If the cache misses, it is the cache's responsibility to fetch the data from the database, store it, and return it to the application.
• Trade-offs:
  • Pros: Simplifies application code (no data-fetching logic).
  • Cons: Requires a cache provider that supports this pattern (e.g., AWS DAX for DynamoDB) or a library that abstracts it. Less flexibility.
• Interview Answer: "Read-Through is great when you want to treat the cache like the primary data source from the application's perspective, simplifying the code. It's common in systems that come with a dedicated, integrated cache."

3. Write-Through

• How it works:
  1. Write: The application writes data to the cache, and the cache synchronously writes that data to the database. The operation is only complete after both writes succeed.
• Trade-offs:
  • Pros: Guarantees strong consistency between the cache and the database. Reads will never get stale data.
  • Cons: Higher write latency, as you have to wait for two network operations. Can pollute the cache with data that might never be read.
• Interview Answer: "I'd use Write-Through when data consistency is absolutely critical and I cannot tolerate any stale reads. Think financial ledgers, real-time inventory counts, etc."

4. Write-Back (or Write-Behind)

• How it works:
  1. Write: The application writes data only to the cache, which returns success immediately. The cache then asynchronously writes the data to the database in the background, often in batches.
• Trade-offs:
  • Pros: Extremely fast writes and high write throughput. Reduces load on the database.
  • Cons: Risk of data loss. If the cache fails before the data is persisted to the database, the write is lost forever. Eventual consistency is the best you can hope for.
• Interview Answer: "Write-Back is ideal for write-heavy workloads where top performance is key and you can tolerate a small amount of data loss. For example, logging user interactions, metrics collection, or updating a 'last seen' timestamp."

5. Write-Around

• How it works:
  1. Write: The application writes data directly to the database, completely bypassing the cache.
  2. Read: Follows the Cache-Aside pattern. Data is only loaded into the cache on a read miss.
• Trade-offs:
  • Pros: Prevents the cache from being filled with data that is never read. Good for "write-once, read-maybe" workloads.
  • Cons: Every read following a write will be a cache miss, resulting in higher read latency.
• Interview Answer: "I'd choose Write-Around for data that has a high write volume but low read frequency, like application logs or event streams. This keeps the cache clean for more valuable, 'hot' data."

Cache Invalidation and Eviction

Knowing how to write to a cache is only half the battle. You also need a strategy for removing data. There are two key concepts here:

1. Invalidation: How do you remove data that is no longer accurate (stale)?
2. Eviction: When the cache is full, which items do you remove to make space for new ones?

Invalidation: The Hard Part

As the quote says, this is tough. Let's look at the primary strategies.

Let's summarize and add an advanced technique from the AWS caching article LINK.

• Time-To-Live (TTL): The simplest strategy. You set an expiration time on each cache entry.
  • Pro: Easy to implement; guarantees the cache is eventually cleared.
  • Con: Data can be stale for the duration of the TTL.
• Write Invalidation: When your application writes an update to the database, it explicitly sends a DELETE command to the cache for that key.
  • Pro: Minimizes the window of inconsistency.
  • Con: Requires more application logic. Race conditions are possible if a read happens between the DB write and the cache delete.
• Soft and Hard TTLs (Advanced): This is a production-grade resilience pattern.
  • Soft TTL: A short TTL (e.g., 5 minutes). When an item expires, the next request will try to refresh it from the database.
  • Hard TTL: A longer TTL (e.g., 1 hour). If the database is down and the refresh fails, the application is allowed to serve the stale data from the cache up until the hard TTL is reached.
  • This pattern gracefully trades consistency for availability during downstream outages. Mentioning this shows a mature, production-oriented mindset.

Eviction Policies

When your cache runs out of memory, it needs to decide what to discard. This is what eviction policies are for.

Here is a quick summary:

• Least Recently Used (LRU): Evicts the item that hasn't been accessed for the longest time. This is the most common and often the default policy.
• Least Frequently Used (LFU): Evicts the item that has been accessed the fewest times. This is useful when you have a set of "celebrity" items that are always popular, even if they aren't accessed every minute.
• First-In, First-Out (FIFO): Evicts the oldest item, regardless of how often or recently it was accessed. Simple but usually not optimal.

Choosing the Right Strategy: An Interview Framework

In an interview, the goal is not to state one "correct" answer but to discuss the trade-offs.

Interviewer: "We need to design the caching layer for a product catalog in an e-commerce system. What strategy would you propose?"

Your Thought Process & Ideal Answer:

1. Clarify and State Assumptions: "Great question. The best strategy depends on what data we're caching and its requirements. Let's break it down. For product details like names, descriptions, and images, the data is read very frequently but updated infrequently. For inventory/stock count, it's updated frequently and must be consistent."

2. Propose a Solution for Read-Heavy Data: "For the product details, I'd recommend a Cache-Aside strategy.

   • Why? It's a read-heavy workload, and this pattern is simple and resilient. If our Redis cluster has issues, the application can fall back to the database.
   • Invalidation: We can use a combination of a moderate TTL (e.g., 1 hour) and active write invalidation. When a merchant updates a product, our service will update the database and immediately issue a DELETE to the cache for that product's key.
   • Eviction: I'd start with an LRU eviction policy, as it's a good general-purpose choice for ensuring popular products stay in the cache."

3. Propose a Solution for Consistency-Critical Data: "For inventory count, the requirements are different. Showing a customer they can buy an out-of-stock item is a bad experience.

   • Option 1 (High Consistency): We could use a Write-Through cache. Every stock change would update the cache and DB synchronously. However, this would slow down the checkout process.
   • Option 2 (Pragmatic Approach): A more practical solution is to use Cache-Aside but with a very short TTL (e.g., 5-10 seconds). This provides near-real-time data without the latency penalty of Write-Through. The system must be able to handle the small chance of a race condition at checkout. For a senior role, this is a better, more nuanced answer."

4. Summarize: "So, in summary, we'd use a hybrid approach: Cache-Aside with LRU and a long TTL for static product data, and Cache-Aside with a very short TTL for dynamic inventory data."

This type of structured, trade-off-driven response demonstrates senior-level architectural thinking.

Conclusion

You've now explored the core architectural patterns for caching. You understand that caching is not a monolith, but a series of deliberate design choices about how and where data is stored and managed.

Key Takeaways:

• Caching is layered: It can exist on the client, at the edge (CDN), in-process, or as an external service.
• Cache-Aside is your default: It's the most common, flexible, and resilient pattern.
• Choose other patterns for specific needs: Use Write-Through for consistency, Write-Back for write performance, and Write-Around to avoid cache pollution.
• Invalidation is critical: You must have a plan for handling stale data, whether through TTLs, active invalidation, or a combination. The "soft/hard TTL" pattern is an excellent one to have in your toolkit for resilience.
• Eviction policies matter: LRU is a safe default, but LFU can be better for highly skewed traffic.

Preview of the Next Lesson:

We've now covered two fundamental building blocks of scalable systems: load balancing and caching. We're ready to start assembling them into a complete design. In our next lesson, "Applying System Design: Notification Service," we will tackle our first end-to-end design problem. We will architect a system that can send notifications (email, SMS, push) at scale, forcing us to make decisions about APIs, asynchronous processing, and, of course, where to apply caching and load balancing for maximum effect.