Caching Strategies and Use Cases

1. Why is Caching a Cornerstone of Backend Performance?

At its core, caching is about reducing latency. Accessing data from main memory (RAM) is orders of magnitude faster than accessing it from a disk-based database. For systems that need to handle thousands or millions of requests, this difference is monumental.

To understand the sheer scale of this performance gap and why caching is so effective, please watch the beginning of the following video from a Staff Engineer at Meta.

The key takeaway is that a cache acts as a high-speed, temporary storage layer for frequently accessed data, saving your application from making slow, repetitive trips to the primary data source (like a PostgreSQL database).

2. Core Caching Patterns and Strategies

Now that we understand why we cache, let's explore how. A caching strategy, or pattern, defines the rules for how your application, cache, and database interact. It dictates when data is read from the cache, when it's written to the cache, and how the cache and database are kept in sync.

The image below provides a great visual summary of the most common strategies we will discuss.

The following video provides an excellent, in-depth explanation of the most important patterns. We will use this as our primary guide.

Let's consolidate what you've just learned about each strategy.

A. Cache-Aside (Lazy Loading)

This is the most common caching strategy and the one you will likely implement most often.

• Read Path:
  1. The application tries to read data from the cache.
  2. Cache Miss: If the data is not in the cache, the application reads the data directly from the database.
  3. The application then writes the data into the cache.
  4. The data is returned to the client.
  5. Cache Hit: If the data is in the cache, it is returned directly to the client.
• Write Path: The application writes data directly to the database. The cache is not updated on a write. The old, stale data in the cache will either expire or be overwritten after the next cache miss.
• Pros:
  • Resilient: The cache and database are decoupled. If the cache fails, the application can still function by falling back to the database (though more slowly).
  • Lean Cache: Only data that is actually requested is cached.
• Cons:
  • Stale Data: Data in the cache can become stale if it's updated in the database but not invalidated in the cache.
  • Cache Miss Penalty: The first read for any piece of data will always be a cache miss, resulting in three steps (cache read, DB read, cache write), which is slower than just a DB read.

B. Write-Through

This strategy prioritizes data consistency.

• Read Path: Usually combined with Cache-Aside or Read-Through for reads.
• Write Path:
  1. The application writes data to the cache.
  2. The cache synchronously writes the data to the database.
  3. The operation is only considered complete after both the cache and the database have been updated.
• Pros:
  • High Consistency: Data in the cache and database is always consistent after a write operation.
• Cons:
  • Higher Write Latency: Every write has to go to two systems, making it slower than writing to the database alone.
  • Cache Pollution: Data that is written but rarely read might fill up the cache unnecessarily.

C. Write-Behind (or Write-Back)

This strategy is optimized for high-throughput writes.

• Read Path: Usually combined with Cache-Aside or Read-Through.
• Write Path:
  1. The application writes data directly and only to the cache.
  2. The cache then asynchronously writes the data to the database in the background, often in batches.
• Pros:
  • Extremely Low Write Latency: The application gets a response as soon as the data is in the fast in-memory cache.
  • Resilient to DB Downtime: Writes can still be accepted even if the database is temporarily unavailable.
• Cons:
  • Risk of Data Loss: If the cache fails before the data is written to the database, those writes are permanently lost. This makes it unsuitable for critical data like financial transactions.

D. Read-Through

This is a variation of Cache-Aside that simplifies application logic.

• Read Path:
  1. The application tries to read data from the cache.
  2. Cache Miss: The cache itself is responsible for fetching the data from the database, updating itself, and returning the data to the application.
• Write Path: Typically paired with a write strategy like Write-Through or Write-Behind.
• Pros:
  • Simpler Application Code: The application only ever interacts with the cache, abstracting the database away.
• Cons:
  • Requires Specialized Provider: Your cache (e.g., Redis) needs a plugin or library to communicate with your specific database, adding complexity to the cache layer. Standard Redis does not support this out of the box.

3. Keeping the Cache Fresh: Eviction and Invalidation

A cache has limited memory. This introduces two fundamental challenges:

1. What do you do when the cache is full? (Eviction)
2. How do you handle data that has changed in the database? (Invalidation)

Eviction Policies

When the cache runs out of space, an eviction policy decides which items to remove. The video you watched earlier covers the most common ones.

In summary:

• Least Recently Used (LRU): Evicts the item that hasn't been accessed for the longest time. This is a great general-purpose default.
• Least Frequently Used (LFU): Evicts the item that has been accessed the fewest times. Useful when some items are vastly more popular than others, regardless of when they were last accessed.
• Time To Live (TTL): Each item is stored with an expiration time. Once the time is up, it's automatically removed. This is excellent for data where freshness is important and some staleness is acceptable.

Invalidation Strategies

Eviction handles a full cache, but invalidation handles stale data. If you use a Cache-Aside strategy, what happens when a user updates their profile picture? The write goes to the database, but the old picture might remain in the cache, serving stale content to other users.

There are two primary ways to solve this:

1. Time-Based Invalidation (TTL): This is the simplest approach. By setting a short TTL on cached items (e.g., 5 minutes), you guarantee that stale data will be purged automatically after a short period. This is often "good enough" for non-critical data.

2. Explicit Invalidation (Invalidate-on-Write): When data is updated in the database, the application code takes explicit action to remove the corresponding entry from the cache.

   • In a Write-Through pattern, this is handled automatically.
   • In a Cache-Aside pattern, your update logic would look like this:
     1. Update the data in the database.
     2. Send a command to DELETE the corresponding key from the cache.
   • The next read for that data will be a cache miss, forcing a fetch of the fresh data from the database, which then repopulates the cache.

This article provides a good overview of these invalidation approaches.

4. Conclusion: Choosing Your Strategy

There is no single "best" caching strategy. The right choice is a trade-off between performance, data consistency, and implementation complexity. The article you just read provides a helpful decision matrix.

Here's a simplified way to think about it:

• Need good performance for reads and can tolerate slightly stale data? Start with Cache-Aside and a reasonable TTL. This is the most common and versatile pattern.
• Need absolute data consistency and can tolerate slower writes? Use Write-Through.
• Need blazing-fast writes and can risk losing some data on a crash? Use Write-Behind.

Key Takeaways:

• Caching improves performance by storing frequently accessed data in fast in-memory storage like Redis.
• Cache-Aside is the most common strategy, where the application manages fetching data from the database on a cache miss.
• Write-Through ensures data consistency by writing to both cache and database synchronously, at the cost of write speed.
• Write-Behind offers the fastest writes by updating the database asynchronously, but risks data loss.
• Read-Through simplifies application logic by delegating database fetching to the cache provider.
• You must have strategies for managing your cache, including eviction policies (like LRU, TTL) to handle a full cache and invalidation strategies to handle stale data.

Preview of the Next Lesson:
We have covered the essential theory. In the next lesson, "Implement caching in Django using Redis as the cache backend," we will put this knowledge into practice. You'll configure Django to use the same Redis server from our Celery setup as a cache backend and implement the Cache-Aside strategy to speed up one of our API endpoints.