Hello!
In our last lesson, we discussed JWT security and mentioned how a fast, in-memory store like Redis could be used to maintain a token denylist. That was just one specific application of a much broader and more fundamental concept in backend engineering: caching.
Welcome to a new module focused on application-level caching. Caching is one of the most powerful techniques in your toolkit for building fast, scalable, and cost-effective applications. Understanding it deeply is non-negotiable for any serious backend developer.
This first lesson lays the groundwork. We will explore the fundamental principles of caching to address the learning outcome: Explain why application-level caching improves performance, and define cache hit, cache miss, and hit ratio.
By the end of this lesson, you will understand:
- The core principle that makes caching work across all of computing.
- Why caching is critical for reducing latency and database load in a web application.
- The essential terminology used to describe and measure the effectiveness of a cache.
Let's begin.
The Fundamental Idea: Storing Things Closer
At its heart, caching is about a simple trade-off: using a small amount of fast, expensive storage to avoid accessing a large amount of slow, cheaper storage. The goal is to keep frequently accessed data closer to where it's needed.
A great analogy is doing research at a library. To get a book, you have to travel to the library, find it in the stacks, and bring it back. If you need it again, you could repeat the trip. Or, you could keep the book on your desk. Your desk is a small, limited-space, but extremely fast-access "cache" for the library's vast, but slow-to-access, collection.
This same principle applies throughout computer systems. The following video provides an excellent introduction to this idea.
This video from the Simply Explained channel, 'Caching - Simply Explained', uses the library analogy and explains the concept of a memory hierarchy in hardware, which is the foundation for why caching is so effective.
Watch the first 3 minutes and 28 seconds of the video. Pay close attention to: The library analogy and the trade-off between speed and size. The concept of a 'memory hierarchy' in computer hardware (CPU cache vs. RAM vs. Disk).
As the video explains, modern computers have many layers of caches, from tiny, ultra-fast caches inside the CPU to your computer's main RAM. A backend system introduces another layer: an application-level cache.
Why Application-Level Caching Matters
In a backend application, the "slow library" is often your primary database (like PostgreSQL) or an external API you depend on. While databases are highly optimized, they are almost always the slowest part of a request-response cycle for a few key reasons:
- Disk I/O: Most databases store the bulk of their data on disk (SSD/HDD). Accessing data from a disk is orders of magnitude slower than accessing it from memory (RAM).
- Network Latency: The database often runs on a different server from your application, so every query involves a network round-trip.
- Computational Cost: Complex queries involving joins, sorting, or aggregations require significant CPU work by the database server.
Consider a request to load a social media feed. As the following article explains, this single action can trigger dozens of separate database queries, putting immense strain on the database.
What Is Caching | System Design - AlgoMaster.io
This article, 'What Is Caching?', provides a concrete example of why caching is so critical in a real-world application and quantifies the performance impact.
Read the sections 'Why Caching Matters' and 'Performance Impact'. The table in this section clearly illustrates the 10x or more improvement caching can provide across response time, database load, and even cost.
The solution is to introduce a cache that sits between your application and the database. This is typically an in-memory data store like Redis or Memcached. Because these tools store data entirely in RAM, they are incredibly fast.
When your application needs data, it first checks the cache. If the data is there, it's returned immediately, and the slow database query is completely avoided. This dramatically improves performance in two ways:
- Reduces Latency: The user gets a response much faster because the application doesn't have to wait for the database.
- Reduces Backend Load: The database has to do far less work, which means it can handle more concurrent users and you may be able to run it on smaller, cheaper hardware.
The Mechanics: Cache Hit vs. Cache Miss
When your application looks for a piece of data in the cache, one of two things can happen. These two scenarios are the most fundamental concepts in caching.
- Cache Hit: The requested data is found in the cache. This is the ideal scenario. The application retrieves the data directly from the fast cache and returns it to the user.
- Cache Miss: The requested data is not in the cache. The application must perform the "slow" operation: it queries the primary database for the data. Before returning the data to the user, it typically stores a copy in the cache for next time.
The following diagram perfectly illustrates these two flows.
Notice the difference in the number of steps. A cache hit is a simple two-step process (request -> cache -> response), while a cache miss involves the full, multi-step round trip to the database.
Measuring Effectiveness: The Cache Hit Ratio
Simply having a cache isn't the whole story. You need to know if it's actually effective. The single most important metric for this is the Cache Hit Ratio.
The hit ratio is the percentage of requests that result in a cache hit. It's calculated with a simple formula:
Let's turn back to the AlgoMaster.io article to understand what makes a good hit ratio.
What Is Caching | System Design - AlgoMaster.io
This section of the article defines the Cache Hit Ratio and provides a useful table for interpreting its value.
Read the sections 'Cache Hit and Miss' and 'Cache Hit Ratio'. Pay attention to the interpretation table. A high hit ratio has a massive impact on your system's overall capacity.
As the article states, a hit ratio above 90% is typically considered good to excellent. The impact is exponential: a 90% hit ratio means that 9 out of 10 requests are served at lightning speed and never even touch your database. This effectively increases the capacity of your application by 10x without changing the database at all.
Conversely, a low hit ratio (< 50%) suggests your caching strategy isn't working well. The data might not be suitable for caching, or the cache size might be too small.
A Practical Demonstration
Theory is great, but seeing the impact is better. The following video demonstrates adding Redis caching to a simple application. Although the code is in Node.js, the principles are universal and the performance difference is undeniable.
The 'Redis Crash Course' video by Web Dev Simplified provides a fantastic real-world example. It shows exactly how a cache fits into an application and the dramatic performance improvement it yields.
Watch from 01:22 to 03:04, and then from 13:09 to 20:42. Focus on: Why Redis is so fast (in-memory vs. disk). Where Redis sits architecturally (between the app and the database). In the demo, observe the response time before caching (~400ms) and after caching (~40ms). Notice the 'cache miss' log on the first request and 'cache hit' on subsequent requests.
The demo makes it clear: a simple caching layer can make your API endpoints an order of magnitude faster.
Conclusion
In this lesson, we established the fundamental principles of application-level caching. Let's summarize the key takeaways:
- Caching improves performance by storing frequently accessed data in a fast, in-memory store (like Redis), avoiding slow operations like database queries or external API calls.
- This reduces latency for the user and decreases the load on your backend systems, particularly the database.
- A Cache Hit occurs when data is found in the cache, resulting in a fast response.
- A Cache Miss occurs when data is not in the cache, forcing a slower fetch from the primary data source.
- The Cache Hit Ratio is the critical metric for measuring cache effectiveness. A high hit ratio (>90%) dramatically increases your application's throughput and perceived performance.
You now understand why caching is so important and the basic terminology to describe it.
Preview of the Next Lesson
Knowing why we cache is the first step. The next is understanding how. There are several common strategies for deciding when and how to read from and write to the cache. In our next lesson, we will "Describe caching patterns: cache-aside, write-through, and read-through." We'll explore these different approaches and their respective trade-offs, moving you closer to implementing an effective caching strategy in your own applications.