Hello! Welcome to your fourth lesson in the "Multi-Core Programming Concepts" module.
In our last lesson, we explored memory barriers (DMB, DSB, ISB), which are the tools we use to enforce the ordering of memory operations. We established that these are crucial for ensuring synchronization logic works correctly on weakly-ordered architectures like ARM. However, ordering is only half the story. If a core writes to its local cache, how and when does that write become visible to other cores?
This lesson directly addresses that question, fulfilling the learning outcome: Describe cache coherency issues in multi-core systems and the role of hardware coherency mechanisms. Understanding how the hardware ensures a consistent view of memory is the bedrock upon which all multi-core software is built. For a low-level driver developer, this is non-negotiable fundamental knowledge, and a common topic in senior-level interviews.
The Problem: When Caches Disagree
Modern multi-core processors give each core its own high-speed private L1 cache to reduce memory latency. This is great for single-core performance. But in a multi-core system where cores share main memory, these private caches create a significant problem.
Imagine Core 0 reads a variable x from memory. It stores a copy of x in its private cache. Now, Core 1 reads the same variable x, storing another copy in its private cache. What happens when Core 0 modifies its local copy of x? Core 1's copy is now stale, holding an old value. If Core 1 reads x again, it will get the wrong data from its cache, leading to incorrect program behavior. This is the cache coherency problem.
To see a concrete illustration of this problem and why it's so critical for inter-core communication, please watch the beginning of the following video.
The video '4 2 1 Cache Coherence' provides a clear introduction to the cache coherence problem and uses simple, effective examples to show how data inconsistency can arise and break a program.
Please watch from the beginning to 05:52. Pay attention to: The definition of the cache coherence problem. The step-by-step example where P0 reads an old value after P3 has written a new one. The shared-flag program example, which demonstrates how a core can get stuck in an infinite loop because it never sees the updated flag value from the other core's cache.
As the video demonstrates, without a mechanism to keep caches in sync, even simple flag-based synchronization would fail. The system would not have a unified, "shared" memory but rather a collection of separate memories that happen to be initialized from the same source. To formalize this, a memory system is considered coherent if it provides a few key guarantees, including:
- A write by a processor will eventually be seen by all other processors.
- Writes to the same memory location are seen in the same order by all processors (write serialization).
The Solution: Hardware Coherency Protocols
Fortunately, hardware designers have solved this problem for us. Instead of requiring software to manually manage cache consistency (which would be incredibly complex and slow), modern multi-core processors implement a cache coherency protocol in hardware.
The most common approach in systems like ARM-based SoCs is a snooping protocol.
Before we dive into a specific protocol, let's understand the general idea of snooping from the 'Cache coherency primer' blog post. This post provides excellent context from a software developer's point of view.
Please read the section titled 'Coherency protocols'. It explains the core idea: caches are independent, but they all 'snoop' on a shared bus to monitor what other caches are doing.
The fundamental idea of snooping is that every cache controller monitors (snoops) the memory bus. When it sees a transaction for a memory address that it holds a copy of, it can take action—for example, invalidating its local copy if another core writes to that address.
The MESI Protocol
One of the most famous and widely used snooping protocols is MESI. The name is an acronym for the four states that a cache line can be in: Modified, Exclusive, Shared, and Invalid. These states allow the hardware to track the ownership and status of each cache line across the entire system.
Here is a brief overview of the states:
- Invalid (I): The cache line does not contain valid data.
- Shared (S): The cache line holds a clean copy of the data from main memory. Multiple cores can hold a copy of the same line in the Shared state. Reads can be served from the cache, but writes cannot.
- Exclusive (E): The cache line holds a clean copy of the data, and no other core has a copy. This is an optimization: since this core has the only copy, it can write to the line and transition it to the Modified state without notifying other cores.
- Modified (M): The cache line has been modified (is "dirty") and the data in main memory is stale. This core is the only one with a copy. If another core requests this data, this core must provide the up-to-date data and write it back to memory.
These states and the transitions between them form a state machine managed entirely by the hardware.
To see how these states work in practice, let's watch a detailed walkthrough.
MESI Cache Coherence Protocol | Vasileios Trigonakis
The video 'MESI Cache Coherence Protocol' by Vasileios Trigonakis is a fantastic, clear explanation of the protocol, walking through the states and their transitions step by step.
Please watch from 00:50 to the end. The video first defines each of the four states (M, E, S, I) and then explains the state transitions caused by both local core requests and remote coherence messages from other cores.
Test your understanding!
A core wants to write to a cache line that is currently in the Shared (S) state in its local cache. According to the MESI protocol, what must happen before the write can proceed?
Show answer
Because the line is Shared, other cores might also have a copy. The writing core cannot simply modify its local copy. It must first gain exclusive ownership. To do this, it will broadcast a request on the bus (often called a "Read for Ownership" or an "invalidate" message). All other cores that have a copy of that line in the Shared state will snoop this message and invalidate their copies (transitioning them to the Invalid (I) state). Once the writing core receives acknowledgements and knows it has the only valid copy, it can transition its cache line to the Modified (M) state and perform the write.
Coherency in Practice: ARM MPCore and the SCU
In ARM multi-core processors like the Cortex-A series you've worked with, this coherency logic is managed by a hardware block called the Snoop Control Unit (SCU). The SCU sits between the L1 caches of the cores and the next level of the memory system (e.g., an L2 cache or main memory).
The SCU implements the MESI protocol (or a variant) and snoops all transactions from the cores. It maintains a duplicate set of all L1 cache tags, allowing it to efficiently determine which cores hold which cache lines. ARM's implementation includes performance optimizations like Direct Data Intervention (DDI), where a request from one core for data that is "dirty" in another core's cache can be served directly from that cache (cache-to-cache transfer) without a slow round-trip to main memory.
Let's read the official ARM explanation.
Application Note 228: ARM MPCore - Using DMA with the L1 Cache Controller
This ARM Application Note is written for device driver developers like you. It explains how MESI and the SCU work in ARM MPCore processors and introduces some practical hardware optimizations.
Please read section 2 'Cache coherency in multi-core processing systems' and section 2.1 'Coherency Protocols in ARM MPCore processors'. This will ground the general MESI concept in the specific ARM hardware you'd encounter.
Beyond CPU Coherency: The I/O Problem
So far, we've only discussed coherency between CPU cores. But what about other bus masters, like a DMA controller? A DMA controller accesses memory directly, without going through a CPU core's cache. This creates a new coherency challenge, often called I/O Coherency.
Consider this driver scenario:
- Your driver code running on Core 0 prepares a data packet in a memory buffer. Because of the write-back cache, this new data might only exist in Core 0's L1 cache (in the 'Modified' state). Main memory is still stale.
- Your driver then programs a network card's DMA controller to transmit the packet from that memory buffer.
- The DMA controller reads the buffer directly from main memory and transmits the stale, old data. The new packet never leaves the SoC.
On older or simpler systems, this problem is solved in software. The driver is responsible for explicitly cleaning (flushing) the cache before an outbound DMA transfer and invalidating the cache before an inbound DMA transfer.
More advanced ARM cores offer a hardware solution: the Accelerator Coherency Port (ACP). This is a special slave port on the SCU. If you connect a DMA controller to the ACP, its memory accesses become fully coherent with the CPU caches. The SCU will automatically snoop the DMA's accesses and ensure it gets the correct data, even if it's sitting in a CPU's L1 cache.
The same ARM application note discusses these issues and solutions in detail. This is highly practical knowledge for a driver developer.
Application Note 228: ARM MPCore - Using DMA with the L1 Cache Controller
Let's continue with the ARM Application Note to understand the I/O coherency problem and the different ways to solve it.
Please read the following sections: Section 2.2 'Coherency and non-CPU bus masters': Introduces the I/O coherency problem. Section 3.2 'Using DMA in cached systems': Explains the stale data problem with DMA and contrasts the software solution (clean/invalidate) with the hardware I/O coherent solution. Section 4.3 'The Accelerator Coherency Port (ACP) and I/O Coherency': Describes the hardware solution that simplifies driver software immensely.
Understanding the difference between an I/O coherent system (with ACP) and a non-I/O coherent system is a major differentiator in an interview, as it demonstrates deep knowledge of system-level architecture and its direct impact on driver implementation.
Interview Corner
Q1: What is the cache coherency problem in a multi-core system?
A: It's the issue of ensuring that all cores have a consistent view of shared memory when each core has its own private cache. If one core modifies a shared data item in its local cache, other cores' caches holding the same data become stale, leading to them reading incorrect values.
Q2: Can you explain the MESI protocol at a high level? What do the four states mean?
A: MESI is a hardware-level snooping protocol to maintain cache coherency. Each cache line is in one of four states:
- Modified: This core has the only copy, it's dirty (different from main memory), and it's responsible for providing the data to others.
- Exclusive: This core has the only copy, but it's clean (matches main memory). It can be written to silently without notifying other cores.
- Shared: Multiple cores may have a copy of this line, and it's clean. Reads are fine, but a write requires an upgrade to an exclusive state first.
- Invalid: The cache line is not valid.
Q3: You are writing a network driver that uses DMA on a multi-core ARM SoC. What cache coherency issues must you consider?
A: The primary concern is I/O coherency. The DMA controller is a separate bus master that doesn't see the CPU caches. If my driver prepares a transmit buffer, the data might be sitting dirty in a CPU's L1 cache. If the DMA reads from main memory, it will get stale data.
The solution depends on the system architecture.
- If the system is not I/O coherent, my driver is responsible for software cache maintenance. I must explicitly clean (flush) the cache range for the buffer before starting the DMA transfer, and invalidate the cache range for a receive buffer before the CPU accesses the data brought in by DMA.
- If the system is I/O coherent, for example, using ARM's Accelerator Coherency Port (ACP), the hardware handles this. The DMA's accesses are snooped by the Snoop Control Unit (SCU), which ensures the DMA always sees the most up-to-date data, whether it's in a cache or in main memory. This greatly simplifies the driver code.
Conclusion
Today we have built on our knowledge of memory ordering by exploring memory visibility. You've seen how private caches create the cache coherency problem and how sophisticated hardware protocols like MESI, managed by units like ARM's SCU, solve it automatically.
Key Takeaways:
- Private caches in a shared memory system lead to the cache coherency problem, where different cores can see different values for the same memory location.
- Hardware snooping protocols, like MESI, solve this by having cache controllers monitor bus traffic and maintain a state for each cache line (Modified, Exclusive, Shared, Invalid).
- In ARM systems, the Snoop Control Unit (SCU) implements the coherency protocol.
- I/O Coherency is a separate but related problem concerning non-CPU bus masters like DMA controllers. This can be handled by software cache maintenance (clean/invalidate) or by hardware mechanisms like the Accelerator Coherency Port (ACP).
Next Lesson Preview:
We now have the complete hardware picture: memory barriers ensure ordering, and coherency protocols ensure visibility. With this foundation, we can finally build robust software communication mechanisms. In our next lesson, we will Implement inter-processor communication using shared memory and Inter-Processor Interrupts (IPIs).