AoS vs. SoA Memory Layout

The Default: Array of Structures (AoS)

When we think about a collection of objects in C++, our first instinct is usually to define a struct or class for the object and then place it in a container like std::vector.

For example, to represent a collection of particles in a simulation, we'd naturally write:

// A single particle object
struct Particle {
    float x, y, z;       // Position
    float vx, vy, vz;    // Velocity
    float mass;
};

// A collection of many particles
std::vector<Particle> particles;

This is the Array of Structures (AoS) pattern. It's an array where each element is a complete structure. This is intuitive, readable, and maps directly to object-oriented thinking.

In memory, this vector would look like a series of Particle objects laid out one after another.

Let's watch a short video that provides a clear, hand-drawn visualization of this layout and begins to hint at the performance problem.

As the video explained, the problem arises when our processing loop doesn't need all the data in the struct. Imagine a loop that only updates the particles' positions based on their velocities. For each particle, we only need x, y, z and vx, vy, vz. However, because we're using an AoS layout, when we access particles[i], the CPU loads a 64-byte cache line that contains not just the position and velocity, but also the mass and potentially data from the next particle.

This inefficiency is at the heart of the problem. We are paying the high cost of a memory transfer to load data into the cache, but a significant portion of that loaded data is irrelevant to our current task. This wastes memory bandwidth and pollutes the cache with data we don't need.

The following article provides a concise explanation of this exact issue.

The Alternative: Struct of Arrays (SoA)

If AoS is inefficient for this type of access pattern, what's the alternative? We restructure our data to match how we access it. This leads us to the Struct of Arrays (SoA) pattern.

Instead of an array of Particle structs, we create a single struct that contains arrays of each component:

// A collection of particles, organized by data type
struct Particles {
    std::vector<float> x, y, z;       // Positions
    std::vector<float> vx, vy, vz;    // Velocities
    std::vector<float> mass;
};

Now, all x positions are in one contiguous block of memory, all y positions are in another, and so on. Look again at the "AoS vs SoA visualization" diagram above. The bottom half shows this SoA layout, where all the X components are grouped, then all the Ys, then all the Zs.

Let's return to the video to see how this SoA layout helps.

The benefits of the SoA layout are a direct consequence of the CPU cache principles we learned in the last lesson:

1. Full Cache Line Utilization: When our code needs to update particle positions, it loops through the x vector. The CPU loads a cache line, and that cache line is packed only with x position data. Every byte transferred is useful. This maximizes our use of the expensive memory bandwidth. The diagram "AoS and SoA Memory Layouts" clearly shows this on the "Structure of Arrays" side.

2. Hardware Prefetching: Because we are iterating through a contiguous array (e.g., particles.x), the CPU's hardware prefetcher recognizes this simple, linear memory access pattern. It will begin fetching the next cache lines of x data into the cache before the code even asks for them, effectively hiding the memory latency and leading to a chain of cache hits.

3. SIMD Vectorization: This is a powerful optimization that your background in CS and engineering will help you appreciate. Modern CPUs have special instructions called SIMD (Single Instruction, Multiple Data) that can perform the same operation on multiple pieces of data at once. For example, a single SIMD instruction could add 4 or 8 floats from one array to 4 or 8 floats from another. This is like getting parallel computation for free. The SoA layout, with its contiguous arrays of homogeneous data, is the ideal format for the compiler to automatically generate (or for a programmer to manually write) these highly-efficient SIMD instructions. The AoS layout makes this much harder, as the data is interleaved.

The Trade-Offs: When to Use Which

SoA sounds amazing, but it's not a silver bullet. The best layout depends entirely on your data access patterns.

The benchmark you just read provides the crucial insight. The choice between AoS and SoA boils down to this:

• Use AoS when: You typically access all or most fields of a single object at once. Think of random access, like fetching a specific user's complete profile from a database. Adding or removing whole objects is also simpler.
• Use SoA when: You typically access the same one or two fields across a large number of objects. This is common in scientific computing, game development, and high-performance graphics—like your whiteboard app, where a render loop might process the position of thousands of stroke points every frame.

For a final, detailed comparison, let's turn to Vittorio Romeo's talk.

Conclusion

Today, we have directly contrasted the two primary ways of organizing collections of data in memory. You now have the conceptual tools to analyze a data structure not just for its logical correctness, but for its mechanical sympathy with the underlying hardware.

Key Takeaways:

• Array-of-Structs (AoS): std::vector<MyStruct>. Lays out entire objects contiguously. It's intuitive and good for accessing a whole object at a time. Its weakness is poor cache utilization when only a subset of fields is needed.
• Struct-of-Arrays (SoA): struct { std::vector<Type1> field1; std::vector<Type2> field2; }. Lays out all instances of a single field contiguously. It's extremely cache-efficient for batch operations on a subset of fields and is ideal for hardware prefetching and SIMD vectorization.
• The Choice is Contextual: The optimal layout depends entirely on the access patterns of your performance-critical code. AoS is for whole-object access; SoA is for single-field access across many objects.

For your whiteboard project, a single user stroke is composed of many points, each with (x, y, pressure). When rendering, we need to process the x, y, and pressure for thousands, or even millions, of points across the entire canvas. This sounds like a classic case where the SoA pattern could provide a significant performance boost.

In our next lesson, we will analyze the specific performance trade-offs of AoS vs. SoA for storing and processing our stroke data. We'll consider different operations like rendering, selection, and undo/redo to make an informed decision before we start refactoring our code.