STL Containers for Numerical Buffers

Why Contiguous Memory Matters

Before diving into the containers themselves, it's crucial to understand why organizing data in a single, uninterrupted block of memory is so fundamental to high-performance code. Your experience optimizing models in Python has likely involved libraries like NumPy, which are fast precisely because they operate on contiguous memory arrays under the hood. In C++, we get to work with this concept directly.

The performance benefits of contiguous memory stem directly from how modern CPUs are designed.

As the text explains, sequential data access allows the CPU to predict what memory it will need next and pre-load it into its caches, drastically reducing the time spent waiting for data from main memory. This is the foundation upon which high-performance numerical libraries are built. Now, let's look at the C++ tools for creating and managing these contiguous buffers.

std::vector: The Dynamic Workhorse

std::vector is the most versatile and commonly used container in C++. It is a dynamically-sized array that stores its elements contiguously in memory allocated on the heap. This makes it the perfect tool when you need a buffer whose size isn't known at compile time or needs to change during its lifetime—for example, to hold input data of varying batch sizes.

A std::vector object itself is small (typically three pointers' worth of data), but it manages a much larger block of memory on the heap. Its key properties are its size (the number of elements it currently holds) and its capacity (the number of elements it can hold before it must reallocate its memory). Understanding the relationship between size and capacity is the key to using std::vector efficiently.

The following video gives a great practical introduction to std::vector, including its memory management and how to access its elements.

The single most important performance rule for std::vector is: if you know how many elements you're going to add, call reserve() first.

Consider filling a vector with a million elements:

// ANTI-PATTERN: Potentially many reallocations
std::vector<float> weights;
for (int i = 0; i < 1'000'000; ++i) {
    weights.push_back(get_weight(i)); // Each push_back might trigger a reallocation
}

// GOOD: One allocation, no reallocations in the loop
std::vector<float> weights;
weights.reserve(1'000'000); // Allocate memory once
for (int i = 0; i < 1'000'000; ++i) {
    weights.push_back(get_weight(i)); // Just places element in pre-allocated memory
}

Each reallocation involves allocating a new, larger block of memory, moving all existing elements from the old block to the new one, and then freeing the old block. By calling reserve(), you pre-empt this entire process, resulting in a massive performance gain.

std::array: For Fixed-Size Buffers

While std::vector is for dynamic data, many numerical components have a fixed size known at compile time. Think of a 3D position vector (x, y, z), the dimensions of a tensor, or a small convolution kernel. For these cases, std::array is the ideal choice.

std::array<T, N> is a lightweight wrapper around a C-style array (T arr[N]).

• Its size N is a compile-time constant and is part of the type itself.
• It does not allocate memory on the heap. The data is stored directly within the std::array object itself, which usually means it lives on the stack.
• This makes it extremely fast: there is zero allocation overhead, and its memory is often located in the CPU's L1 cache.

Choosing between std::vector and std::array is a fundamental design decision:

• Use std::vector when the size is unknown at compile time or needs to change. This will be your default for things like model inputs, activations, and datasets.
• Use std::array when the size is fixed and known at compile time. This is perfect for coordinates, colors, small kernels, or shape/stride information.

std::span: The Universal, Non-Owning View

So far, we have two types of owning containers. But what happens when you want to write a function that processes data from one of these containers?

// Option 1: Takes a vector
void process_data(const std::vector<float>& data);

// Option 2: Takes an array
void process_data(const std::array<float, 1024>& data);

Both of these are too restrictive. The first function can't accept an std::array, and the second can't accept a std::vector. This forces callers to create temporary copies or limits the utility of your function.

The modern C++ solution is std::span (introduced in C++20). A std::span is a non-owning view over any contiguous block of data. It's essentially a pointer and a size bundled together into a single, safe object. It does not own the memory it points to, and it never allocates.

This makes std::span the ideal type for function parameters that operate on buffers.

By using std::span, you decouple your algorithms from the specific container used to store the data, writing more generic and efficient code.

Ownership vs. View: A Summary

The distinction between owning containers (vector, array) and non-owning views (span) is a central theme in modern C++. It mirrors the distinction we saw in the last lesson between owning smart pointers (unique_ptr, shared_ptr) and non-owning raw pointers or references.

This table provides a concise summary:

Traversing Buffers with Iterators

Now that we have containers, how do we access their elements? While you can use the [] operator, the most general and powerful way to traverse an STL container is with iterators.

An iterator is an object that behaves like a pointer to an element in a container. You can get an iterator to the first element with .begin() and an iterator that points one-past-the-last element with .end(). The standard algorithms library is built entirely around iterators, allowing you to sort, search, transform, and process elements in any STL-compliant container.

Iterator Invalidation Rule for std::vector:
Any operation that changes the capacity of a std::vector (like push_back on a full vector, or reserve) invalidates all iterators, pointers, and references to its elements. Operations that change the size but not the capacity (like push_back when there is spare capacity) invalidate iterators at or after the point of insertion.

The simplest rule to remember is: if you modify a vector, assume your old iterators are no longer valid.

std::vector<int> v = {10, 20, 40, 50};
auto it = v.begin() + 2; // Points to 40

// This insertion might cause a reallocation, invalidating 'it'
v.insert(it, 30); 

// DANGEROUS: 'it' may be a dangling pointer now.
// *it = 42; // This could be undefined behavior.

Conclusion

In this lesson, we have covered the three fundamental tools for managing contiguous memory buffers in modern C++. You now have a solid foundation for representing the numerical data at the heart of any deep learning model.

Key Takeaways:

• Contiguous memory is key to performance due to CPU caching and SIMD.
• std::vector is your default choice for a dynamic, heap-allocated buffer. Always reserve() capacity when you know the size upfront to prevent costly reallocations.
• std::array is the high-performance choice for buffers with a fixed size known at compile time. It avoids heap allocation entirely.
• std::span is the modern, safe, and efficient way to pass non-owning views of contiguous data into functions, decoupling your algorithms from specific container types.
• Iterators are the generalized mechanism for traversing containers. Be acutely aware of iterator invalidation rules when modifying a std::vector.

In our next lesson, we will introduce templates, the C++ feature that allows you to write generic code that works with different data types. This will enable us to write a function like process(std::span<T> data) that can operate on buffers of float, int, or any other numerical type, a cornerstone of writing reusable library code.