Welcome to the first lesson of your C++ journey for deep learning! This course is designed to equip you with the skills to write high-performance numerical code, moving from the high-level comfort of Python frameworks to the low-level power of C++.
Today, we'll focus on setting up your workshop. Just as a data scientist needs a well-configured environment with libraries like NumPy and PyTorch, a C++ developer needs a robust toolchain. By the end of this lesson, you will be able to set up a modern C++ compiler, use the CMake build system to manage a project with multiple files, and get acquainted with essential debugging and code-checking tools. This foundational setup will be the bedrock for everything we build throughout this course.
The C++ Build Process: From Source to Executable
Unlike Python, which is typically interpreted, C++ is a compiled language. This means your human-readable source code must be translated into machine-readable instructions that the CPU can execute directly. This translation process is handled by a compiler.
The overall process, often called the build process, involves several stages. At a high level, the compiler takes your .cpp source files and turns them into intermediate "object files" (.o). Then, a program called a linker combines these object files, along with any pre-compiled libraries you're using, into a single final executable file.

As projects grow beyond a single file, managing this process manually becomes tedious and error-prone. This is where build systems come in.
Core Components of the Toolchain
A modern C++ toolchain consists of several key components that work together. We will set up and explore each of them.
1. The Compiler: GCC and Clang
The two dominant compilers in the C++ world, especially on Linux, are:
- GCC (GNU Compiler Collection): The long-standing standard, with
g++being its C++ compiler. - Clang: A more modern compiler built on the LLVM infrastructure, known for its fast compilation, excellent diagnostics (error messages), and strong integration with other development tools.
For this course, we'll primarily use Clang, but the concepts and most commands are nearly identical for GCC. Both are excellent choices for high-performance computing.
To get started, you'll need to install these tools. The following guide provides the necessary commands for an Ubuntu-based system, which you can use natively or via the Windows Subsystem for Linux (WSL).
How to Configure a C/C++ Development Environment on Ubuntu
This article provides a straightforward guide to installing the essential C++ development tools on Ubuntu.
Focus on the first two main sections. First, read about installing the compilers. Pay attention to the commands for installing build-essential (which includes GCC/G++) and clang. You can skip the parts about specific version management for now.
Once you have a compiler installed, you can compile a simple program directly from your terminal. For example, given a file hello.cpp, you would compile it with Clang like this:
clang++ -std=c++17 -o hello hello.cpp
The -std=c++17 flag tells the compiler to use the C++17 standard, and -o hello specifies that the output executable should be named hello. The next video discusses these flags and others that are crucial for writing robust code.
CPP-02 Modern C++: Compilation, Debugging, Functions, Header/Source, Libraries, CMake (2018, Igor)
This video covers some of the most important compiler flags you'll use daily. Understanding these will help you catch bugs early and control code optimization.
Watch the segment from compilation flags. The instructor discusses warning flags like -Wall (show all warnings) and -Werror (treat warnings as errors), and the optimization flag -O3. You can skim the part about assembly code for now; the key takeaway is that optimization flags significantly alter the final machine code for better performance.
2. The Build System: CMake
While direct compiler commands work for single files, any real project will involve multiple source files, libraries, and complex dependencies. A build system automates the process of compiling and linking these components correctly.
We will use CMake, which is the de-facto standard for modern C++. CMake is a meta-build system; it uses simple configuration files you write to generate the native build files for your platform (e.g., Makefiles on Linux or Visual Studio projects on Windows). This makes your project portable and easy to manage.
This video from the Visual Studio Code team gives a great introduction to why we need a build system and how CMake fits in.
C++ in VS Code: Building your Code with CMake
This video provides a concise and clear walkthrough of setting up a simple C++ project with CMake in Visual Studio Code.
Watch the entire video. It covers: What a build system is and why CMake is a popular choice. Installing the CMake Tools extension in VS Code. The structure of a CMakeLists.txt file, which is the heart of a CMake project. Using a CMakePresets.json file to define build configurations. The process of configuring, building, and running the project.
The core of a CMake project is the CMakeLists.txt file. Let's look at a simple but complete example for a project with a library and an executable. Imagine this file structure:
my_project/
├── CMakeLists.txt
├── app/
│ └── main.cpp
└── src/
├── math_ops.cpp
└── math_ops.h
A CMakeLists.txt file for this project might look like this:
# Set the minimum required version of CMake
cmake_minimum_required(VERSION 3.16)
# Define the project
project(MyMathProject LANGUAGES CXX)
# Set the C++ standard for the whole project
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
# Add a library from our source files
add_library(math_ops src/math_ops.cpp)
# Make sure the compiler can find the header files for our library
target_include_directories(math_ops PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/src)
# Add an executable from our application file
add_executable(my_app app/main.cpp)
# Link the executable against our library
target_link_libraries(my_app PRIVATE math_ops)
To build this project, you would typically use an "out-of-source" build:
# From the my_project/ directory
mkdir build
cd build
cmake ..
cmake --build .
This sequence generates the build files in the build directory and then invokes the native build tool (like make) to compile and link your application, my_app. This keeps your source directory clean.
For a deeper dive into organizing code into header and source files and linking them into libraries and executables, the following resource is helpful.
CPP-02 Modern C++: Compilation, Debugging, Functions, Header/Source, Libraries, CMake (2018, Igor)
This part of the lecture explains the separation of concerns between header (.h) and source (.cpp) files, how they form libraries, and how CMake simplifies managing this multi-file structure.
Watch from Functions and file organization, where the instructor discusses creating functions and separating declarations into headers and definitions into source files. Then, watch the section from introduction to CMake to see how it automates the complex process of linking these files together.
3. The Debugger: GDB
Coming from Python, you are likely used to a development cycle where errors are reported immediately. In C++, bugs can be more subtle, leading to crashes or incorrect results far from the actual source of the error. A debugger is an indispensable tool that lets you pause your program's execution, inspect the state of variables, and step through the code line by line.
The standard debugger on Linux is GDB (the GNU Debugger). To use it, you must compile your code with the -g flag, which tells the compiler to include debugging information in the executable.
# Compiling with debug symbols
clang++ -std=c++17 -g -o my_app main.cpp
While GDB is a powerful command-line tool, most developers use it through an IDE like VS Code, which provides a much more intuitive graphical interface for setting breakpoints and inspecting variables.
How to Configure a C/C++ Development Environment on Ubuntu
This article gives a quick tour of GDB and how to integrate it with VS Code.
First, read the section on Essential GDB Commands to familiarize yourself with basic concepts like running (run), setting breakpoints (break), and stepping through code (next, step). Then, look at the launch.json configuration in the section on VS Code setup, which shows how to configure the VS Code debugger to launch your compiled program with GDB.
4. The Sanitizers: Your Guardian Angels
C++ gives you direct control over memory, but this power comes with responsibility. A whole class of bugs, such as accessing an array out of bounds or using memory after it has been freed, can lead to unpredictable behavior and security vulnerabilities. These are often called undefined behavior (UB).
Sanitizers are powerful tools, integrated into the compiler, that add runtime checks to your code to automatically detect these kinds of bugs. They are a modern C++ developer's best friend. The two most important ones to start with are:
- AddressSanitizer (ASan): Detects memory errors like buffer overflows (stack, heap, or global), use-after-free, and memory leaks.
- UndefinedBehaviorSanitizer (UBSan): Catches other forms of undefined behavior, like signed integer overflow, invalid bit shifts, and misaligned memory access.
You enable sanitizers with a compiler flag. For a typical debug build, you would combine them.
clang++ -std=c++17 -g -fsanitize=address,undefined -o my_app main.cpp
When you run a program compiled with sanitizers, it will immediately crash with a detailed report if it detects an error, pointing you to the exact line of code where the bug occurred.
The following handbook is an outstanding, in-depth guide to sanitizers. We will refer to it frequently throughout the course. For now, we will focus on the fundamentals and how to integrate them into our CMake workflow.
The Complete C/C++ Sanitizers Handbook
This is a comprehensive guide to understanding and using sanitizers in C++. It's one of the most valuable resources for writing correct and robust code.
To begin, read these introductory sections to understand the "what" and "why": What sanitizers are Why they are crucial in C++ The main types of sanitizers How to choose which sanitizer to use Next, focus on how to enable them with the correct flags and best practices: Read about typical usage for ASan. Skim section 7 to understand what UBSan does. Read about recommended compiler flags for getting useful sanitizer reports. The key flags are -g, -fno-omit-frame-pointer, and a moderate optimization level like -O1. Finally, see how to integrate this into a modern CMake project. This is the pattern we will adopt. Read Part II, Section 2 to learn the best practice for adding sanitizer flags in CMake using an INTERFACE target. Read Part II, Section 3 to see how CMakePresets.json can be used to create specific build configurations, like one for sanitizers. Study the complete, practical examples in Part III: The Sanitizers.cmake module. The example top-level CMakeLists.txt that uses the module. The corresponding CMakePresets.json file.
Using sanitizers in your development and testing builds is not optional; it is a professional standard that will save you countless hours of debugging.
Conclusion
In this lesson, we have assembled the essential toolchain for professional C++ development. You now have a conceptual and practical understanding of:
- Compilers (Clang/G++): The tools that turn your source code into an executable.
- Build Systems (CMake): The standard way to automate the build process for multi-file projects in a clean and portable manner.
- Debuggers (GDB): The tool for inspecting your program's state and finding logical errors.
- Sanitizers (ASan/UBSan): A critical safety net for automatically detecting memory errors and undefined behavior at runtime.
This setup forms the foundation upon which we will build all our high-performance deep learning components. Having a robust and automated build-and-test environment is the first step toward writing reliable, high-quality code.
In our next lesson, we will start writing C++ code in earnest. We'll explore how C++ manages data in functions, focusing on references, const-correctness, and the idioms of passing arguments by value versus by reference. These concepts are fundamental to writing code that is both safe and efficient.