Multi-file C++ Programs: Structure and Dependencies

1. From a Single File to a Modular Project

For small programs, writing all your code in a single main.cpp file is simple. However, as projects grow, this becomes unmanageable. Imagine an AI framework like PyTorch or TensorFlow, which contains millions of lines of code. A single-file approach would be impossible to navigate, maintain, or collaborate on.

The solution is separate compilation, a cornerstone of C and C++. We break a large program into logical, self-contained units called modules. Each module typically consists of:

• A header file (.h or .hpp): This is the public interface. It contains declarations of functions, classes, and variables that the module provides for others to use. A declaration tells the compiler what exists, but not how it works.
• A source file (.cpp): This is the private implementation. It contains the definitions of the functions and variables declared in the header. A definition provides the actual logic or allocates the storage.

This separation is powerful. It allows you to change the implementation in a .cpp file and only recompile that single file, not the entire project. The other parts of the program, which only depend on the unchanged header file declarations, can be linked with the newly compiled module, drastically speeding up development.

2. The Four Stages of Compilation

When you type a command like g++ my_program.cpp -o my_program, the compiler is actually performing a sequence of four distinct steps. Understanding these stages is key to debugging build issues, especially link-time errors which are common in multi-file projects.

Let's explore each stage. The following video by Mike Shah provides a comprehensive walkthrough of this entire pipeline. We will watch it in segments as we discuss each part.

Stage 1: Preprocessing

The preprocessor handles directives that start with a #. The most important of these is #include. When the preprocessor sees #include "my_header.h", it literally finds that file and pastes its entire content into the current file. The output of this stage is a single, "pure" C++ source file with no # directives.

This copy-paste behavior can cause problems. If a header file is included more than once in the same source file (often indirectly through other headers), you'll get "multiple definition" errors. To prevent this, we use header guards.

#ifndef MY_HEADER_H
#define MY_HEADER_H

// All your declarations go here...
class MyClass { ... };
int myFunction(int);




#endif // MY_HEADER_H

The first time the preprocessor sees this file, MY_HEADER_H is not defined, so it defines it and processes the file's content. On any subsequent inclusion within the same compilation unit, MY_HEADER_H is already defined, and the preprocessor skips the entire file content.

Stage 2: Compilation

The compiler takes the preprocessed C++ code and translates it into platform-specific assembly code. This is where the compiler checks for syntax errors, performs type checking, and does most of its optimization work.

Stage 3: Assembly

The assembler takes the assembly code and translates it into machine code, also known as object code. This code is stored in an object file (typically with a .o or .obj extension). This file contains the binary instructions for your functions but isn't a complete program yet. It may have references to functions or variables defined in other modules that it doesn't know the final memory address of yet.

Stage 4: Linking

The linker is the final stage. Its job is to take one or more object files and "link" them together to produce a single executable file. It performs two main tasks:

1. Symbol Resolution: For every function call, the linker finds the object file containing that function's definition and connects the call to it. If it can't find a definition for a function that was declared and used, it produces an "undefined reference" or "unresolved external symbol" error. This is a very common link-time error.
2. Relocation: The linker arranges all the object files in a final memory layout and updates all the function and variable addresses to their final locations.

The following video provides a fantastic mental model for this process, clearly distinguishing between compile-time and link-time errors.

3. Compiling a Multi-File Program Manually

Let's put this theory into practice. Imagine a simple project with three files:

• math_functions.h: Declares a function add.
• math_functions.cpp: Defines the add function.
• main.cpp: Uses the add function.

math_functions.h

#ifndef MATH_FUNCTIONS_H
#define MATH_FUNCTIONS_H

int add(int a, int b);




#endif

math_functions.cpp

#include "math_functions.h"

int add(int a, int b) {
    return a + b;
}

main.cpp

#include <iostream>
#include "math_functions.h"

int main() {
    std::cout << "3 + 4 = " << add(3, 4) << std::endl;
    return 0;
}

To build this, we follow the stages:

1. Compile main.cpp into main.o. The -c flag tells g++ to stop after the assembly stage and produce an object file.

   g++ -c main.cpp -o main.o
   

   This works because main.cpp includes math_functions.h, so the compiler knows that a function named add exists (its declaration). It trusts that the linker will find the definition later.

2. Compile math_functions.cpp into math_functions.o.

   g++ -c math_functions.cpp -o math_functions.o
   

3. Link main.o and math_functions.o into an executable.

   g++ main.o math_functions.o -o my_app
   

   The linker now takes both object files. It sees that main.o needs the add function and finds its definition in math_functions.o. It resolves the symbol and produces the final my_app executable.

The following segments from Mike Shah's video walk through exactly this process, including how to handle header guards to prevent errors.

4. Linking External Libraries

Your work in high-performance computing will almost always involve using pre-built libraries (e.g., CUDA libraries like cuBLAS, Intel's MKL, or graphics libraries). The linking process is similar, but you need to tell the compiler where to find the library's header files and the library's object code.

There are three key flags for this:

• -I<path>: (Capital 'I') Specifies an additional directory to search for header files. e.g., -I/usr/local/include.
• -L<path>: (Capital 'L') Specifies an additional directory to search for library files. e.g., -L/usr/local/lib.
• -l<name>: (Lowercase 'L') Specifies the name of the library to link. The compiler automatically prepends lib and appends .so (on Linux) or .a. For example, -lm links the math library libm.so.

This next segment of the video demonstrates this with a graphics library, but the principle is identical for any external library you'll use in AI.

5. Automating the Build with make

As you can see, typing these commands manually is tedious and error-prone. For any real project, we use a build system to automate this process. The most classic tool for this on Unix-like systems is make.

A Makefile is a script that defines rules for building your project. Each rule has a target (the file to be built), dependencies (files needed to build the target), and commands (the shell commands to run). make is smart enough to only re-run commands if a dependency has changed.

Let's analyze the Makefile from the article.

# Compiler and flags
CC = g++
CFLAGS = -Wall -std=c++17




# Default target
all: main




# Build the final binary
main: main.o mathfun.o
    $(CC) $(CFLAGS) main.o mathfun.o -o main




# Compile main.cpp
main.o: main.cpp mathfun.h
    $(CC) $(CFLAGS) -c main.cpp




# Compile mathfun.cpp
mathfun.o: mathfun.cpp mathfun.h
    $(CC) $(CFLAGS) -c mathfun.cpp




# Clean build artifacts
clean:
    rm -f *.o main

When you type make, it looks for the first target, all, which depends on main. The main target depends on main.o and mathfun.o. make sees this, then checks the rules for main.o and mathfun.o, running their compilation commands if necessary. Finally, once the .o files are up-to-date, it runs the linking command for the main target. This automates our entire manual process.

Conclusion

This lesson completes our journey through the essential systems programming foundations of C++. We've dissected the process of turning human-readable source code into a machine-executable program, a critical skill for building robust and performant applications.

Key Takeaways:

• Separate Compilation: Projects are split into modules (header/source pairs) for maintainability and faster builds. Headers declare interfaces (.h), and source files define implementations (.cpp).
• The Build Pipeline: Building an executable is a four-stage process: preprocessing (handling # directives), compilation (C++ to assembly), assembly (assembly to object code), and linking (combining object files into an executable).
• Compile vs. Link Errors: Syntax errors are caught by the compiler. "Undefined reference" errors are caught by the linker when it can't find a function's definition.
• Core Commands: g++ -c file.cpp compiles a source file into an object file. g++ file1.o file2.o -o program links object files together.
• External Libraries: The -I, -L, and -l flags are used to tell the linker where to find the headers and object code for third-party libraries.
• Automation: Build systems like make automate the compilation and linking process based on declared dependencies.

We have now covered pointers, memory management, bitwise operations, templates, and the build process. With this solid software foundation, we are ready to look deeper into the hardware. In the next module, we will begin our exploration of computer architecture, starting with the heart of any traditional computer: the CPU. We'll examine how it executes instructions and manages memory, setting the stage for contrasting it with the massively parallel architecture of the GPU.