CMake for Multi-File C Projects

Why a Build System Generator?

When a project consists of a single source file, compiling it is simple: gcc main.c -o my_app. However, our project will involve dozens of files, libraries, and platform-specific settings. A build system automates this complexity.

While Make is a classic tool for this, it requires you to write Makefiles manually, which can be platform-specific and tedious. CMake operates a level above this: it's a build system generator. You write simple, platform-agnostic configuration files, and CMake uses them to generate native build files (like Makefiles on Linux, Ninja files, or even Visual Studio projects on Windows).

This video provides an excellent overview of this progression from direct compilation to using CMake.

The CMake Workflow: Out-of-Source Builds

A core principle when using CMake is the out-of-source build. This means all temporary files generated during compilation (object files, Makefiles, the final executable) are kept in a separate directory, typically named build. This keeps your source code directory clean and pristine.

The standard workflow is:

1. Create a build directory inside your project's root.
2. Change into the build directory.
3. Run cmake .. to point CMake to the root CMakeLists.txt file in the parent directory. This configures the project and generates the native build files.
4. Run the build tool itself (e.g., make or ninja) to compile your code.

This next video segment demonstrates this workflow and introduces the most basic CMakeLists.txt file.

Professional Project Structure

For a project of this scope, a well-organized directory structure is non-negotiable. The standard convention separates public header files from implementation source files. This modularity is key for managing complexity and creating reusable libraries.

A common and effective structure looks like this:

The key principles are:

• A top-level CMakeLists.txt defines the overall project.
• The src directory contains all .c implementation files and its own CMakeLists.txt to define build targets (libraries, executables).
• The include directory contains all public .h header files. It's common practice to have a subdirectory inside include with your project's name (e.g., include/hnsw_from_scratch/) to prevent file name collisions when your library is used by other projects.

This guide on Modern CMake provides a great written explanation of this structure.

Defining and Linking Targets

The core of a multi-file project in CMake involves defining targets and the dependencies between them. A target can be an executable or a library. For our project, we will define:

1. A library (hnsw_core) that will contain all the core logic for our vector index.
2. An executable (hnsw_app) that will act as a testbed or client for our library.

To do this, we need a few more crucial CMake commands:

• add_library(): Defines a library target from a set of source files.
• add_subdirectory(): Instructs CMake to process a CMakeLists.txt file in a subdirectory.
• target_include_directories(): Tells a target where to find its header files.
• target_link_libraries(): Links one target against another (e.g., linking our executable against our library).

The following video provides a clear, practical demonstration of setting up a project with an internal library and an executable. Although it uses C++, the CMake commands and concepts are identical for C.

Your Turn: Building the Project Skeleton

Now, let's apply these concepts to create the initial structure for our hnsw_from_scratch project.

1. Create the following directory and file structure:

   hnsw_from_scratch/
   ├── CMakeLists.txt
   ├── include/
   │   └── hnsw_from_scratch/
   │       └── hnsw.h
   └── src/
       ├── CMakeLists.txt
       ├── hnsw.c
       └── main.c
   

2. Populate the files with this initial content:

   include/hnsw_from_scratch/hnsw.h:

   
   
   
   #ifndef HNSW_H
   #define HNSW_H
   
   // A function to be defined in our library
   void say_hello_from_library(void);
   
   
   
   
   #endif // HNSW_H
   

   src/hnsw.c:

   
   
   
   #include "hnsw_from_scratch/hnsw.h"
   #include <stdio.h>
   
   void say_hello_from_library(void) {
       printf("Hello from the hnsw_core library!\n");
   }
   

   src/main.c:

   
   
   
   #include "hnsw_from_scratch/hnsw.h"
   
   int main(void) {
       say_hello_from_library();
       return 0;
   }
   

3. Write the CMake configuration files:

   Top-level CMakeLists.txt:

   
   
   
   # Specify the minimum version of CMake required.
   cmake_minimum_required(VERSION 3.10)
   
   
   
   
   # Define the project name and specify the language (C).
   project(HNSW_From_Scratch C)
   
   
   
   
   # Set the C standard to C11 and enforce it.
   set(CMAKE_C_STANDARD 11)
   set(CMAKE_C_STANDARD_REQUIRED ON)
   
   
   
   
   # Tell CMake to look for targets in the 'src' subdirectory.
   add_subdirectory(src)
   

   src/CMakeLists.txt:

   
   
   
   # Define a library target named 'hnsw_core' from the source file hnsw.c.
   add_library(hnsw_core hnsw.c)
   
   
   
   
   # Specify the include directories for the 'hnsw_core' target.
   # PUBLIC means that any target linking against hnsw_core will also get this include path.
   # ${PROJECT_SOURCE_DIR} is a CMake variable pointing to the project's root directory.
   target_include_directories(hnsw_core
       PUBLIC
           ${PROJECT_SOURCE_DIR}/include
   )
   
   
   
   
   # Define an executable target named 'hnsw_app' from the source file main.c.
   add_executable(hnsw_app main.c)
   
   
   
   
   # Link our executable against our library.
   # PRIVATE means the dependency is needed to build hnsw_app, but not anything that links to it.
   target_link_libraries(hnsw_app PRIVATE hnsw_core)
   

With this structure in place, you can now compile and run your project using the out-of-source build workflow:

# From the hnsw_from_scratch/ directory
mkdir build
cd build
cmake ..
make
./src/hnsw_app 

If everything is configured correctly, you should see the output: Hello from the hnsw_core library!.

Conclusion

In this lesson, you have laid the essential groundwork for our entire project. You now have a scalable, professional build environment that can handle the increasing complexity of our vector search engine.

The key takeaways are:

• CMake is a cross-platform build system generator that automates compilation using CMakeLists.txt files.
• Out-of-source builds (mkdir build && cd build && cmake ..) are crucial for keeping your source directory clean.
• A good project structure separates interfaces (include/) from implementations (src/).
• CMake projects are built from targets (add_library, add_executable) which are connected using commands like target_link_libraries and target_include_directories.

In our next lesson, we will dive into the first piece of performance-critical code: implementing a custom arena memory allocator. With our build system ready, we can immediately start writing, compiling, and testing this new component within the hnsw_core library.