CMake for Dynamic Shared Libraries

Static vs. Shared Libraries: A Quick Refresher

Before diving into CMake, let's briefly revisit the two main types of libraries in the C/C++ world. Given your computer science background, this will likely be a review, but it's important to frame our goal.

For our project, creating a shared library is ideal. It allows the core vector search logic to be maintained and updated independently and enables different applications—even those written in other languages like Python—to use it without being recompiled.

Instructing CMake to Build a Shared Library

By default, when you use the add_library() command in CMake, it creates a static library. We can change this behavior in a couple of ways.

The most direct method is to add the SHARED keyword to the command:
add_library(hnsw SHARED ${HNSW_SOURCES})

However, modern CMake projects typically offer a more flexible approach by allowing the user of the build system to choose the library type. This is done using a special CMake variable called BUILD_SHARED_LIBS.

To implement this, you simply add an option() to your root CMakeLists.txt file. This creates a user-configurable cache variable that controls the behavior of add_library() whenever the type isn't explicitly specified.

# In your project's root CMakeLists.txt
cmake_minimum_required(VERSION 3.16)
project(HNSW C)




# Provide a user-facing option to control the library type. Defaults to ON.
option(BUILD_SHARED_LIBS "Build libraries as shared" ON)




# ... other project-wide settings ...

add_subdirectory(src) # Assuming your library code is in a 'src' directory

With this in place, any add_library(hnsw ${HNSW_SOURCES}) call inside src/CMakeLists.txt will now produce a shared library by default, but a user can override this by running cmake -DBUILD_SHARED_LIBS=OFF ...

Controlling Symbol Visibility

One of the most important aspects of creating a high-quality shared library, especially for performance-critical systems, is controlling symbol visibility. When you build a shared library, which functions and global variables should be accessible to the outside world?

• MSVC (Windows): Hides all symbols by default. You must explicitly export your public API using __declspec(dllexport).
• GCC/Clang (Linux/macOS): Exports all symbols by default. This is convenient but dangerous, as it "leaks" internal implementation details, increasing the chances of name collisions and preventing certain compiler optimizations.

The best practice is to hide everything by default and only expose the functions in your public API. While you can do this with complex preprocessor macros, modern CMake provides a much cleaner, automated solution: the GenerateExportHeader module.

To implement this for our hnsw library, we'll add the following to its CMakeLists.txt:

# In src/CMakeLists.txt




# ... add_library(hnsw ...) ...

include(GenerateExportHeader)
generate_export_header(hnsw
    EXPORT_MACRO_NAME HNSW_API



    # This creates hnsw_export.h in the build directory
)




# Add the build directory to the include path so hnsw_export.h can be found
target_include_directories(hnsw PUBLIC
    "${CMAKE_CURRENT_BINARY_DIR}"



    # ... other include directories ...
)

This command generates a file named hnsw_export.h containing the necessary logic. It also automatically defines a preprocessor macro (like hnsw_EXPORTS) for your library target, so the generated header knows whether it's being compiled as part of the library (exporting) or by a consumer (importing).

You then include this generated header in hnsw.h and use the HNSW_API macro on all public function declarations:

// In hnsw.h




#ifndef HNSW_H
#define HNSW_H




#include "hnsw_export.h" // Include the generated header
#include <stddef.h>
#include <stdint.h>




#ifdef __cplusplus
extern "C" {



#endif

typedef struct HnswIndex HnswIndex;

// Add the HNSW_API macro to every function in the public API
HNSW_API HnswIndex* hnsw_load(const char* path, size_t dim);
HNSW_API void hnsw_destroy(HnswIndex* index);
// ... and so on for all other public functions




#ifdef __cplusplus
} // extern "C"



#endif




#endif /* HNSW_H */

Versioning and ABI Compatibility

A shared library defines not just an API (the source code contract) but also an Application Binary Interface (ABI), which is the low-level contract covering calling conventions, data structure layout, and symbol names. If you change the ABI (e.g., by adding a parameter to a function), programs compiled against the old library version will crash.

Operating systems use version numbers embedded in the library filenames to manage this. On Linux, you'll often see a pattern like this:

• libhnsw.so.1.0.0: The real, fully-versioned library file.
• libhnsw.so.1: The soname, a symbolic link pointing to the real file. The runtime loader uses this to find a compatible library. It only changes when the ABI breaks (a major version bump).
• libhnsw.so: The linker name, another symlink used by the compiler when linking new applications.

CMake manages this with two target properties: VERSION and SOVERSION.

To add versioning to our library, we set these properties on the target. Let's say we're releasing version 1.0.0.

# In src/CMakeLists.txt
set_property(TARGET hnsw PROPERTY VERSION "1.0.0")
set_property(TARGET hnsw PROPERTY SOVERSION "1")

This tells CMake to create libhnsw.so.1.0.0 and the appropriate libhnsw.so.1 symlink, ensuring correct runtime behavior and compatibility checks.

Installation and Runtime Linking

You've configured CMake, run make, and now have a libhnsw.so.1.0.0 file in your build/src directory. But if you try to run an application linked against it from a different directory, you'll get an error: error while loading shared libraries: libhnsw.so.1: cannot open shared object file.

The operating system's dynamic linker doesn't know to look in your build directory. To solve this, we must create a proper install step that places the library and any executables in a standard, relocatable layout.

CMake's install() command is used for this. It copies build artifacts from the messy build tree to a clean install prefix. Critically, for Linux/macOS, we also need to embed an RPATH into our executables. This is a path that tells the dynamic linker where to look for shared libraries. By using the special $ORIGIN variable, we can tell it to "look in the same directory as the executable itself."

Here's how we add a complete installation configuration to our project.

In your root CMakeLists.txt:

# At the top of your root CMakeLists.txt
include(GNUInstallDirs) # Defines standard install paths like CMAKE_INSTALL_LIBDIR




# On non-Apple Unix, set the install RPATH to $ORIGIN
if(UNIX AND NOT APPLE)
  set(CMAKE_INSTALL_RPATH "$ORIGIN")
endif()

In your src/CMakeLists.txt:

# After all other target properties
install(TARGETS hnsw
    RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
    LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
    ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
    PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} # Installs public headers
)

This comprehensive install command handles all cases:

• LIBRARY: Installs the .so/.dylib files to the lib directory.
• ARCHIVE: Installs the static library (.a) or Windows import library (.lib) to the lib directory.
• RUNTIME: Installs the Windows .dll to the bin directory (where executables go).
• PUBLIC_HEADER: Installs the headers associated with the target (like hnsw.h) into the include directory.

Now, after building, you can run cmake --install . (or make install) to deploy all the components of your library to the location specified by CMAKE_INSTALL_PREFIX.

Conclusion

In this lesson, you've learned how to transform your C project from a collection of source files into a professional, distributable shared library using modern CMake practices. This is a critical step in creating software that can be used by others and integrated into larger systems.

Key Takeaways:

• Shared Library Creation: Use add_library(.. SHARED) or, preferably, make it configurable with the BUILD_SHARED_LIBS option.
• Symbol Visibility: Always hide internal symbols and explicitly export your public API. Use CMake's GenerateExportHeader module to automate this for cross-platform compatibility.
• Library Versioning: Use the VERSION and SOVERSION target properties to manage your library's ABI version, which is crucial for runtime compatibility on Unix-like systems.
• Installation: Use the install(TARGETS ...) command with standard destinations from GNUInstallDirs to create a relocatable package.
• Runtime Linking: Solve the "library not found" error on Linux by setting an INSTALL_RPATH to $ORIGIN, making your executables find libraries located alongside them.

Your C library is now complete, professionally packaged, and ready for use. In our next module, we'll leverage this work by creating Python bindings, allowing you to call your high-performance C functions directly from a simple and powerful Python script.