Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 3

Navigating the LLVM Ecosystem

Recommended resources

Welcome back! In our last lesson, we examined the "why" behind LLVM's three-phase design, focusing on the powerful capabilities enabled by its library-based architecture and its complete Intermediate Representation. We saw how this design facilitates features like Link-Time Optimization and robust testing.

Now, we will shift from the conceptual "why" to the concrete "what." This lesson provides a map of the LLVM ecosystem. We'll identify the major components you'll interact with as a contributor: the core libraries, key subprojects like Clang, LLDB, and MLIR, and the essential command-line tools. Having this "lay of the land" is the first practical step toward navigating the vast LLVM codebase and finding your place within it.

The LLVM Umbrella Project

In our previous discussions, we've used the term "LLVM" to refer to the optimizer and code generator. However, in the wider community, "LLVM" also refers to a much larger collection of projects. It's helpful to think of it as an "umbrella project" that encompasses not just the compiler core, but an entire toolchain.

This diagram shows the basic flow of a compiler built with LLVM. A "Front end" (like Clang) produces LLVM IR. This IR is then processed by a series of "Analysis passes" and "Transform passes" (the optimizer), and finally, a "Back end" generates machine code. The entire LLVM ecosystem is built around this central IR and the libraries that operate on it.

The official LLVM homepage provides the best definition of this structure.

The LLVM Compiler Infrastructure Project

This reading from the official LLVM website introduces the concept of LLVM as an umbrella project and then lists its primary sub-projects. This will give you a canonical overview of the entire ecosystem.

First, read the introductory section, from the first two paragraphs, to understand the official definition of the LLVM project. Next, read the list of "primary sub-projects." Focus on the descriptions for: LLVM Core: The optimizer and code generator. Clang: The C-family language frontend. LLDB: The debugger. MLIR: A new compiler infrastructure for modern hardware. LLD: The linker. compiler-rt: The runtime library providing low-level support functions.

This reading gives you the official "who's who" of the LLVM world. Let's distill the roles of the most critical subprojects for a new contributor.

  • LLVM Core: This is what we've been calling the middle-end and back-end. It contains the optimization passes (opt) and the code generation infrastructure (llc). All other projects in the ecosystem are built on top of these core libraries.
  • Clang: This is the frontend for C, C++, and Objective-C. Crucially, Clang is designed as a library for parsing C++ code. This allows for the creation of powerful source-level tools like the Clang Static Analyzer and clang-tidy, which can find bugs and enforce coding standards without performing a full compilation. Many GSoC projects revolve around Clang and its related tools.
  • LLDB: This is LLVM's native debugger. It is a prime example of LLVM's modularity. It reuses Clang's libraries for parsing expressions, LLVM's libraries for JIT compilation and disassembly, and other core components. This tight integration makes it a very powerful and efficient debugger.
  • MLIR: Standing for Multi-Level Intermediate Representation, MLIR is one of the most active areas of development. It's a new compiler infrastructure that aims to generalize the concept of an IR to better handle domain-specific languages (like in machine learning) and diverse hardware targets (CPUs, GPUs, TPUs). Understanding MLIR is increasingly important for contributions in high-performance computing and AI compilers.
  • LLD: A high-performance linker intended as a drop-in replacement for system linkers. Its speed can dramatically reduce build times for large C++ projects, including LLVM itself.

Your interest in low-level systems programming and performance makes Clang, MLIR, and the LLVM Core itself particularly relevant areas for potential contributions.

The Developer's Toolkit

While the subprojects are the major components, your day-to-day development and debugging will involve a set of command-line tools. These tools are the primary user interface for the libraries we've just discussed. The "Getting Started" guide provides a good summary of the most important ones.

Getting Started with the LLVM System

This section from the official documentation lists the key command-line tools that are built from the LLVM libraries.

Please read the subsection on tools. Pay attention to the purpose of llvm-as, llvm-dis, llc, lli, and opt.

To see these tools in action, let's watch a short demonstration. The video will show how a simple C++ file is compiled to LLVM IR, and how that IR can then be interpreted, assembled, and disassembled.

Programming Language with LLVM [1/20] Introduction to LLVM IR and tools

This video by Dmitry Soshnikov provides a practical demonstration of the core LLVM tools.

Watch the segment from this timestamp. It walks through using clang -emit-llvm to generate IR, lli to execute it, llvm-as to create bitcode, and llvm-dis to convert it back to human-readable text.

Here's a quick summary of the key tools you'll be using constantly:

Tool Purpose
clang The frontend driver; compiles C/C++ to LLVM IR and then to object code.
opt The middle-end driver; applies optimization passes to LLVM IR files.
llc The backend driver; compiles LLVM IR to target-specific assembly code.
lli The LLVM interpreter/JIT; directly executes LLVM IR or bitcode.
llvm-dis The disassembler; converts binary LLVM bitcode (.bc) to human-readable IR (.ll).
llvm-as The assembler; converts human-readable IR (.ll) to binary bitcode (.bc).
llvm-link Links multiple bitcode files into a single one.
llvm-reduce A powerful debugging utility that automatically shrinks a test case causing a bug. We will explore this tool in detail in Module 3.

Navigating the Source Code: The Monorepo

Now that we know the names of the projects and tools, the final step is to understand where they live in the source tree. The LLVM project recently migrated to a "monorepo" model, where all the primary subprojects reside in a single Git repository. This simplifies building and testing the entire ecosystem.

The following video provides a concise overview of this directory structure.

2019 LLVM Developers’ Meeting: E. Christopher & J. Doerfert “Introduction to LLVM”

This introductory talk from the LLVM Developers' Meeting explains the layout of the monorepo and how the tools relate back to the libraries.

First, watch the segment on the source layout from the beginning. It explains that subprojects like clang and lldb are simply directories within the main llvm-project repository. Then, watch the segment from on tools as library users. It reinforces the idea that the tools in the tools directory are excellent examples of how to use the LLVM APIs, a key insight for any aspiring contributor.

To summarize the monorepo structure within the llvm-project/ directory:

  • llvm/: The root directory for the LLVM Core project. This is the heart of the whole system.
    • llvm/include/llvm/: Public C++ headers for the core libraries.
    • llvm/lib/: Source code for the core libraries (e.g., lib/Transforms, lib/CodeGen).
    • llvm/tools/: Source code for the core tools (opt, llc, etc.).
  • clang/: The root directory for the Clang project. It has its own include, lib, and tools subdirectories.
  • lldb/: The root directory for the LLDB project.
  • mlir/: The root directory for the MLIR project.
  • ...and so on for the other subprojects like lld and polly.

When you want to understand how a tool like opt works, you'll look in llvm/tools/opt/. When you want to fix a bug in a C++ frontend, you'll likely be working within the clang/ directory. This organized structure makes the massive codebase much more manageable.

Conclusion

In this lesson, we've surveyed the LLVM landscape, moving from high-level projects to the specific tools and directories you'll encounter as a developer.

Key Takeaways:

  • The LLVM Project is an umbrella for a collection of toolchain technologies, not just a single compiler.
  • Its main subprojects include LLVM Core (optimizer/backend), Clang (C++ frontend), LLDB (debugger), MLIR (next-gen compiler infrastructure), and LLD (linker).
  • A suite of command-line tools (opt, llc, lli, llvm-dis) provide the primary interface for interacting with the LLVM libraries.
  • All of these projects reside in a single monorepo, with a consistent directory structure (lib, include, tools, test) that simplifies navigation and development.

You now have a map of the territory. In our next lesson, we will take the essential next step: setting up your development environment by checking out the source code and building the entire LLVM project from scratch.