Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 14

Exploring the LLVM Source Tree

Recommended resources

Following our last session where we learned to find potential work items in the LLVM issue tracker, we'll now build a mental map of the project's source code. Finding a bug is the first step, but knowing where to look for the code responsible is what transforms you from an observer into a contributor.

This lesson directly addresses the learning outcome: Navigate the LLVM source tree directory structure to locate core components (include, lib, tools, test). By the end of our 60 minutes, you will be able to take a bug report from a sub-project like Clang or a core component like an optimization pass, and confidently identify the specific directories containing the relevant implementation, header files, and tests. This skill is the bridge between identifying a problem and beginning to solve it.

A Bird's-Eye View: The llvm-project Monorepo

First, it's important to understand that the llvm-project you have on your machine is a "monorepo." It contains not just the core LLVM optimizer and code generator, but also its major sub-projects like Clang, MLIR, LLDB, and others, all in one large repository. This structure facilitates development across different parts of the ecosystem.

The layout of this monorepo can seem intimidating. A great way to get oriented is with a guided tour. The blog post "A Tourist's Guide to the LLVM Source Code" by John Regehr provides an excellent, high-level overview of the main directories.

A Tourist's Guide to the LLVM Source Code - Embedded in Academia

This post offers a clear and concise tour of the LLVM source tree from the perspective of a compiler engineer. It helps to quickly build a conceptual model of where everything lives.

Read the introduction and the first major section that walks through the top-level directories. Pay close attention to the descriptions of these four key directories: include: The public header files. lib: The source code implementation. test: The regression test suite. tools: The command-line executables.

This first pass should give you a high-level map. The core logic of LLVM follows a classic software engineering pattern you've likely seen in your CS courses: separating public interfaces (APIs) from their implementations, and separating libraries from the executable tools that use them.

  • include/: This is the public face of the LLVM libraries. When you write a pass or tool, you will #include headers from here. It defines the "contract"—the classes and functions available for you to use.
  • lib/: This is where the implementation of that contract lives. It's the bulk of the C++ source code.
  • tools/: These are the executable programs that link against the libraries in lib/. For example, opt is a tool that applies optimization passes, and llc is a tool that generates machine code.
  • test/: For nearly every feature or bug fix, there is a corresponding test here. This directory is crucial for contribution, as you'll need to run these tests and add new ones for your changes.

Diving into the Core: The lib Directory

The lib directory is where most of the compiler's intelligence resides. This is where you'll spend the majority of your time when debugging an optimization or a code generation issue. The "Tourist's Guide" provides an excellent breakdown of this directory's contents.

A Tourist's Guide to the LLVM Source Code - Embedded in Academia

Now, let's zoom in on the lib directory, which the author rightly calls "basically everything important."

Read the section that details the subdirectories of lib. As you read, focus on understanding the purpose of these key areas, which are highly relevant for a contributor: Analysis: Contains code that analyzes the IR but doesn't change it, such as alias analysis. IR: The implementation of core data structures like Instruction, BasicBlock, and Function. Target: The processor-specific backends (e.g., X86, AArch64). Your interest in low-level engineering and performance systems makes this a particularly relevant area. Transforms: Home to the optimization passes that modify the IR, like InstCombine (peephole optimizations) and Scalar (classic scalar optimizations). This is a hotspot for bug fixes.

The Authoritative Reference

While the blog post is a fantastic guide, the official LLVM documentation provides the ground truth. The "Directory Layout" section of the "Getting Started" guide is the definitive reference for what each directory contains. Let's use it to solidify your understanding.

Getting Started with the LLVM System

This official documentation provides a detailed, systematic breakdown of the entire source tree. It's an indispensable resource for any LLVM developer.

Please read the entire Directory Layout section. Don't worry about memorizing every single directory. Instead, use this reading to reinforce your understanding of the major areas we've discussed (include, lib, tools, test) and to see how the project is officially organized. Pay special attention to the descriptions of the subdirectories within llvm/lib, such as llvm/lib/Transforms/ and llvm/lib/Target/.

From Issue to Code: A Practical Workflow

Now, let's connect this knowledge back to the goal of fixing a bug. Imagine you've found an interesting bug report from the last lesson. The issue has the label clang:codegen. How do you find the relevant code?

  1. Identify the Project: The clang: prefix tells you the issue is in the Clang frontend. The codegen part suggests it's related to how Clang generates code (in this case, LLVM IR).
  2. Navigate to the Sub-project: In the llvm-project monorepo, you'll find a top-level clang/ directory.
  3. Apply the Pattern: Inside clang/, you'll see a familiar structure: include/, lib/, tools/, test/.
    • The core implementation for Clang's code generation lives in clang/lib/CodeGen/.
    • The related public APIs would be in clang/include/clang/CodeGen/.
    • Tests specifically for Clang's code generation would be in clang/test/CodeGen/.

This pattern holds for other sub-projects as well:

  • An MLIR dialect bug (mlir:affine)? Look in mlir/lib/Dialect/Affine/.
  • A core LLVM optimization issue (scalar-opts)? Look in llvm/lib/Transforms/Scalar/.
  • An X86 backend bug (backend:x86)? Look in llvm/lib/Target/X86/.

The directory structure is intentionally designed to be predictable. By combining the information from an issue's labels with your knowledge of the source layout, you can quickly narrow down the search space from millions of lines of code to a handful of relevant files.

Conclusion

You now possess a mental map of the LLVM-project source tree. This is a fundamental skill that separates prospective contributors from active ones. You can now not only find a task but also locate the exact place in the code where that task needs to be addressed.

Key Takeaways:

  • The LLVM source code is organized into a monorepo containing the core project and its sub-projects like Clang and MLIR.
  • The four most critical top-level directories are include (APIs), lib (implementation), tools (executables), and test (regression tests).
  • The lib directory is where most of the core logic lives, with subdirectories like Transforms, Analysis, and Target containing the optimization passes, analysis frameworks, and backend implementations.
  • Sub-projects like clang/ and mlir/ mirror this directory structure, making the entire monorepo easier to navigate.
  • You can map an issue's labels directly to a directory path (e.g., label:instcombine -> llvm/lib/Transforms/InstCombine/).

We've found an issue and located the code. The next step in any developer's workflow is to run the existing tests to reproduce the bug. In our next lesson, we will do exactly that by learning how to run the LLVM regression test suite using the llvm-lit testing tool.