Debugging with DWARF: An `objdump` and `dwarfdump` Tutorial

The Foundation: Generating Debug Information

As you know, the -g flag is the key to creating a debuggable binary. But what does it actually do? It instructs the compiler to generate a rich set of metadata that describes your program—functions, variables, types, and the layout of your source files—and embed this information into the output object file.

This extra data doesn't alter the executable code itself, so it generally doesn't affect runtime performance. However, it does increase the file size on disk. The most common format for this information on Linux and many embedded systems is DWARF (Debugging With Attributed Record Formats).

Let's watch a short segment that introduces this concept.

As the video highlights, the debug information lives in the executable file but stays on disk when you run the program. This is a crucial point for interviews: enabling debug info increases binary size but does not consume RAM or slow down the execution of the program itself.

Finding the DWARF: ELF Debug Sections

On most systems you'll work with, from Linux servers to embedded ARM SoCs, executables use the ELF (Executable and Linkable Format). This format organizes the file into various "sections." Code goes in the .text section, initialized global variables in .data, and so on.

When you compile with -g, the compiler adds several new sections, all typically prefixed with .debug_. These sections contain the DWARF data.

We can see these sections for ourselves using the readelf utility, a standard tool for inspecting ELF files.

Let's create a simple C file, debug_test.c:

// debug_test.c



#include <stdio.h>

static int add(int x, int y) {
    return x + y;
}

int main() {
    int result = add(5, 10);
    printf("Result: %d\n", result);
    return 0;
}

First, compile it without debug symbols and list the sections:

$ gcc -o debug_test_nostrip debug_test.c
$ readelf -S debug_test_nostrip | grep .debug



# (No output is expected)

Now, compile it with debug symbols (-g) and check again:

$ gcc -g -o debug_test_g debug_test.c
$ readelf -S debug_test_g | grep .debug
  [27] .debug_aranges    PROGBITS        ...
  [28] .debug_info       PROGBITS        ...
  [29] .debug_abbrev     PROGBITS        ...
  [30] .debug_line       PROGBITS        ...
  [31] .debug_str        PROGBITS        ...
  [32] .debug_line_str   PROGBITS        ...

As you can see, a whole family of .debug_* sections has appeared. This is where the DWARF data lives.

Peeking Inside: Interpreting DWARF Data

Now that we know where to find the DWARF information, how do we read it? The raw data is binary, but tools like dwarfdump, objdump -g, and readelf --debug-dump can parse it into a human-readable format.

DWARF represents program information as a tree of Debugging Information Entries (DIEs).

• Each DIE has a Tag (DW_TAG_*) that defines what it is (e.g., a function, a variable, a type).
• Each DIE has Attributes (DW_AT_*) that describe it (e.g., name, source file, line number, memory address).

Let's examine the structure with a practical resource.

Practical Inspection with dwarfdump and addr2line

Let's apply this knowledge to our debug_test_g binary.

1. Inspecting a Function's DIE

We can use dwarfdump to find the DIE for our main function.

# The exact command might be `dwarfdump` or `llvm-dwarfdump`
$ dwarfdump --name=main debug_test_g

The output will be verbose, but you will find an entry like this (abbreviated for clarity):

0x0000008e: DW_TAG_subprogram
              DW_AT_external    (true)
              DW_AT_name        ("main")
              DW_AT_decl_file   (".../debug_test.c")
              DW_AT_decl_line   (8)
              DW_AT_low_pc      (0x1149)
              DW_AT_high_pc     (0x1173)
              ...

This tells us everything a debugger needs:

• It's a function (DW_TAG_subprogram).
• Its name is "main" (DW_AT_name).
• It's declared in debug_test.c on line 8 (DW_AT_decl_*).
• Its machine code resides from address 0x1149 up to (but not including) 0x1173 (DW_AT_low_pc, DW_AT_high_pc).

2. Mapping Address to Line with addr2line

The .debug_line section contains a highly optimized mapping of program addresses to source lines. The addr2line utility provides a simple interface to this map.

First, let's find an address inside main. We can use objdump:

$ objdump -d debug_test_g | grep -A 5 '<main>:'
0000000000001149 <main>:
    1149:	f3 0f 1e fa          	endbr64 
    114d:	55                   	push   %rbp
    114e:	48 89 e5             	mov    %rsp,%rbp
    1151:	48 83 ec 10          	sub    $0x10,%rsp
    1155:	ba 0a 00 00 00       	mov    $0xa,%edx   <-- Let's use address 0x1155

Now, use addr2line to translate 0x1155 back to its source location:

$ addr2line -e debug_test_g 0x1155
.../debug_test.c:9

It correctly identifies line 9: int result = add(5, 10);. This is the core mechanism that enables step-by-step debugging at the source level.

For a deeper dive into how addr2line works, including some of the complexities involved, let's return to Greg Law's talk.

The Cost of Debugging: Size and Modern Solutions

As we've established, debug info is "free" at runtime but not on disk. For large projects, the debug information can be many times larger than the executable code itself. This can be a problem for storage on embedded devices or for distribution.

This leads to a classic dilemma: ship small binaries without debug info, making field crash analysis nearly impossible, or ship huge binaries? Fortunately, modern toolchains provide better options.

1. Strip and Store: The traditional method is to strip the debug information from the binary you ship, but keep a copy of the unstripped binary on a server. When a crash dump comes in, you use the unstripped version to analyze it. This requires careful version management.

2. Split DWARF (-gsplit-dwarf): A more advanced GCC/Clang feature. The bulk of the DWARF info is put into separate .dwo files. The main executable contains only a link to these files. This keeps the executable small and speeds up link times, as the linker doesn't have to process gigabytes of DWARF data. The debugger can find and load the .dwo files when needed.

3. Debug Info Servers (debuginfo-d): This is the enterprise-grade solution used by companies like Google, Meta, and Red Hat. All debug information (and source code) is placed on a central HTTP server. Binaries are identified by a unique "build ID". When GDB loads a binary, it reads the build ID, queries the server, and downloads the correct debug info on-demand. This completely decouples the executable from its debug data.

Let's see a quick demo of these last two concepts.

Conclusion

You have now completed the journey from C source code to a final, debuggable executable. You understand not only the compilation stages but also the critical metadata that bridges the gap between the human-readable source and the machine's binary world.

Key Takeaways:

• Source-level debugging is enabled by metadata, typically in the DWARF format, which is generated by the compiler when using the -g flag.
• This metadata is stored in dedicated .debug_* sections within an ELF executable file and is not loaded into memory at runtime.
• Utilities like readelf allow you to see the ELF sections, while dwarfdump and objdump let you inspect the human-readable DWARF tree of tags and attributes.
• addr2line is a powerful utility that uses the DWARF line number table to map a machine code address directly back to a source file and line.
• The significant size of debug information presents a practical challenge, addressed by modern solutions like split DWARF and debug info servers.

Preview of the Next Module:
This lesson concludes Module 3, "The Build Process and Program Execution." We are now moving from the software toolchain to the hardware it runs on.

Our next module, "CPU and System Architecture," is a critical differentiator for interviews at semiconductor companies. We will begin with the lesson: "Explain the L1/L2/L3 cache hierarchy, including cache lines, sets, ways, and associativity." This will be our first step into understanding the hardware that makes our software perform.