Calculating Code Complexity from an AST

What is Cyclomatic Complexity?

At its core, cyclomatic complexity is a quantitative measure of the complexity of a program. It was developed by Thomas J. McCabe, Sr. in 1976 and measures the number of linearly independent paths through a program's source code. In simpler terms, it counts the number of decisions in your code. A higher number means more branching (if, while, for, etc.), which can lead to more potential bugs and a greater testing burden.

To get a clear picture of what "paths" mean, let's look at how code maps to a Control Flow Graph (CFG).

As you can see, every if statement and for loop creates a new branch in the execution path. The simplest way to calculate cyclomatic complexity is to start with a baseline of 1 (for the single path through a function with no branches) and add 1 for every branching construct.

To solidify this definition, the following reading provides a concise overview.

From Concept to Code: Finding Complexity in the AST

Our task is to translate this idea of "counting branches" into an algorithm that operates on an AST. Since you're now familiar with AST nodes from our previous lesson, we can map complexity-inducing statements directly to specific node types in the ast module:

• Conditional Statements: An if statement creates a branch. This corresponds to the ast.If node. We add 1 for each ast.If we find. Note that an if/elif/else chain is represented by nested orelse blocks within a single ast.If node at each level, so we just need to count each If node we encounter.
• Loops: for and while loops also create branches (the choice to enter the loop or not, and to continue or exit). These correspond to ast.For and ast.While nodes. We add 1 for each.
• Exception Handling: A try...except block introduces a new path if an exception occurs. The except clause is represented by an ast.ExceptHandler node. We add 1 for each handler.
• Boolean Operators: This is a crucial detail. A condition like if user.is_active and user.has_permission: contains two logical decision points. The expression is parsed into an ast.BoolOp node with an op of ast.And. This node's values attribute holds a list of the conditions being joined. If there are n conditions, there are n-1 operators, adding n-1 to the complexity. For example, a and b and c has three conditions and two operators (and), so it adds 2 to the complexity.

The image below shows an AST visualizer that has calculated complexity. The tool does exactly what we're describing: it traverses the tree on the right and counts the relevant nodes to arrive at the final number.

Implementation: Building a Complexity Analyzer

Now, let's build on the ast.NodeVisitor class we created in the last lesson. We'll write a new visitor that is specifically designed to calculate cyclomatic complexity. It will traverse the AST of a function and increment a counter whenever it encounters one of the branching nodes we just identified.

The following resource provides an excellent, well-structured example of a ComplexityAnalyzer.

Let's adapt that example into a complete, runnable script. This script will define the ComplexityAnalyzer class, parse a Python file, and report the cyclomatic complexity for each function it finds.

Create a new Python file named complexity_analyzer.py and add the following code:

import ast
import sys

class ComplexityAnalyzer(ast.NodeVisitor):
    """
    A NodeVisitor class to calculate cyclomatic complexity of functions.
    """
    def __init__(self):
        self.functions = {}
        self._current_function = None
        self._complexity = 0

    def visit_FunctionDef(self, node):
        """
        Called when visiting a function definition.
        Sets the context for a new function and initializes its complexity count.
        """
        self._current_function = node.name



        # Start with a base complexity of 1 for each function
        self._complexity = 1
        



        # The 'generic_visit' method ensures that we visit all child nodes
        # of this FunctionDef, such as the statements in its body.
        self.generic_visit(node)
        



        # After visiting all children, store the final complexity score.
        self.functions[self._current_function] = self._complexity
        self._current_function = None




    # --- Increment complexity for each branching construct ---

    def visit_If(self, node):
        self._complexity += 1
        self.generic_visit(node)

    def visit_For(self, node):
        self._complexity += 1
        self.generic_visit(node)

    def visit_While(self, node):
        self._complexity += 1
        self.generic_visit(node)

    def visit_ExceptHandler(self, node):



        # This node represents an 'except' block
        self._complexity += 1
        self.generic_visit(node)
        
    def visit_With(self, node):



        # 'with' statements can add complexity if they have multiple context expressions
        self._complexity += len(node.items) -1
        self.generic_visit(node)

    def visit_BoolOp(self, node):



        # For 'and' and 'or', add complexity for each additional condition
        # e.g., 'a and b' adds 1, 'a and b and c' adds 2
        self._complexity += len(node.values) - 1
        self.generic_visit(node)

    def report(self):
        """Prints the complexity of each function found."""
        print("--- Cyclomatic Complexity Report ---")
        if not self.functions:
            print("No functions found.")
            return
            
        for func_name, complexity in sorted(self.functions.items()):
            print(f"Function '{func_name}': Cyclomatic Complexity = {complexity}")

def analyze_file_complexity(file_path):
    """
    Parses a Python file and analyzes the cyclomatic complexity of its functions.
    """
    try:
        with open(file_path, 'r') as f:
            source_code = f.read()
        tree = ast.parse(source_code)
        
        analyzer = ComplexityAnalyzer()
        analyzer.visit(tree)
        analyzer.report()
        
    except FileNotFoundError:
        print(f"Error: File not found at '{file_path}'")
    except SyntaxError as e:
        print(f"Error: Could not parse file due to a syntax error. {e}")




# --- Example Usage ---
if __name__ == "__main__":



    # To test this script, create a file named 'sample_code.py'
    # with some Python functions, or pass a path to another python file.
    
    sample_code_content = """
def simple_function():
    print("Hello")

def function_with_if(x):
    if x > 10:
        return "Large"
    else:
        return "Small"

def complex_function(items, user_is_admin):
    count = 0
    for item in items:
        try:
            if item.is_valid() and (item.value > 100 or user_is_admin):
                count += 1
        except AttributeError:
            print("Warning: item has no 'is_valid' or 'value' attribute")
    return count
"""



    # Create a dummy file for analysis
    with open("sample_code.py", "w") as f:
        f.write(sample_code_content)
    
    print("Analyzing 'sample_code.py':")
    analyze_file_complexity("sample_code.py")

To run this:

1. Save the code above as complexity_analyzer.py.
2. Run it from your terminal: python complexity_analyzer.py.

The script will create a sample_code.py file and then analyze it. The expected output is:

Analyzing 'sample_code.py':
--- Cyclomatic Complexity Report ---
Function 'complex_function': Cyclomatic Complexity = 6
Function 'function_with_if': Cyclomatic Complexity = 2
Function 'simple_function': Cyclomatic Complexity = 1

Let's break down the complex_function score of 6:

• Base complexity: 1
• for item in items: +1 (now 2)
• try...: +0 (the try itself doesn't branch, the except does)
• if item.is_valid() and (...): +1 for the if (now 3)
• ... and ...: +1 for the and operator (now 4)
• ... or user_is_admin: +1 for the or operator (now 5)
• except AttributeError:: +1 for the except handler (now 6)

You have now successfully created a tool that statically analyzes Python code and extracts a meaningful metric for quality and maintainability.

Conclusion

In this lesson, we bridged the gap from parsing code to extracting a valuable feature for ML models. You now have a tangible tool for quantifying one aspect of technical debt.

Here are the key takeaways:

• Cyclomatic Complexity is a metric that counts the number of independent execution paths in a function, serving as an indicator of its complexity.
• It can be calculated by traversing a function's AST and counting branching nodes like If, For, While, ExceptHandler, and BoolOp.
• The ast.NodeVisitor pattern is highly effective for implementing this kind of analysis, allowing you to isolate logic for each node type in separate visit_* methods.

Preview of the Next Lesson:
So far, we have mined data from two key software artifacts: Git history for change metrics and source code for complexity metrics. For a project like flaky test detection, another critical data source is the output of your CI/CD pipeline.

In our next lesson, we will shift focus to "Parse CI/CD test execution logs to extract pass/fail histories and timing data." You'll learn how to read and process log files to build a historical record of test outcomes, which is the primary dataset needed to identify which tests are flaky.