Closures and the `nonlocal` Keyword

1. What is a Closure? The "Captured State"

A closure is a function object that has access to variables in its enclosing lexical scope, even when the function is called outside that scope. To form a closure in Python, you need three things:

1. An enclosing function that contains another function.
2. An inner function defined within the enclosing function.
3. The enclosing function must return the inner function.

The magic is that the returned inner function carries with it a "memory" of the variables from its enclosing scope.

To get a more formal definition and see the components of a closure laid out, please read the following sections from a Real Python article.

To make this concept of "scopes" more concrete, let's watch a video that uses a visual debugger to step through a closure's execution.

2. How Closures Work: Cells and __closure__

So, how does the inner function actually retain this state? It's not magic; it's a specific implementation detail in Python.

The key rule to remember is: Where Python will look for a variable is determined at compile time, but the value of that variable is looked up at runtime.

When Python compiles a function with a nested function that references variables from the outer scope, it packages those non-local variables into special objects called cells. The inner function then holds a reference to these cells in its __closure__ attribute. A cell is essentially an extra layer of indirection that allows the value to be updated and shared.

Now, let's go even deeper and inspect these cell objects ourselves. You can programmatically access the closure and even modify its contents, though this is generally for inspection, not something you'd do in production code.

This deep dive shows that closures are a well-defined mechanism, not just a vague concept. They provide a robust way to bundle a function with its required state.

3. Modifying Captured State: The nonlocal Keyword

What happens if you want the inner function to change the state it captured?

Consider this attempt to create a simple counter:

def make_counter():
    count = 0
    def counter():



        # We want to increment the 'count' from the outer scope
        count += 1 # This will FAIL
        return count
    return counter

c = make_counter()



# c() # This would raise an UnboundLocalError

This code fails with an UnboundLocalError. Why? Because the assignment count += 1 (which is count = count + 1) makes Python assume that count is a local variable within the counter function. Since count was never initialized locally before the assignment, an error occurs. Python does not automatically assume you mean to modify the count from the enclosing scope.

To fix this, you must explicitly tell Python your intention with the nonlocal keyword.

Here is the corrected counter example:

def make_counter():
    count = 0  # Enclosing scope variable
    def counter():
        nonlocal count  # Tell Python we're modifying the 'count' from the enclosing scope
        count += 1
        return count
    return counter

c1 = make_counter()
print(c1())  # Output: 1
print(c1())  # Output: 2




# Each closure has its own independent state
c2 = make_counter()
print(c2())  # Output: 1

The nonlocal statement resolves the ambiguity. It tells the compiler that assignments to count inside the counter function should refer to the count variable in the nearest enclosing scope, not create a new local one.

4. A Classic Pitfall: Closures and Late Binding

A common trap when creating closures inside a loop is "late binding". The closed-over variables are looked up when the inner function is called, not when it's defined.

Consider this code:

funcs = []
for i in range(3):
    def print_i():
        print(i)
    funcs.append(print_i)




# What will this print?
for f in funcs:
    f()

You might expect it to print 0, 1, 2. Instead, it prints:

2
2
2

This happens because all three print_i functions in the list share the same closure over the variable i. When they are finally called, the loop has already completed, and i has its final value of 2.

The standard Pythonic solution is to use a default argument to capture the value at definition time.

Here's the fix:

funcs_fixed = []
for i in range(3):



    # The default argument i=i captures the *current* value of i for each lambda
    def make_printer(val=i):
        def printer():
            print(val)
        return printer
    funcs_fixed.append(make_printer())

for f in funcs_fixed:
    f()

This now correctly prints 0, 1, 2. The default argument is evaluated when the function is defined, effectively freezing the value of i for each iteration.

Exercise: Create a Running Average Closure

Your task is to build a function factory make_running_average that creates a stateful averaging function. The returned function should accept a number and return the average of all numbers it has seen so far.

Requirements:

1. The make_running_average function should initialize the necessary state.
2. The returned closure must update this state with each call.
3. You will need to use nonlocal to modify the captured state variables (e.g., total sum and count).

Starter Code:

# average.py

def make_running_average():
    """
    A function factory that creates a stateful running average function.
    """



    # TODO: Initialize state variables for the closure
    count = 0
    total = 0

    def running_average(new_value):
        """
        Accepts a new value, updates the state, and returns the current average.
        """



        # TODO: Use nonlocal to modify the state
        # TODO: Calculate and return the new average
        pass
    
    return running_average





# test_average.py
import pytest
from average import make_running_average

def test_running_average():



    # Create one averager instance
    avg1 = make_running_average()
    assert avg1(10) == 10.0
    assert avg1(20) == 15.0  # (10 + 20) / 2
    assert avg1(30) == 20.0  # (10 + 20 + 30) / 3




    # Create a second, independent averager to prove state is isolated
    avg2 = make_running_average()
    assert avg2(5) == 5.0
    assert avg2(15) == 10.0 # (5 + 15) / 2
    



    # Check that the first averager's state was not affected
    assert avg1(40) == 25.0 # (10 + 20 + 30 + 40) / 4

# average.py (FIXED)

def make_running_average():
    """
    A function factory that creates a stateful running average function.
    """
    count = 0
    total = 0

    def running_average(new_value):
        """
        Accepts a new value, updates the state, and returns the current average.
        """
        nonlocal count, total
        count += 1
        total += new_value
        return total / count
    
    return running_average

Conclusion

You have now moved beyond simply using function factories to understanding the deep mechanics of how they operate. Closures are a fundamental concept that bridge the gap between simple functions and full-fledged classes.

Key Takeaways:

• A closure is an inner function returned by an outer function, which "remembers" variables from its enclosing scope.
• This "memory" is implemented using cell objects, which provide a layer of indirection that Python stores in the inner function's __closure__ attribute.
• To modify an immutable captured variable (like a number or string), you must use the nonlocal keyword to tell Python you intend to change the variable in the enclosing scope.
• Be wary of late binding when creating closures in a loop; use default arguments to capture the value at definition time.

In our next lesson, we will explore the functools module, which provides powerful tools for working with functions. You'll learn about functools.wraps (essential for building robust decorators, which are often closures), functools.partial for creating specialized functions, functools.lru_cache for easy memoization, and functools.singledispatch for function-based polymorphism.