Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 9

Closures for Encapsulating State

Recommended resources

In our last lesson, we established that functions in Python are first-class objects, a principle that allows them to be passed as arguments and, crucially, returned from other functions. This capability is not just an interesting language feature; it is the direct mechanism that enables the powerful concept we are exploring today: closures.

You've mentioned that you're already familiar with closures, but your goal is to achieve a senior-level, deep understanding. This lesson is designed to do exactly that. We will move beyond simply using closures and dissect how they actually work, why they exist, and how they provide an elegant way to manage state—a common task in the complex systems you build. By the end of this lesson, you will be able to define closures precisely and use them to encapsulate state without needing to write a full class.

From Inner Functions to Closures

At its core, a closure is built upon the idea of nested functions. In our previous lesson, we saw how a function can return another function. A closure is a special kind of returned function—one that "remembers" the environment in which it was created.

Let's formalize this.

Python Closures: Common Use Cases and Examples

The RealPython article "Python Closures" provides a very clear, step-by-step introduction. It starts by defining inner functions and then builds up to the formal definition of a closure.

Start by reading the introductory section, Getting to Know Closures. Then, read the next two subsections, Function Closures, which walk through the process of turning a simple inner function into a true closure. Pay close attention to the three components required to create one.

As the article states, for a closure to exist, you need three things:

  1. An enclosing function that contains a nested, inner function.
  2. The inner function must refer to variables from the enclosing function. These are called free variables.
  3. The enclosing function must return the inner function.

When these conditions are met, the returned inner function carries with it a reference to the enclosing scope's variables. This is the essence of a closure. It's a function object bundled with its lexical environment.

The image below provides a great mental model for this process.

This diagram illustrates the lifecycle of a closure. In step 1, `parent_function(10)` is called. It creates the `child_function` and then returns it. The `parent_function`'s frame is removed from the call stack. In step 2, when `closure(5)` is called later, the `child_function` object still has access to the variable `x` from its original enclosing scope, even though `parent_function` has finished executing.

How Closures "Remember": A Look Under the Hood

The idea that a function "remembers" its environment might seem like magic. To truly master closures, we need to understand the mechanism Python uses to achieve this. It involves a subtle interplay between compile-time analysis and runtime execution.

The key is that Python's compiler determines how variables will be looked up, even before the code runs.

Functions within functions, closures, and variable scopes in Python

The mCoding video "Functions within functions, closures, and variable scopes" offers one of the clearest explanations of the mechanics behind closures.

First, watch the segment from the one rule to understand the critical distinction between compile-time scope detection and runtime lookup. Then, watch the segment from UnboundLocalError, which shows how the compiler's decision can lead to an UnboundLocalError. Finally, and most importantly, watch from the discussion on closures to see how Python uses an intermediate object, called a cell, to store free variables.

As the video explains, when the compiler sees an inner function referencing a variable from an outer function, it doesn't just store the variable's value at that moment. Instead, it creates an intermediary object called a cell. The inner function gets a reference to this cell. The cell, in turn, holds the reference to the actual object (like an integer or a string).

This level of indirection is what allows the closure to always access the latest value of the variable, and it's what keeps the variable alive even after the outer function's scope is destroyed.

We can inspect this mechanism directly through the __closure__ special attribute. Every function object has this attribute. For a normal function, it's None. But for a closure, it's a tuple of cell objects containing the free variables.

def outer_func(x):



    # x is a free variable for inner_func
    def inner_func(y):



        # y is a local variable
        return x + y
    return inner_func

closure = outer_func(10)




# The closure object is the returned inner_func
print(closure)



# <function outer_func.<locals>.inner_func at ...>




# Inspect the __closure__ attribute
print(closure.__closure__)



# (<cell at ...: int object at ...>,)




# We can even access the content of the cell
print(closure.__closure__[0].cell_contents)



# 10

This is not something you'd typically do in production code, but inspecting __closure__ is invaluable for demystifying how closures work.

For a more rigorous, code-driven exploration of this internal property, the following resource is excellent.

Understanding Python's closures - Github-Gist

This Gist by Dmitry Soshnikov provides a deep, technical dive into the __closure__ attribute with commented code examples. It's a fantastic resource for solidifying your internal model of closures.

Focus on the first three technical points in the code: Read the comments and code under Magic property __closure__. This demonstrates how nested free variables are captured. Next, review Function without free variables, which shows that a closure is only formed if free variables are actually used. Finally, look at A closure is formed if there's most inner function. This reveals a subtlety: an intermediate function will still form a closure if a deeper nested function needs its variables.

The Primary Use Case: Encapsulating State

Now that we understand the what and the how, let's focus on the why. One of the most powerful applications of closures is to create stateful functions—functions that maintain state across multiple calls, without the boilerplate of a class.

This directly addresses your goal of building production-level systems. Often, you need a small piece of stateful logic—a counter, a running average, a cache—but defining a whole class feels like overkill. A closure is often the more Pythonic and lightweight solution.

As this diagram shows, closures are central to several advanced patterns, including creating stateful functions and enabling encapsulation, which we'll explore now.

Python Closures: Common Use Cases and Examples

Let's return to the RealPython article to see practical examples of state management.

Read the section Building Stateful Functions. The cumulative_average example is a perfect illustration of a function that remembers its history. The data list is encapsulated within the closure's scope, persisting between calls to stream_average.

Closures for Data Encapsulation

Let's take this a step further. Your experience with object-oriented programming in Django involves encapsulating data within class instances. While Python doesn't have true private variables (the _single_underscore is just a convention), closures can provide a stronger form of data hiding.

The variables in the enclosing scope are not attributes on the function object, so they can't be accessed with . notation. They live only inside the closure's cell objects.

Python Closures: Common Use Cases and Examples

This final reading from the RealPython article compares a class-based Stack implementation with a closure-based one to highlight this improved encapsulation.

Read the entire section Achieving Encapsulation. Notice how the closure-based Stack prevents direct access to the _items list, which is still possible in the class-based version. The article rightly points out that a determined user can still access the data via __closure__, but this is far less obvious and less likely to be done by accident.

This pattern is a powerful tool in your arsenal. For simple stateful objects, a factory function that returns a closure can be cleaner, more concise, and offer better encapsulation than a simple class.

Conclusion

In this lesson, we took a deep dive into Python closures, moving from a surface-level understanding to a more fundamental one. We've seen that a closure is much more than just a nested function; it's a powerful construct for bundling a function with its lexical environment.

Here are the key takeaways for your senior-level toolkit:

  • Definition: A closure is a function object that retains bindings to free variables from its enclosing scope, even after that scope has ceased to exist.
  • Mechanism: This is achieved at compile time, when Python identifies the need for a closure and creates cell objects to hold references to the free variables. The function's __closure__ attribute holds a tuple of these cells.
  • State Encapsulation: Closures provide a clean, lightweight, and effective way to create stateful functions, offering a powerful alternative to classes for simpler use cases.
  • Data Hiding: They can offer stronger data encapsulation than Python's conventional underscore-prefixed attributes, as the closed-over variables are not directly accessible as attributes of the function object.

In our next lesson, we will address a crucial aspect of working with closures: how to modify the values of the captured variables. This will lead us directly into a discussion of Python's full scoping rules (the LEGB rule) and the purpose of the nonlocal keyword. This knowledge is the final piece needed to fully master the behavior of nested functions and closures.