Nested Scope Examples

To clarify the prior section’s points, let’s illustrate with some real code. Here is what an enclosing function scope looks like:

X = 99                   # Global scope name: not used

def f1():
    X = 88               # Enclosing def local
    def f2():
        print(X)         # Reference made in nested def
    f2()

f1()                     # Prints 88: enclosing def local

广告：个人专属 VPN，独立 IP，无限流量，多机房切换，还可以屏蔽广告和恶意软件，每月最低仅 5 美元

First off, this is legal Python code: the def is simply an executable statement, which can appear anywhere any other statement can—including nested in another def. Here, the nested def runs while a call to the function f1 is running; it generates a function and assigns it to the name f2, a local variable within f1’s local scope. In a sense, f2 is a temporary function that lives only during the execution of (and is visible only to code in) the enclosing f1.

But notice what happens inside f2: when it prints the variable X, it refers to the X that lives in the enclosing f1 function’s local scope. Because functions can access names in all physically enclosing def statements, the X in f2 is automatically mapped to the X in f1, by the LEGB lookup rule.

This enclosing scope lookup works even if the enclosing function has already returned. For example, the following code defines a function that makes and returns another function:

def f1():
    X = 88
    def f2():
        print(X)         # Remembers X in enclosing def scope
    return f2            # Return f2 but don't call it

action = f1()            # Make, return function
action()                 # Call it now: prints 88

In this code, the call to action is really running the function we named f2 when f1 ran. f2 remembers the enclosing scope’s X in f1, even though f1 is no longer active.

Factory functions

Depending on whom you ask, this sort of behavior is also sometimes called a closure or factory function. These terms refer to a function object that remembers values in enclosing scopes regardless of whether those scopes are still present in memory. Although classes (described in Part VI of this book) are usually best at remembering state because they make it explicit with attribute assignments, such functions provide an alternative.

For instance, factory functions are sometimes used by programs that need to generate event handlers on the fly in response to conditions at runtime (e.g., user inputs that cannot be anticipated). Look at the following function, for example:

>>> def maker(N):
...     def action(X):                    # Make and return action
...         return X ** N                 # action retains N from enclosing scope
...     return action
...

This defines an outer function that simply generates and returns a nested function, without calling it. If we call the outer function:

>>> f = maker(2)                          # Pass 2 to N
>>> f
<function action at 0x014720B0>

what we get back is a reference to the generated nested function—the one created by running the nested def. If we now call what we got back from the outer function:

>>> f(3)                                  # Pass 3 to X, N remembers 2: 3 ** 2
9
>>> f(4)                                  # 4 ** 2
16

it invokes the nested function—the one called action within maker. The most unusual part of this is that the nested function remembers integer 2, the value of the variable N in maker, even though maker has returned and exited by the time we call action. In effect, N from the enclosing local scope is retained as state information attached to action, and we get back its argument squared.

If we now call the outer function again, we get back a new nested function with different state information attached. That is, we get the argument cubed instead of squared, but the original still squares as before:

>>> g = maker(3)                          # g remembers 3, f remembers 2
>>> g(3)                                  # 3 ** 3
27
>>> f(3)                                  # 3 ** 2
9

This works because each call to a factory function like this gets its own set of state information. In our case, the function we assign to name g remembers 3, and f remembers 2, because each has its own state information retained by the variable N in maker.

This is an advanced technique that you’re unlikely to see very often in most code, except among programmers with backgrounds in functional programming languages. On the other hand, enclosing scopes are often employed by lambda function-creation expressions (discussed later in this chapter)—because they are expressions, they are almost always nested within a def. Moreover, function nesting is commonly used for decorators (explored in Chapter 38)—in some cases, it’s the most reasonable coding pattern.

As a general rule, classes are better at “memory” like this because they make the state retention explicit in attributes. Short of using classes, though, globals, enclosing scope references like these, and default arguments are the main ways that Python functions can retain state information. To see how they compete, Chapter 18 provides complete coverage of defaults, but the next section gives enough of an introduction to get us started.

Retaining enclosing scopes’ state with defaults

In earlier versions of Python, the sort of code in the prior section failed because nested defs did not do anything about scopes—a reference to a variable within f2 would search only the local (f2), then global (the code outside f1), and then built-in scopes. Because it skipped the scopes of enclosing functions, an error would result. To work around this, programmers typically used default argument values to pass in and remember the objects in an enclosing scope:

def f1():
    x = 88
    def f2(x=x):                # Remember enclosing scope X with defaults
        print(x)
    f2()

f1()                            # Prints 88

This code works in all Python releases, and you’ll still see this pattern in some existing Python code. In short, the syntax arg = val in a def header means that the argument arg will default to the value val if no real value is passed to arg in a call.

In the modified f2 here, the x=x means that the argument x will default to the value of x in the enclosing scope—because the second x is evaluated before Python steps into the nested def, it still refers to the x in f1. In effect, the default remembers what x was in f1 (i.e., the object 88).

That’s fairly complex, and it depends entirely on the timing of default value evaluations. In fact, the nested scope lookup rule was added to Python to make defaults unnecessary for this role—today, Python automatically remembers any values required in the enclosing scope for use in nested defs.

Of course, the best prescription for most code is simply to avoid nesting defs within defs, as it will make your programs much simpler. The following is an equivalent of the prior example that banishes the notion of nesting. Notice the forward reference in this code—it’s OK to call a function defined after the function that calls it, as long as the second def runs before the first function is actually called. Code inside a def is never evaluated until the function is actually called:

>>> def f1():
...     x = 88                  # Pass x along instead of nesting
...     f2(x)                   # Forward reference okay
...
>>> def f2(x):
...     print(x)
...
>>> f1()
88

If you avoid nesting this way, you can almost forget about the nested scopes concept in Python, unless you need to code in the factory function style discussed earlier—at least, for def statements. lambdas, which almost naturally appear nested in defs, often rely on nested scopes, as the next section explains.

Nested scopes and lambdas

While they’re rarely used in practice for defs themselves, you are more likely to care about nested function scopes when you start coding lambda expressions. We won’t cover lambda in depth until Chapter 19, but in short, it’s an expression that generates a new function to be called later, much like a def statement. Because it’s an expression, though, it can be used in places that def cannot, such as within list and dictionary literals.

Like a def, a lambda expression introduces a new local scope for the function it creates. Thanks to the enclosing scopes lookup layer, lambdas can see all the variables that live in the functions in which they are coded. Thus, the following code works, but only because the nested scope rules are applied:

def func():
    x = 4
    action = (lambda n: x ** n)          # x remembered from enclosing def
    return action

x = func()
print(x(2))                              # Prints 16, 4 ** 2

Prior to the introduction of nested function scopes, programmers used defaults to pass values from an enclosing scope into lambdas, just as for defs. For instance, the following works on all Python releases:

def func():
    x = 4
    action = (lambda n, x=x: x ** n)     # Pass x in manually
    return action

Because lambdas are expressions, they naturally (and even normally) nest inside enclosing defs. Hence, they are perhaps the biggest beneficiaries of the addition of enclosing function scopes in the lookup rules; in most cases, it is no longer necessary to pass values into lambdas with defaults.

Scopes versus defaults with loop variables

There is one notable exception to the rule I just gave: if a lambda or def defined within a function is nested inside a loop, and the nested function references an enclosing scope variable that is changed by that loop, all functions generated within the loop will have the same value—the value the referenced variable had in the last loop iteration.

For instance, the following attempts to build up a list of functions that each remember the current variable i from the enclosing scope:

>>> def makeActions():
...     acts = []
...     for i in range(5):                       # Tries to remember each i
...         acts.append(lambda x: i ** x)        # All remember same last i!
...     return acts
...
>>> acts = makeActions()
>>> acts[0]
<function <lambda> at 0x012B16B0>

This doesn’t quite work, though—because the enclosing scope variable is looked up when the nested functions are later called, they all effectively remember the same value (the value the loop variable had on the last loop iteration). That is, we get back 4 to the power of 2 for each function in the list, because i is the same in all of them:

>>> acts[0](2)                                   # All are 4 ** 2, value of last i
16
>>> acts[2](2)                                   # This should be 2 ** 2
16
>>> acts[4](2)                                   # This should be 4 ** 2
16

This is the one case where we still have to explicitly retain enclosing scope values with default arguments, rather than enclosing scope references. That is, to make this sort of code work, we must pass in the current value of the enclosing scope’s variable with a default. Because defaults are evaluated when the nested function is created (not when it’s later called), each remembers its own value for i:

>>> def makeActions():
...     acts = []
...     for i in range(5):                       # Use defaults instead
...         acts.append(lambda x, i=i: i ** x)   # Remember current i
...     return acts
...
>>> acts = makeActions()
>>> acts[0](2)                                   # 0 ** 2
0
>>> acts[2](2)                                   # 2 ** 2
4
>>> acts[4](2)                                   # 4 ** 2
16

This is a fairly obscure case, but it can come up in practice, especially in code that generates callback handler functions for a number of widgets in a GUI (e.g., button-press handlers). We’ll talk more about defaults in Chapter 18 and lambdas in Chapter 19, so you may want to return and review this section later.[39]

Arbitrary scope nesting

Before ending this discussion, I should note that scopes may nest arbitrarily, but only enclosing function def statements (not classes, described in Part VI) are searched:

>>> def f1():
...     x = 99
...     def f2():
...         def f3():
...             print(x)        # Found in f1's local scope!
...         f3()
...     f2()
...
>>> f1()
99

Python will search the local scopes of all enclosing defs, from inner to outer, after the referencing function’s local scope and before the module’s global scope or built-ins. However, this sort of code is even less likely to pop up in practice. In Python, we say flat is better than nested—except in very limited contexts, your life (and the lives of your coworkers) will generally be better if you minimize nested function definitions.

 

[39] In the section Function Gotchas at the end of this part of the book, we’ll also see that there is an issue with using mutable objects like lists and dictionaries for default arguments (e.g., def f(a=[]))—because defaults are implemented as single objects attached to functions, mutable defaults retain state from call to call, rather then being initialized anew on each call. Depending on whom you ask, this is either considered a feature that supports state retention, or a strange wart on the language. More on this at the end of Chapter 20.