Class Blunders I: Decorating Class Methods

When I wrote the first tracer function decorator above, I naively assumed that it could also be applied to any method—decorated methods should work the same, but the automatic self instance argument would simply be included at the front of *args. Unfortunately, I was wrong: when applied to a class’s method, the first version of the tracer fails, because self is the instance of the decorator class and the instance of the decorated subject class in not included in *args. This is true in both Python 3.0 and 2.6.

I introduced this phenomenon earlier in this chapter, but now we can see it in the context of realistic working code. Given the class-based tracing decorator:

广告：个人专属 VPN，独立 IP，无限流量，多机房切换，还可以屏蔽广告和恶意软件，每月最低仅 5 美元

class tracer:
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original function
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)

decoration of simple functions works as advertised earlier:

@tracer
def spam(a, b, c):                           # spam = tracer(spam)
    print(a + b + c)                         # Triggers tracer.__init__

spam(1, 2, 3)                                # Runs tracer.__call__
spam(a=4, b=5, c=6)                          # spam is an instance attribute

However, decoration of class methods fails (more lucid readers might recognize this as our Person class resurrected from the object-oriented tutorial in Chapter 27):

class Person:
    def __init__(self, name, pay):
        self.name = name
        self.pay  = pay

    @tracer
    def giveRaise(self, percent):            # giveRaise = tracer(giverRaise)
        self.pay *= (1.0 + percent)

    @tracer
    def lastName(self):                      # lastName = tracer(lastName)
        return self.name.split()[-1]

bob = Person('Bob Smith', 50000)             # tracer remembers method funcs
bob.giveRaise(.25)                           # Runs tracer.__call__(???, .25)
print(bob.lastName())                        # Runs tracer.__call__(???)

The root of the problem here is in the self argument of the tracer class’s __call__ method—is it a tracer instance or a Person instance? We really need both as it’s coded: the tracer for decorator state, and the Person for routing on to the original method. Really, self must be the tracer object, to provide access to tracer’s state information; this is true whether decorating a simple function or a method.

Unfortunately, when our decorated method name is rebound to a class instance object with a __call__, Python passes only the tracer instance to self; it doesn’t pass along the Person subject in the arguments list at all. Moreover, because the tracer knows nothing about the Person instance we are trying to process with method calls, there’s no way to create a bound method with an instance, and thus no way to correctly dispatch the call.

In fact, the prior listing winds up passing too few arguments to the decorated method, and results in an error. Add a line to the decorator’s __call__ to print all its arguments to verify this; as you can see, self is the tracer, and the Person instance is entirely absent:

<__main__.tracer object at 0x02D6AD90> (0.25,) {}
call 1 to giveRaise
Traceback (most recent call last):
  File "C:/misc/tracer.py", line 56, in <module>
    bob.giveRaise(.25)
  File "C:/misc/tracer.py", line 9, in __call__
    return self.func(*args, **kwargs)
TypeError: giveRaise() takes exactly 2 positional arguments (1 given)

As mentioned earlier, this happens because Python passes the implied subject instance to self when a method name is bound to a simple function only; when it is an instance of a callable class, that class’s instance is passed instead. Technically, Python only makes a bound method object containing the subject instance when the method is a simple function.

Using nested functions to decorate methods

If you want your function decorators to work on both simple functions and class methods, the most straightforward solution lies in using one of the other state retention solutions described earlier—code your function decorator as nested defs, so that you don’t depend on a single self instance argument to be both the wrapper class instance and the subject class instance.

The following alternative applies this fix using Python 3.0 nonlocals. Because decorated methods are rebound to simple functions instead of instance objects, Python correctly passes the Person object as the first argument, and the decorator propagates it on in the first item of *args to the self argument of the real, decorated methods:

# A decorator for both functions and methods

def tracer(func):                        # Use function, not class with __call__
    calls = 0                            # Else "self" is decorator instance only!
    def onCall(*args, **kwargs):
        nonlocal calls
        calls += 1
        print('call %s to %s' % (calls, func.__name__))
        return func(*args, **kwargs)
    return onCall


# Applies to simple functions

@tracer
def spam(a, b, c):                       # spam = tracer(spam)
    print(a + b + c)                     # onCall remembers spam

spam(1, 2, 3)                            # Runs onCall(1, 2, 3)
spam(a=4, b=5, c=6)


# Applies to class method functions too!

class Person:
    def __init__(self, name, pay):
        self.name = name
        self.pay  = pay

    @tracer
    def giveRaise(self, percent):        # giveRaise = tracer(giverRaise)
        self.pay *= (1.0 + percent)      # onCall remembers giveRaise

    @tracer
    def lastName(self):                  # lastName = tracer(lastName)
        return self.name.split()[-1]

print('methods...')
bob = Person('Bob Smith', 50000)
sue = Person('Sue Jones', 100000)
print(bob.name, sue.name)
sue.giveRaise(.10)                       # Runs onCall(sue, .10)
print(sue.pay)
print(bob.lastName(), sue.lastName())    # Runs onCall(bob), lastName in scopes

This version works the same on both functions and methods:

call 1 to spam
6
call 2 to spam
15
methods...
Bob Smith Sue Jones
call 1 to giveRaise
110000.0
call 1 to lastName
call 2 to lastName
Smith Jones

Using descriptors to decorate methods

Although the nested function solution illustrated in the prior section is the most straightforward way to support decorators that apply to both functions and class methods, other schemes are possible. The descriptor feature we explored in the prior chapter, for example, can help here as well.

Recall from our discussion in that chapter that a descriptor may be a class attribute assigned to objects with a __get__ method run automatically when that attribute is referenced and fetched (object derivation is required in Python 2.6, but not 3.0):

class Descriptor(object):
    def __get__(self, instance, owner): ...

class Subject:
    attr = Descriptor()

X = Subject()
X.attr         # Roughly runs Descriptor.__get__(Subject.attr, X, Subject)

Descriptors may also have __set__ and __del__ access methods, but we don’t need them here. Now, because the descriptor’s __get__ method receives both the descriptor class and subject class instances when invoked, it’s well suited to decorating methods when we need both the decorator’s state and the original class instance for dispatching calls. Consider the following alternative tracing decorator, which is also a descriptor:

class tracer(object):
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original func
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)
    def __get__(self, instance, owner):      # On method attribute fetch
        return wrapper(self, instance)

class wrapper:
    def __init__(self, desc, subj):          # Save both instances
        self.desc = desc                     # Route calls back to  decr
        self.subj = subj
    def __call__(self, *args, **kwargs):
        return self.desc(self.subj, *args, **kwargs)  # Runs tracer.__call__

@tracer
def spam(a, b, c):                           # spam = tracer(spam)
    ...same as prior...                      # Uses __call__ only

class Person:
    @tracer
    def giveRaise(self, percent):            # giveRaise = tracer(giverRaise)
        ...same as prior...                  # Makes giveRaise a descriptor

This works the same as the preceding nested function coding. Decorated functions invoke only its __call__, while decorated methods invoke its __get__ first to resolve the method name fetch (on instance.method); the object returned by __get__ retains the subject class instance and is then invoked to complete the call expression, thereby triggering __call__ (on (args...)). For example, the test code’s call to:

sue.giveRaise(.10)                           # Runs __get__ then __call__

run’s tracer.__get__ first, because the giveRaise attribute in the Person class has been rebound to a descriptor by the function decorator. The call expression then triggers the __call__ method of the returned wrapper object, which in turn invokes tracer.__call__.

The wrapper object retains both descriptor and subject instances, so it can route control back to the original decorator/descriptor class instance. In effect, the wrapper object saves the subject class instance available during method attribute fetch and adds it to the later call’s arguments list, which is passed to __call__. Routing the call back to the descriptor class instance this way is required in this application so that all calls to a wrapped method use the same calls counter state information in the descriptor instance object.

Alternatively, we could use a nested function and enclosing scope references to achieve the same effect—the following version works the same as the preceding one, by swapping a class and object attributes for a nested function and scope references, but it requires noticeably less code:

class tracer(object):
    def __init__(self, func):                # On @ decorator
        self.calls = 0                       # Save func for later call
        self.func  = func
    def __call__(self, *args, **kwargs):     # On call to original func
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        return self.func(*args, **kwargs)
    def __get__(self, instance, owner):                # On method fetch
        def wrapper(*args, **kwargs):                  # Retain both inst
            return self(instance, *args, **kwargs)     # Runs __call__
        return wrapper

Add print statements to these alternatives’ methods to trace the two-step get/call process on your own, and run them with the same test code as in the nested function alternative shown earlier. In either coding, this descriptor-based scheme is also substantially subtler than the nested function option, and so is probably a second choice here; it may be a useful coding pattern in other contexts, though.

In the rest of this chapter we’re going to be fairly casual about using classes or functions to code our function decorators, as long as they are applied only to functions. Some decorators may not require the instance of the original class, and will still work on both functions and methods if coded as a class—something like Python’s own staticmethod decorator, for example, wouldn’t require an instance of the subject class (indeed, its whole point is to remove the instance from the call).

The moral of this story, though, is that if you want your decorators to work on both simple functions and class methods, you’re better off using the nested-function-based coding pattern outlined here instead of a class with call interception.