Python: decorating decorators. Again

  • Tutorial
Last year on Habré there was already a very detailed article in two parts about decorators. The purpose of this new article is to cut to the chase and immediately engage in interesting, meaningful examples, in order to then have time to understand examples even more sophisticated than in previous articles.
The target audience is programmers who are already familiar (for example in C #) with higher-order functions and closures, but are used to the fact that annotations in functions are “meta-information”, which manifests itself only in reflection. The peculiarity of Python that immediately catches the eye of such programmers is that the presence of a decorator before declaring a function allows you to change the behavior of this function:



How it works? Nothing tricky: a decorator is just a function that takes an argument to decorate a function and returns a “fixed” one:

def timed(fn):
    def decorated(*x):
        start = time()
        result = fn(*x)
        print "Executing %s took %d ms" % (fn.__name__, (time()-start)*1000)
        return result
    return decorated
@timed
def cpuload():
    load = psutil.cpu_percent()
    print "cpuload() returns %d" % load
    return load
print "cpuload.__name__==" + cpuload.__name__
print "CPU load is %d%%" % cpuload()
( Full source )
cpuload .__ name __ == decorated
cpuload () returns 16
Executing cpuload took 105 ms
CPU load is 16%
The declaration is @timed def cpuload(): ...expanded to def cpuload(): ...; cpuload=timed(cpuload), so that as a result, the global name is cpuloadassociated with a function decoratedinside timed, closed to the original function cpuloadthrough a variable fn. As a result, we see cpuload.__name__==decorated

Any expression whose value is a function that takes a function and returns a function can be used as a decorator. Thus, it is possible to create “decorators with parameters” (in fact, decorator factories):

def repeat(times):
    """ повторить вызов times раз, и вернуть среднее значение """
    def decorator(fn):
        def decorated2(*x):
            total = 0
            for i in range(times):
                total += fn(*x)
            return total / times
        return decorated2
    return decorator
@repeat(5)
def cpuload():
    """ внутри функции cpuload ничего не изменилось """
print "cpuload.__name__==" + cpuload.__name__
print "CPU load is %d%%" % cpuload()
( Full source )
cpuload .__ name __ == decorated2
cpuload () returns 7
cpuload () returns 16
cpuload () returns 0
cpuload () returns 0
cpuload () returns 33
CPU load is 11%
The value of the expression repeat(5)is a function decoratorclosed on times=5. This value is used as a decorator; in fact, we def cpuload(): ...; cpuload=repeat(5)(cpuload)

can combine several decorators on one function, then they are applied in the natural order - from right to left. If the two previous examples are combined in @timed @repeat(5) def cpuload():- then we get the output
cpuload .__ name __ == decorated
cpuload () returns 28
cpuload () returns 16
cpuload () returns 0
cpuload () returns 0
cpuload () returns 0
Executing decorated2 took 503 ms
CPU load is 9%
And if you change the order of decorators @repeat(5) @timed def cpuload():- then we get
cpuload .__ name __ == decorated2
cpuload () returns 16
Executing cpuload took 100 ms
cpuload () returns 14
Executing cpuload took 109 ms
cpuload () returns 0
Executing cpuload took 101 ms
cpuload () returns 0
Executing cpuload took 100 ms
cpuload () returns 0
Executing cpuload took 99 ms
CPU load is 6%
In the first case, the announcement turned in cpuload=timed(repeat(5)(cpuload)), in the second case, in cpuload=repeat(5)(timed(cpuload)). Pay attention to the printed function names: you can trace the chain of calls in both cases.

The limiting case of parametric decoration is a decorator that takes a decorator as a parameter :
def toggle(decorator):
    """ позволить "подключать" и "отключать" декоратор """
    def new_decorator(fn):
        decorated = decorator(fn)
        def new_decorated(*x):
            if decorator.enabled:
                return decorated(*x)
            else:
                return fn(*x)
        return new_decorated
    decorator.enabled = True
    return new_decorator
@toggle(timed)
def cpuload():
    """ внутри функции cpuload ничего не изменилось """
print "cpuload.__name__==" + cpuload.__name__
print "CPU load is %d%%" % cpuload()
timed.enabled = False
print "CPU load is %d%%" % cpuload()
( Full source )
cpuload .__ name __ == new_decorated
cpuload () returns 28
Executing cpuload took 101 ms
CPU load is 28%
cpuload () returns 0
CPU load is 0%
The value that controls connecting / disconnecting the decorator is stored in the attribute of the enableddecorated function: Python allows you to "stick" arbitrary attributes to any function.

The resulting function togglecan also be used as a decorator for decorators :

@toggle
def timed(fn):
    """ внутри декоратора timed ничего не изменилось """
@toggle
def repeat(times):
    """ внутри декоратора repeat ничего не изменилось """
@timed
@repeat(5)
def cpuload():
    """ внутри функции cpuload ничего не изменилось """
print "cpuload.__name__==" + cpuload.__name__
print "CPU load is %d%%" % cpuload()
timed.enabled = False
print "CPU load is %d%%" % cpuload()
( Full source )
cpuload .__ name __ == new_decorated
cpuload () returns 28
cpuload () returns 0
cpuload () returns 0
cpuload () returns 0
cpuload () returns 0
Executing decorated2 took 501 ms
CPU load is 5%
cpuload () returns 0
cpuload () returns 16
cpuload () returns 14
cpuload () returns 16
cpuload () returns 0
Executing decorated2 took 500 ms
CPU load is 9%
Um ... no, it didn't work! But why?
Why did the decorator timednot disconnect on the second call cpuload?

Recall that our global name timedis associated with a decorated decorator, i.e. with function new_decorated; it means that the line timed.enabled = Falseactually changes the attribute of the function new_decorated- the common “wrapper” of both decorators. It would be possible to check inside new_decoratedinstead , but then the line would turn off both decorators at once. Let's fix this bug: in order to use the attribute on the “internal” decorator, as before - we’ll stick a couple of methods on the function :if decorator.enabled:if new_decorator.enabled:timed.enabled = False

enablednew_decorated

def toggle(decorator):
    """ позволить "подключать" и "отключать" декоратор """
    def new_decorator(fn):
        decorated = decorator(fn)
        def new_decorated(*x): # без изменений
            if decorator.enabled:
                return decorated(*x)
            else:
                return fn(*x)
        return new_decorated
    def enable():
        decorator.enabled = True
    def disable():
        decorator.enabled = False
    new_decorator.enable = enable
    new_decorator.disable = disable
    enable()
    return new_decorator
print "cpuload.__name__==" + cpuload.__name__
print "CPU load is %d%%" % cpuload()
timed.disable()
print "CPU load is %d%%" % cpuload()
( Entire source ) The
desired result was achieved - it timedturned off, but repeatcontinued to work:
cpuload .__ name __ == new_decorated
cpuload () returns 14
cpuload () returns 16
cpuload () returns 0
cpuload () returns 0
cpuload () returns 0
Executing decorated2 took 503 ms
CPU load is 6%
cpuload () returns 0
cpuload () returns 0
cpuload () returns 7
cpuload () returns 0
cpuload () returns 0
CPU load is 1%
This is one of the most charming features of Python - you can add not only attributes to functions, but also arbitrary function methods. Functions sit on functions and drive functions.

Also popular now: