A pseudo-practical example of closures and decorators

When I was just starting to learn Python, route decorators in the famous flask framework made a big impression on me. Of course, I knew how they could be implemented, but as always, the desire to write (rather than read) exceeded the need to look at the flask source code, and I had to invent something that could look as lapidary as the aforementioned decorators from flask . An exercise on closures, decorators, and scope in Python might look like this:

def do_something(p):
    return p
@implements(do_something, lambda: not p % 2)
def do_mod2_something(p):
    return p / 2
@implements(do_something, lambda: not p % 3)
def do_mod3_something(p):
    return p / 3
do_something(10)  # returns 5
do_something(9)   # returns 3
do_something(11)  # returns 11

How to implement the @implements decorator? Whether such an implementation can be used somewhere in real projects is a question that we rarely take into account when devising exercises for ourselves to understand how certain programs work. It seemed to me that this looks like a kind of override of a function that occurs in other programming languages.

Override

In languages ​​with static typing of data, there is such a technique as replacing the implementation of a function. Using a signature during compilation, a function suitable for the call is selected. In C ++ and Java, for example, this technique is often used to have multiple function implementations for arguments of different data types. To fully understand what is at stake, the following is an almost canonical example of replacing a function with C ++:

#include 
int sum(int a, int b)
{
    std::cout << "int" << std::endl;
    return a + b;
}
double sum(double a, double b)
{
    std::cout << "double" << std::endl;
    return a + b;
}
int main(void)
{
    std::cout << sum(1, 2) << std::endl;
    std::cout << sum(1.1, 3.0) << std::endl;
    return 0;
}

In programming languages ​​with dynamic typing, there is practically no need to support implementations for different types of data. However, what if we have the opportunity to run various implementations of the function depending on the values ​​of the arguments? For example, in FSM, where at each step it is necessary to check the current state and perform a transition to another. Or in the implementation of some very platform-specific functions. Can we in some way, without using the chains from if-then-else, implement this in Python?

It seems like almost everything can be implemented in Python. Of course, not without possible losses in performance, but the presence of such powerful tools as closures and decorators opens up scope for realizing your own bicycles and unhealthy fantasies.

Functions

Functions are first class objects. This is written in every book on programming in Python. This makes it possible to create functions at runtime, change their attributes and generally treat them like ordinary objects.

Quite a lot has been written about decorators not only on this resource, so I don’t want to go deep into this topic. Closures are function objects that store the environment with them. In fact, each decorated function is a closure, carrying with it not only the function code, but also the entire environment that existed inside the decorator during the definition of the function:

In [1]: def m(p):
   ...:     def s():
   ...:         return p
   ...:     return s
   ...: 
In [2]: x = m(10)
In [3]: x.func_closure
Out[3]: (,)

From this example it can be seen that the function x () contains information about an integer object. This object will exist as long as the x () function exists.

In addition, the function contains information about the environment in which it was defined. To do this, use the func_globals attribute, represented by a dictionary that can be changed. These features will be used to implement the @implements decorator.

@implements

def implements(orig_obj, requirements=lambda: False):
    ...

The decorator declares the implementation of the orig_obj object to be decorated if the requirements conditions are met during a call. An example of use was given at the beginning of the article. The implementation of the decorator does not allow calling orig_obj from the implementation of the function, but this is easily solved by adding additional attributes to the functions and checking them during the call of the function to be decorated.

In a nutshell about how the decorator works. When called, the decorator searches for orig_obj in the global namespace using the globals () function. This is necessary to replace the call to the original function with the orig_wrapper handler.

Next, it is checked whether the object found by name is a wrapper for the original function by checking for the presence of the __orig_wrapper__ attribute. If this attribute is absent, then substitution is performed. The __impl__ attribute is added to the replacement function to store implementations and conditions.

As soon as the first decorator has been called, do_something changes its behavior in such a way that before executing its own implementation, it checks all the requirements conditions, and if any condition is satisfied, the decoded function will be called. The implementation uses the above function attribute func_globals so that the lambda expression is executed in the required context.

Conclusion

I’m not sure that this approach to organizing various implementations can be convenient and “ideologically” true, but studying and working on this example was a good exercise for me to understand how closures and scope in Python work.