Python exceptions are now considered anti-pattern

What are exceptions? From the name it’s clear - they arise when an exception occurs in the program. You may ask why exceptions are an anti-pattern, and how do they relate to typing? I tried to figure it out , and now I want to discuss this with you, harazhiteli.

Exception Issues

It’s hard to find flaws in what you face every day. Habit and blindness turns bugs into features, but let's try to look at exceptions with an open mind.

Exceptions are hard to spot

There are two types of exceptions: “explicit” are created by calling raisedirectly in the code you are reading; "Hidden" are hidden in the used functions, classes, methods.

The problem is that the “hidden” exceptions are really hard to notice. I will show you an example of a pure function:

defdivide(first: float, second: float) -> float:return first / second

The function simply divides one number by another, returning float. The types are checked and you can run something like this:  

result = divide(1, 0)
print('x / y = ', result)

Have you noticed? In fact, the printexecution of the program will never reach, because dividing 1 by 0 is an impossible operation, it will call ZeroDivisionError. Yes, such code is type safe, but it cannot be used anyway.

To notice a potential problem even in such a simple and readable code, one needs experience. Anything in Python may not work with different types of exceptions: the division, function calls int, strgenerators, iterators in loops, access to attributes or keys. Even it raise something()can lead to failure. Moreover, I do not even mention input and output operations. And checked exceptions will no longer be supported in the near future.

Restoring normal behavior in place is not possible

But precisely for such a case, we have exceptions. Let's just process ZeroDivisionErrorit and the code will become type safe.

defdivide(first: float, second: float) -> float:try:
        return first / second
    except ZeroDivisionError:
        return0.0

Now everything is all right. But why do we return 0? Why not 1 or None? Of course, in most cases, getting is Nonealmost as bad (if not worse) as an exception, but still you need to rely on business logic and options for using the function.

What exactly are we sharing? Arbitrary numbers, any specific units or money? Not every option is easy to foresee and restore. It may turn out that the next time you use one function, it turns out that you need a different recovery logic.

The sad conclusion: the solution to each problem is individual, depending on the specific context of use.

There is no silver bullet that could cope ZeroDivisionErroronce and for all. And we are not talking about the possibility of complex I / O with repeated requests and timeouts.

Maybe it’s not necessary to handle exceptions exactly where they arise? Maybe just throw it into the code execution process - someone will figure it out later. And then we are forced to return to the current state of affairs.

Execution process unclear

Ok, let's hope someone else catches the exception and maybe handles it. For example, the system may ask the user to change the entered value, because it cannot be divided by 0. And the function divideclearly should not be responsible for recovering from the error.

In this case, you need to check where we caught the exception. By the way, how to determine exactly where it will be processed? Is it possible to go to the right place in the code? It turns out that no, it’s impossible .

It is not possible to determine which line of code will execute after the exception is thrown. Different types of exceptions can be handled with different options except, and some exceptions can be ignored.. And you can throw additional exceptions in other modules that will be executed earlier, and generally break all the logic.

Suppose there are two independent threads in an application: a regular thread that runs from top to bottom, and an exception thread that runs as it pleases. How to read and understand this code?

Only with the debugger turned on in the "catch all exceptions" mode.


Exceptions, as the notorious goto, tear the structure of the program.

Exceptions are not exclusive

Let's look at another example: the usual remote HTTP API access code:

import requests
deffetch_user_profile(user_id: int) -> 'UserProfile':"""Fetches UserProfile dict from foreign API."""
    response = requests.get('/api/users/{0}'.format(user_id))
    response.raise_for_status()
    return response.json()

In this example, literally everything can go wrong. Here is a partial list of possible errors:

• The network may not be available and the request will not be executed at all.
• The server may not work.
• Server may be too busy, timeout will occur.
• The server may require authentication.
• An API may not have such a URL.
• A non-existent user may be transferred.
• May not be enough rights.
• Server may crash due to an internal error while processing your request
• The server may return an invalid or damaged response.
• The server may return invalid JSON that cannot be parsed.

The list goes on and on, so many potential problems lie in the code of the unfortunate three lines. We can say that it generally works only by a lucky chance, and is much more likely to fall with an exception.

How to protect yourself?

Now that we’ve made sure that exceptions can be harmful to the code, let's figure out how to get rid of them. To write code without exception, there are different patterns.

• Everywhere to write except Exception: pass. Dead end. Do not do so.
• To return None. Too evil. As a result, you will either have to start almost every line with if something is not None:and all the logic will be lost behind the garbage of cleansing checks, or suffer all the time from TypeError. Not a good choice.
• Write classes for special use cases. For example, a base class Userwith subclasses for errors of type UserNotFoundand MissingUser. This approach can be used in some specific situations, such as AnonymousUserin Django, but wrapping all possible errors in classes is unrealistic. It will take too much work and the domain model will become unimaginably complex.
• Use containers to wrap the resulting variable or error value in a wrapper and continue to work with the container value. That is why we created the project @dry-python/return. So that functions return something meaningful, typed, and safe.

Let us return to the division example, which returns 0 when an error occurs. Can we explicitly indicate that the function did not succeed without returning a specific numeric value?

from returns.result import Result, Success, Failure
defdivide(first: float, second: float) -> Result[float, ZeroDivisionError]:try:
        return Success(first / second)
    except ZeroDivisionError as exc:
        return Failure(exc)

We enclose the values ​​in one of two wrappers: Successor Failure. These classes are inherited from the base class Result. Types of packed values ​​can be specified in the annotation of the returned function, for example, Result[float, ZeroDivisionError]returns either Success[float]or Failure[ZeroDivisionError].

What does this give us? More exceptions are not exceptional, but are expected problems . Also wrapping the exception in Failuresolves the second problem: the difficulty of identifying potential exceptions.

1 + divide(1, 0)
# => mypy error: Unsupported operand types for + ("int" and "Result[float, ZeroDivisionError]")

Now they are easy to spot. If you see in the code Result, then the function may throw an exception. And you even know his type in advance.

Moreover, the library is fully typed and compatible with PEP561 . That is, mypy will warn you if you try to return something that does not match the declared type.

from returns.result import Result, Success, Failure
defdivide(first: float, second: float) -> Result[float, ZeroDivisionError]:try:
        return Success('Done')
        # => error: incompatible type "str"; expected "float"except ZeroDivisionError as exc:
        return Failure(0)
        # => error: incompatible type "int"; expected "ZeroDivisionError"

How to work with containers?

There are two methods :

• map for functions that return normal values;
• bind for functions that return other containers.

Success(4).bind(lambda number: Success(number / 2))
# => Success(2)
Success(4).map(lambda number: number + 1)
# => Success(5)

The beauty is that such code will protect you from unsuccessful scripts, since .bindthey .mapwill not execute for c containers Failure:

Failure(4).bind(lambda number: Success(number / 2))
# => Failure(4)
Failure(4).map(lambda number: number / 2)
# => Failure(4)

Now you can just concentrate on the correct execution process and be sure that the wrong state will not break the program in an unexpected place. And there is always the opportunity to determine the wrong state, correct it , and return back to the conceived path of the process.

Failure(4).rescue(lambda number: Success(number + 1))
# => Success(5)
Failure(4).fix(lambda number: number / 2)
# => Success(2)

In our approach, “all problems are solved individually,” and “the execution process is now transparent.” Enjoy programming that rides on rails!

But how to expand values ​​from containers?

Indeed, if you work with functions that do not know anything about containers, you need the values ​​themselves. Then you can use methods .unwrap()or .value_or():

Success(1).unwrap()
# => 1
Success(0).value_or(None)
# => 0
Failure(0).value_or(None)
# => None
Failure(1).unwrap()
# => Raises UnwrapFailedError()

Wait, we had to get rid of the exceptions, and now it turns out that all calls .unwrap()can lead to another exception?

How not to think about UnwrapFailedErrors?

Ok, let's see how to live with the new exceptions. Consider this example: you need to check user input and create two models in the database. Each step may end with an exception, which is why all methods are wrapped in Result:

from returns.result import Result, Success, Failure
classCreateAccountAndUser(object):"""Creates new Account-User pair."""# TODO: we need to create a pipeline of these methods somehow...def_validate_user(
        self, username: str, email: str,
    ) -> Result['UserSchema', str]:"""Returns an UserSchema for valid input, otherwise a Failure."""def_create_account(
        self, user_schema: 'UserSchema',
    ) -> Result['Account', str]:"""Creates an Account for valid UserSchema's. Or returns a Failure."""def_create_user(
        self, account: 'Account',
    ) -> Result['User', str]:"""Create an User instance. If user already exists returns Failure."""

Firstly, you don’t have to expand the values ​​in your own business logic at all:

classCreateAccountAndUser(object):"""Creates new Account-User pair."""def__call__(self, username: str, email: str) -> Result['User', str]:"""Can return a Success(user) or Failure(str_reason)."""return self._validate_user(
            username, email,
        ).bind(
            self._create_account,
        ).bind(
            self._create_user,
        )
   # ...

Everything will work without any problems, no exceptions will be raised, because it is not used .unwrap(). But is it easy to read such code? Not. And what is the alternative? @pipeline:

from result.functions import pipeline
classCreateAccountAndUser(object):"""Creates new Account-User pair."""    @pipelinedef__call__(self, username: str, email: str) -> Result['User', str]:"""Can return a Success(user) or Failure(str_reason)."""
        user_schema = self._validate_user(username, email).unwrap()
        account = self._create_account(user_schema).unwrap()
        return self._create_user(account)
   # ...

Now this code is well read. Here's how .unwrap()they @pipelinework together: whenever a method .unwrap()fails and the Failure[str]decorator @pipelinecatches it and returns it Failure[str]as the resulting value. This is how I propose to remove all exceptions from the code and make it really safe and typed.

Wrap it all together

Ok, now we’ll apply the new tools, for example, with a request to the HTTP API. Remember that each line can throw an exception? And there is no way to get them to return the container with Result. But you can use the @safe decorator to wrap unsafe functions and make them safe. Below are two code options that do the same thing:

from returns.functions import safe
@safedefdivide(first: float, second: float) -> float:return first / second
# is the same as:defdivide(first: float, second: float) -> Result[float, ZeroDivisionError]:try:
        return Success(first / second)
    except ZeroDivisionError as exc:
        return Failure(exc)

The first, with @safe, is easier and better to read.

The last thing to do in the example with the request to the API is to add a decorator @safe. The result is this code:

import requests
from returns.functions import pipeline, safe
from returns.result import Result
classFetchUserProfile(object):"""Single responsibility callable object that fetches user profile."""#: You can later use dependency injection to replace `requests`#: with any other http library (or even a custom service).
    _http = requests
    @pipelinedef__call__(self, user_id: int) -> Result['UserProfile', Exception]:"""Fetches UserProfile dict from foreign API."""
        response = self._make_request(user_id).unwrap()
        return self._parse_json(response)
    @safedef_make_request(self, user_id: int) -> requests.Response:
        response = self._http.get('/api/users/{0}'.format(user_id))
        response.raise_for_status()
        return response
    @safedef_parse_json(self, response: requests.Response) -> 'UserProfile':return response.json()

To summarize how to get rid of exceptions and secure the code :

• Use a wrapper @safefor all methods that may throw an exception. It will change the return type of the function to Result[OldReturnType, Exception].
• Use Resultas a container to transfer values ​​and errors into a simple abstraction.
• Use .unwrap()to expand the value from the container.
• Use @pipelineto make call sequences .unwrapeasier to read.

By observing these rules, we can do exactly the same thing - only safe and well readable. All problems that were with exceptions were resolved:

• “Exceptions are hard to spot . ” Now they are wrapped in a typed container Result, which makes them completely transparent.
• "Restoring normal behavior in place is impossible . " Now you can safely delegate the recovery process to the caller. In such a case there is .fix()also .rescue().
• "The sequence of execution is unclear . " Now they are one with the usual business flow. From start to finish.
• “Exceptions are not exceptional . ” We know! And we expect that something will go wrong and ready for anything.

Use Cases and Limitations

Obviously, you cannot use this approach in all of your code. It will be too safe for most everyday situations and is incompatible with other libraries or frameworks. But you must write the most important parts of your business logic exactly as I showed, in order to ensure the correct operation of your system and facilitate future support.

Does the topic make you think or even seems holivarny? Come to Moscow Python Conf ++ on April 5, we will discuss! Besides me, Artyom Malyshev, the founder of the dry-python project and core developer of Django Channels, will be there. He will talk more about dry-python and business logic.