Introducing Testing in Python. Part 3

Friends, we have great news for you. Firstly, the sun is finally shining on the street, which means that spring is beginning to fully take over its rights. The second news is more specialized - on March 20, the first lesson starts in a new thread on the course "Python Developer" , in connection with this we publish the final part of the article "Introduction to Testing in Python", the previous parts of which can be read here and here .

Testing in Multiple Environments

So far, you have been testing for one version of Python using a virtual environment with a specific set of dependencies. But there may always be a need to test the application on several versions of Python or several versions of the package. Tox is an application that automates testing in multiple environments.



Installing Tox

Tox is available on PyPl as a package to install through pip:

$ pip install tox

After installation, you can proceed to configure Tox.

Configuring Tox For your Dependencies,

Tox is configured through a configuration file in the project directory. It contains the following:

• The command to run to run the tests;
• Any additional packages required to run;
• Target versions of Python selected for testing.

Instead of learning the syntax for configuring Tox, you can start by launching a quickstart application.

$ tox-quickstart

The Tox configuration tool will ask you questions and create a file similar to the following in tox.ini:

[tox]
envlist = py27, py36
[testenv]
deps =
commands =
    python -m unittest discover

Before starting Tox, make sure that there is a file in the application folder setup.pywith the steps for installing the package. If not, use the creation guidesetup.py .

And if your project is not intended for distribution on PyPl, you can skip this requirement by adding the following line to the tox.ini file under the heading tox:

[tox]
envlist = py27, py36
skipsdist=True

If you do not create setup.py, and the application has some PyPl dependencies, you will need to clarify them in the section testenv. For example, Django will require the following:

[testenv]
deps = django

At the end of this step, you can run the tests.

Now you can run Tox, and it will create two virtual environments: one for Python 2.7 and one for Python 3.6. The Tox directory is called .tox/. In it, Tox will execute -m unittest discoverfor each virtual environment.

You can start this process by calling Tox from the command line:

$ tox

Tox will produce test results for each environment. When you start Tox for the first time, it takes time to create virtual environments, but when you run Tox for the second time, everything will work much faster.
Tox's results are pretty simple. Environments are created for each version, dependencies are installed, and then test commands are run.

There are a few additional command line options worth remembering.
Running a single environment, for example, Python 3.6:

$ tox -e py36

Re-creating the virtual environment when the dependency changes or side-packages get corrupted :

$ tox -r

Running Tox with less detailed findings:

$ tox -q

Running Tox with more verbose output:

$ tox -v

You can read more about Tox on the Tox documentation site .

Test Automation

So far, you have performed tests manually by running the command. But there are tools for automatically running tests when changes are made and commit them to a repository with a version control system, for example, Git. Test automation tools are often referred to as CI / CD tools, which means “Continuous Integration / Continuous Deployment”. They can run tests, compile and publish applications, and even deploy them to production.
Travis CI is one of the many CI services available.

Travis CI works well with Python, and now you can automate the execution of all created tests in the cloud! Travis CI is free for any open source projects on GitHub and GitLab and is available for a fee for private projects.

To get started, log in and authenticate using your GitHub or GitLab credentials. Then create a file with the name .travis.ymlwith the following contents:

language: python
python:
  - "2.7"
  - "3.7"
install:
  - pip install -r requirements.txt
script:
  - python -m unittest discover

This configuration gives Travis CI the following directions:

• Testing for Python 2.7 and 3.7 (Optionally, you can replace them with any others.)
• Installing all the packages listed in requirements.txt (You can remove this section if you have no dependencies.)
• Running python -m unittest discover to run tests.

After committing and pushing this file, Travis CI will run these commands every time you push to your remote Git repository. Results can be viewed on their website.

What's Next

Now you know how to write tests, add them to your project, execute them and even do it automatically, so you can get acquainted with advanced methods that can come in handy as the test library grows.

Introduction of Linter to the Application

Tox and Travis CI have a test command setup. In this tutorial, we used python -m unittest discover as a test team.

You can provide one or more commands in these tools, which will add new tools to improve the quality of the application.

One such application is the linter. He will look at your code and leave comments. Thus, he can give advice on errors, correct trailing spaces and even anticipate potential bugs.

To learn more about linters, check out the Python Code Quality Tutorial .

Passive Linting with flake8

flake8 is a popular linter that leaves comments about the style of your code in accordance with the PEP 8 specification.

You flake8can install using pip:

$ pip install flake8

Then you can run flake8for a single file, folder or template:

$ flake8 test.py
test.py:6:1: E302 expected 2 blank lines, found 1
test.py:23:1: E305 expected 2 blank lines after class or function definition, found 1
test.py:24:20: W292 no newline at end of file

You will see a list of errors and warnings in your code found flake8.
flake8can be configured on the command line or in the project configuration file. If you want to ignore some rules, for example E305, shown above, you can set this in the configuration. flake8will check the file .flake8in the project folder or file setup.cfg. If you want to use Tox, you can add the settings section flake8to tox.ini.

This example ignores the directories .git and __pycache__as well as rule E305. In addition, the maximum length of a string increases from 80 characters to 90. You will at some point realize that the standard limit of 79 characters per line is not suitable for tests that may contain long method names, string literals with test values ​​and other long pieces of data. Typically, for tests, increase the string length to 120 characters:

[flake8]
ignore = E305
exclude = .git,__pycache__
max-line-length = 90

Alternatively, you can provide these options on the command line:

$ flake8 --ignore E305 --exclude .git,__pycache__ --max-line-length=90

A complete list of settings can be found on the Documentation Site .
Now you can add flake8CI to the configuration. For Travis CI, it will look like this:

matrix:
  include:
    - python: "2.7"
      script: "flake8"

Travis will read the configuration in .flake8and will not be able to complete the build if there are linting errors. Make sure you add the dependency flake8to the file requirements.txt.

Aggressive Linting with the Code Formatter

flake8 is a passive linter that only recommends edits; you will have to enter them into the code yourself. Code formatter is a more aggressive approach. It changes the code automatically according to styles and layouts.

black- a very inexorable formatter. It has no settings and it is very meticulous. Which makes it a great tool to insert into your test pipeline.

Please note: black requires Python version 3.6 and higher.

You blackcan install using pip:

$ pip install black

Then, to start from the command line, specify the file or directory that you want to format:

$ black test.py

Keep the Test Code Clean.

You may notice that when writing tests you will be able to copy-paste code fragments much more often than when creating regular applications. From time to time, tests can be very monotonous, but this is not a reason to drop the code in an inaccurate and fragmented form.

Over time, technical debt will accumulate in your test code , and making the changes necessary for significant changes in the application code in the tests will turn out to be very difficult just because of the structure.

When writing tests, try to follow the DRY principle: Don’t Repeat Yourself.

Test fixtures and functions are a great way to write code that is easy to maintain. Also, don't forget about the ease of reading. Consider deploying linting tools, for example, flake8to your test code:

$ flake8 --max-line-length=120 tests/

Testing to Reveal Performance Decreases between Edits

There are many ways to benchmark code in Python. The standard library has a timeit module that schedules functions several times and shows you the distribution. In this example, test () will be executed 100 times, and then output will be given using print ():

def test():
    # ... your code
if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test()", setup="from __main__ import test", number=100))

If you decide to use pytest as a test runner, check out the pytest-benchmark plugin. It provides a pytest fixture called benchmark. Any called object can be passed benchmark (), it parses the time of the called in pytest results.

You can install pytest-benchmark from PyPl using pip:

$ pip install pytest-benchmark

Then you can add a test using fixture and passing the called object to execution:

def test_my_function(benchmark):
    result = benchmark(test)

Performing pytest will give you benchmark results:



You can learn more on the Documentation Website .

Testing for Identifying Security Errors

Another test that should be run on your application is checking for common errors and security vulnerabilities.

Install banditfrom PyPl using pip:

$ pip install bandit

Then you can transfer the name of your application module with a flag -rand get a brief information:

$ bandit -r my_sum
[main]  INFO    profile include tests: None
[main]  INFO    profile exclude tests: None
[main]  INFO    cli include tests: None
[main]  INFO    cli exclude tests: None
[main]  INFO    running on Python 3.5.2
Run started:2018-10-08 00:35:02.669550
Test results:
        No issues identified.
Code scanned:
        Total lines of code: 5
        Total lines skipped (#nosec): 0
Run metrics:
        Total issues (by severity):
                Undefined: 0.0
                Low: 0.0
                Medium: 0.0
                High: 0.0
        Total issues (by confidence):
                Undefined: 0.0
                Low: 0.0
                Medium: 0.0
                High: 0.0
Files skipped (0):

As in the case with flake8, flag rules banditcan be configured, and if you want to ignore some of them, you can add the following fragment to the file setup.cfgwith the parameters:

[bandit]
exclude: /test
tests: B101,B102,B301

More information on the GitHub website .

Conclusion

Python has made testing available thanks to the built-in commands and libraries necessary to verify the correct operation of applications. It's easy to start testing in Python: you can use unittest and write small, easy-to-maintain methods to test the code.

As you learn more about testing and expanding your application, consider switching to one of the test frameworks such as pytest in order to start using more advanced features.

Thanks for reading. Have an unmistakable future with Python!

And for those who have read the article, we have one more great news. You can get the Python Developer course right now .with a discount of 10,000 rubles!

Part One Part
Two