Acquaintance with testing in Python. Part 1

Transfer

All good!

~~From our desk to yours ...~~ That is, from our Python Developer course, despite the rapidly approaching New Year, we prepared for you an interesting translation about various testing methods in Python.

This guide is for those who have already written a cool application in Python, but have not yet written
tests for them.

Testing in Python is an extensive topic with a lot of subtleties, but it is not necessary to complicate things. In a few simple steps you can create simple tests for the application, gradually increasing the complexity based on them.

In this tutorial, you will learn how to create a basic test, run it, and find all the bugs before users do it! You will learn about the tools available to write and run tests, check the performance of the application, and even look at security issues.

Testing the Code You can

test the code in different ways. In this tutorial, you will learn from simplest to advanced methods.

Automated vs. Manual Testing

Good news! Most likely you have already done the test, but have not yet realized this. Remember how you first launched the app and used it? Did you test and experiment with the features? This process is called exploratory testing, and is a form of manual testing.

Research testing - testing that is conducted without a plan. During research testing, you explore the application.

To create a complete list of manual tests, it is enough to make a list of all the functions of the application, the various types of input that it accepts, and the expected results. Now, every time you change something in the code, you need to re-check each of the elements of this list.

Sounds bleak, right?

Therefore, we need automatic tests. Automatic testing - the execution of a test plan (parts of an application that require testing, the order in which they are tested and expected results) using a script, and not by human hands. Python already has a set of tools and libraries to help you create automated tests for your application. Consider these tools and libraries in our tutorial.

Modular Tests VS. Integration Tests

The world of testing is full of terms, and now, knowing the difference between manual and automated testing, we will go down a level deeper.

Think about how you can test the headlights of a car? You turn on the headlights (let's call it the test step), get out of the car yourself, or ask a friend to check that the headlights are lit (and this is a test proposition). Testing multiple components is called integration testing.

Think of all the things that need to work correctly for a simple task to produce the correct result. These components are similar to parts of your application: all those classes, functions, modules that you wrote.

The main difficulty of integration testing occurs when the integration test does not give the correct result. It is difficult to assess the problem without being able to isolate the broken part of the system. If the lights are not lit, the bulbs may be broken. Or maybe the battery is dead? Or maybe the problem is in the generator? Or even a failure in the computer machine?

Modern cars themselves will notify you of the failure of light bulbs. This is determined by the unit test.

The unit test (unit test) is a small test that checks the correctness of a separate component. A unit test helps isolate a breakdown and fix it faster.

We talked about two types of tests:

Integration test, verifying the components of the system and their interaction with each other;
A unit test that checks a particular component of an application.
You can create both tests in Python. To write a test for the built-in function sum (), you need to compare the output of sum () with known values.

For example, this is how you can verify that the sum of the numbers (1, 2, 3) is 6:

>>> assert sum([1, 2, 3]) == 6, "Should be 6"

Values are correct, so nothing will be displayed in the REPL. If the result is sum()incorrect, it will be issued AssertionError with the message “Should be 6” (“Should be 6”). Check the assertion statement again, but now with incorrect values, to get AssertionError:

>>> assert sum([1, 1, 1]) == 6, "Should be 6"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: Should be 6

In the REPL you will see AssertionError, because the value is sum()not equal to 6.

Instead of the REPL, put it in a new Python file with the name test_sum.pyand execute it again:

deftest_sum():assert sum([1, 2, 3]) == 6, "Should be 6"if __name__ == "__main__":
    test_sum()
    print("Everything passed")

Now you have a written test case (test case), statement and entry point (command line). Now this can be done on the command line:

$ python test_sum.py
Everything passed

You see a successful result, “Everything passed”.

sum()in Python, it accepts any input as the first argument. You checked the list. Let's try to test the tuple. Create a new file test_sum_2.pywith the following code:

deftest_sum():assert sum([1, 2, 3]) == 6, "Should be 6"deftest_sum_tuple():assert sum((1, 2, 2)) == 6, "Should be 6"if __name__ == "__main__":
    test_sum()
    test_sum_tuple()
    print("Everything passed")

After executing test_sum_2.py, the script will generate an error, since s um() (1, 2, 2)must be equal to 5, not 6. As a result, the script produces an error message, a line of code and a traceback:

$ python test_sum_2.py
Traceback (most recent call last):
  File "test_sum_2.py", line 9, in <module>
    test_sum_tuple()
  File "test_sum_2.py", line 5, in test_sum_tuple
    assert sum((1, 2, 2)) == 6, "Should be 6"
AssertionError: Should be 6

You can see how an error in the code causes an error in the console with information about where it occurred and what the expected result was.

Such tests are suitable for simple verification, but what if there are more errors than in one? Test runners come to the rescue. The test runner is a special application designed for conducting tests, checking output data and providing tools for debugging and diagnosing tests and applications.

Choosing a

Test Artist For Python, a lot of test artists are available. For example, unittest is built into the standard Python library. In this tutorial, we will use test cases and unittest test executers. The principles of unittest work are easily adapted for other frameworks. We list the most popular test performers:

unittest;
nose or nose2;
pytest.

It is important to choose a test performer that meets your requirements and experience.

unittest

unittest is built into the standard Python library starting from version 2.1. You will surely come across it in commercial Python applications and open source projects.
In unittest there is a test framework and a test performer. When writing and executing tests you need to follow some important requirements.

unittest requires:

Put tests in classes as methods;
Use special approval methods. Class TestCase instead of the usual assert expression.

To turn a previously written example into a unittest test case, you need to:

Import unittest from standard library;
Create a class called TestSumthat will inherit the class TestCase;
Convert test functions to methods, adding selfas the first argument;
Modify assertions by adding use of the self.assertEqual()method in the class TestCase;
Change the entry point on the command line to a call unittest.main().

Following these steps, create a new test_sum_unittest.py file with this code:

import unittest
classTestSum(unittest.TestCase):deftest_sum(self):
        self.assertEqual(sum([1, 2, 3]), 6, "Should be 6")
    deftest_sum_tuple(self):
        self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")
if __name__ == '__main__':
    unittest.main()

Doing this on the command line will result in one successful completion (indicated by.) And one unsuccessful (indicated by F):

$ python test_sum_unittest.py
.F
======================================================================
FAIL: test_sum_tuple (__main__.TestSum)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_sum_unittest.py", line 9, in test_sum_tuple
    self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")
AssertionError: Should be 6
----------------------------------------------------------------------
Ran 2 tests in0.001s
FAILED (failures=1)

Thus, you have completed two tests with the help of the test performer unittest.

Note: If you are writing test cases for Python 2 and 3, be careful. In versions of Python 2.7 and below, unittest is called unittest 2. When importing from unittest, you will get different versions with different functions in Python 2 and Python 3.

To learn more about unittest, read the unittest documentation .

nose

Over time, after writing hundreds or even thousands of tests for an application, it becomes increasingly difficult to understand and use unittest output data.

Nose is compatible with all tests written with the unittest framework, and can be replaced by a test performer. The development of nose, as an open source application, began to slow down, and nose2 was created. If you start from scratch, it is recommended to use nose2.

To get started with nose2, you need to install it from PyPl and run it on the command line. nose2 will try to find all the test creaks with test*.pyin the name and all the test cases inherited from unittest.TestCase in your current directory:

$ pip install nose2
$ python -m nose2
.F
======================================================================
FAIL: test_sum_tuple (__main__.TestSum)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_sum_unittest.py", line 9, in test_sum_tuple
    self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")
AssertionError: Should be 6
----------------------------------------------------------------------
Ran 2 tests in0.001s
FAILED (failures=1)

This is how the test created by test_sum_unittest.py, from the nose2 test performer is performed. nose2 provides many command line flags for filtering executable tests. To learn more, we advise you to familiarize yourself with the documentation of Nose 2 .

pytest

pytest supports unittest test cases. But the real advantage of pytest is its test cases. Pytest test cases - a series of functions in a Python file with test_ at the beginning of the name.

It has other useful features:

Support for assert built-in expressions instead of using special self.assert * () methods;
Support filtering test cases;
The ability to restart from the last failed test;
An ecosystem of hundreds of plug-ins that extend functionality.

An example TestSum test case for pytest will look like this:

deftest_sum():assert sum([1, 2, 3]) == 6, "Should be 6"deftest_sum_tuple():assert sum((1, 2, 2)) == 6, "Should be 6"

You got rid of TestCase, using classes and command line entry points.
More information can be found on the Pytest Documentation Site .

Writing the First Test Let's

unite everything that we have already learned, and instead of the built-in function, we will sum()test a simple implementation with the same requirements.

Create a new folder for the project, inside which create a new folder called my_sum. Inside my_sum, create an empty file called _init_.py. The presence of this file means that the my_sum folder can be imported as a module from the parent directory.

The folder structure will look like this: Open and create a new function called

project/

│

└── my_sum/

 └── __init__.py

my_sum/__init__.py sum()which takes as input iterated (list, tuple, set) and adds values.

defsum(arg):
    total = 0for val in arg:
        total += val
    return total

This example creates a variable called total, enumerates all values in arg and adds them to total. Then, at the end of the iteration, the result is returned.

Where to Write a Test

You can start writing a test by creating a file test.pycontaining your first test case. For testing, the file should have the ability to import your application, so put it test.pyin a folder above the package. The directory tree will look like this: You will notice that as you add new tests, your file becomes more cumbersome and difficult to maintain, so we advise you to create a folder and split the tests into several files. Make sure all file names begin with

project/

│

├── my_sum/

│ └── __init__.py

|

└── test.py

tests/test_so that test workers understand that Python files contain tests that need to be performed. On large projects, tests are divided into several directories depending on their purpose or use.

Note: And what is your application is a single script?
You can import any script attributes: classes, functions, or variables using the built-in function __import__(). Instead from my_sum import sum write the following:

target = __import__("my_sum.py")
sum = target.sum

When used, __import__()you do not have to turn the project folder into a package, and you can specify the file name. This is useful if the file name conflicts with the names of the standard package libraries. For example, if it math.pyconflicts with the math module.

How to Structure a Simple Test

Before writing tests, you need to solve a few questions:

What do you want to test?
Are you writing a unit test or integration test?

Now you are testing sum(). For it, you can check different behaviors, for example:

Is it possible to sum up the list of integers?
Is it possible to sum up a tuple or a set?
Is it possible to sum up the list of floating point numbers?
What happens if you give a bad value to the input: a single integer or a string?
What happens if one of the values is negative?

The easiest way to test a list of integers. Create a file test.pywith the following code:

import unittest
from my_sum import sum
classTestSum(unittest.TestCase):deftest_list_int(self):"""
        Test that it can sum a list of integers
        """
        data = [1, 2, 3]
        result = sum(data)
        self.assertEqual(result, 6)
if __name__ == '__main__':
    unittest.main()

The code in this example is:

Imports sum()from the package my_sum()you created;
Defines a new test case class called TestSum, which inherits unittest.TestCase;
Defines a test method .test_list_int()for testing an integer list. The method .test_list_int()will do the following

Declare a variable datawith a list of values (1, 2, 3);
Assigns a value to a my_sum.sum(data)variable result;
Determines that the result value is 6 using a method .assertEqual()on the unittest.TestCaseclass.

Specifies the entry point of the command line that launches the unittest executor .main().

If you don't know what self is, or how it is defined .assertEqual(), you can refresh your knowledge of object-oriented programming with Python 3 Object-Oriented Programming .

How to Write Statements

The final step in writing a test is to check that the output matches the known values. This is called assertion. There are several general guidelines for writing statements:

Check that the tests are repeatable and run them several times to make sure that they give the same results each time;
Check and confirm the results that relate to your input data - check that the result is really the sum of the values in the example sum().

In unittest, there are many methods for validating the values, types, and existence of variables. Here are some of the most commonly used methods:

Method	Equivalent
.assertEqual (a, b)	a == b
.assertTrue (x)	bool (x) is true
.assertFalse (x)	bool (x) is False
.assertIs (a, b)	a is b
.assertIsNone (x)	x is None
.assertIn (a, b)	a in b
.assertIsInstance (a, b)	isinstance (a, b)

In .assertIs(), .assertIsNone(), .assertIn(), and .assertIsInstance()there are conflicting methods called .assertIsNot()and so on.

Side Effects

Writing tests is more difficult than just looking at the return value of a function. Often, code execution changes other parts of the environment: class attributes, filesystem files, values in the database. This is an important part of testing, which is called side effects. Decide whether you are testing a side effect before including it in your list of statements.

If you find that there are a lot of side effects in the block of code that you want to test, then you are violating the principle of sole responsibility.. Violating the principle of sole responsibility means that a piece of code does too many things and requires refactoring. Adherence to the sole responsibility principle is a great way to design code for which it’s easy to write simple repeatable unit tests and, ultimately, create reliable applications.

Running the First Test

You created the first test and now you need to try it out. It is clear that it will be passed, but before creating more complex tests, you need to make sure that even such tests are performed successfully.

Running

Test Performers A test runner is a Python application that executes test code, verifies assertions, and displays test results in the console. At the end of test.py, add this small piece of code:

if __name__ == '__main__':
    unittest.main()

This is the command line entry point. If you run this script by running python test.pyon the command line, it will invoke unittest.main(). This launches the test runner, detecting all classes in this file that are inherited from unittest.TestCase.

This is one of the many ways to run the unittest executor. If you have a single test file called test.py, calling python test.py is a great way to get started.

Another way is to use the unittest command line. Let's try:

$ python -m unittest test

This will execute the same test module (called test) via the command line. You can add additional parameters to change the output. One of them is -v for verbose. Let's try the following:

$ python -m unittest -v test
test_list_int (test.TestSum) ... ok
----------------------------------------------------------------------
Ran 1 tests in 0.000s

We ran one test from test.py and output the results to the console. Verbose mode lists the names of the tests performed and the results of each one.

Instead of providing the name of the module containing the tests, you can request auto-detection using the following:


$ python -m unittest discover

This command will search for files in the current directory with test*.pya name to test them.

If there are several test files and the naming pattern is followed test*.py, you can pass the directory name with the -s flag and the folder name.

$ python -m unittest discover -s tests

unittest will run all the tests in a single test plan and display the results.
Finally, if your source code is not in the root directory, but in a subdirectory, for example, in a folder called src /, you can use the -t flag to tell unittest where to run the tests in order to import modules correctly:

$ python -m unittest discover -s tests -t src

unittest will find all the files test*.pyin the directory src/inside testsand then execute them.

Understanding the Results Testing

This was a very simple example where everything went well, so let's try to understand the output of the failed test.

sum() must accept other lists of numeric type, for example, fractions.

To the beginning of the code in the file, test.pyadd an expression to import the type Fraction from the module of the fractions standard library.

from fractions import Fraction

Now add a test with an assertion, expecting an incorrect value. In our case, we expect that the sum of ¼, ¼ and ⅖ will be equal to 1:

import unittest
from my_sum import sum
classTestSum(unittest.TestCase):deftest_list_int(self):"""
        Test that it can sum a list of integers
        """
        data = [1, 2, 3]
        result = sum(data)
        self.assertEqual(result, 6)
    deftest_list_fraction(self):"""
        Test that it can sum a list of fractions
        """
        data = [Fraction(1, 4), Fraction(1, 4), Fraction(2, 5)]
        result = sum(data)
        self.assertEqual(result, 1)
if __name__ == '__main__':
    unittest.main()

If you run the tests again with python -m unittest test, get the following:

$ python -m unittest test
F.
======================================================================
FAIL: test_list_fraction (test.TestSum)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 21, in test_list_fraction
    self.assertEqual(result, 1)
AssertionError: Fraction(9, 10) != 1
----------------------------------------------------------------------
Ran 2 tests in0.001s
FAILED (failures=1)

In this output, you see the following:

The first line shows the results of all tests: one failed (F), one passed (.);
FAIL shows some details of the failed test:

The name of the test method ( test_list_fraction);
Test module ( test) and test case ( TestSum);
Treysbek line with an error;
Approval details with expected result (1) and actual result (Fraction (9, 10))

Remember, you can add additional information to the test output with the -v flag to the command python -m unittest.

Running tests from PyCharm

If you are using PyCharm IDE, you can run unittest or pytest by completing the following steps:

In the Project tool window, select the tests directory.
In the context menu, select the command start unittest. For example, 'Unittests in my Tests ...'.

This will execute unittest in a test window and display the results in PyCharm:

More information is available on the PyCharm website .

Running Tests from Visual Studio Code

If you are using Microsoft Visual Studio Code IDE, unittest, nose, and pytest support is already built into the Python plugin.

If you have it installed, you can customize the test configuration by opening Command Palette with Ctrl + Shift + P and writing “Python test”. You will see a list of options:

Select Debug All Unit Tests, after which VSCode will send a request to set up a test framework. Click on the gear to select the test person (unittest) and home directory (.).

After the setup is completed, you will see the status of the tests at the bottom of the screen and you can quickly access the test logs and rerun the tests by clicking on the icons:

We see that the tests are performed, but some of them failed.

THE END

In the next part of the article, we will look at tests for frameworks such as Django and Flask.

We are waiting for your questions and comments here and, as always, you can go to Stanislav on an open day .

The second part of

Tags:

Acquaintance with testing in Python. Part 1

Also popular now: