Good, bad, evil - testing in a beginners project

Foreword: the task was received at the university - to assemble a scrum team, select a project and work on it for a semester. Our team chose web application development (react + flask). In this article I will try to tell you what tests should have been and analyze what we did on the backend.



Expectations


Tests are necessary, first of all, in order to convince everyone (including ourselves) that the program behaves as it should in test situations . Secondly, they ensure the performance of the code covered by tests in the future. Writing tests is a useful process, because in its process you can often stumble on problematic places, recall some extreme cases, see problems with interfaces, etc.


When developing any systems, you need to remember at least three types of tests:


  • Unit tests are tests that verify that functions do what they need.
  • Integration tests are tests that verify that several functions together do the right thing.
  • System tests are tests that verify that the entire system does what it needs to.

In one of the posts from google, a table was published with the characteristics of three types of tests. "Small", "Medium" and "Large".



Unit tests


Unit tests correspond to small tests - they should be fast and only check the correctness of specific parts of the program. They should not access the database, should not work in complex multi-threaded environments. They control compliance with specifications / standards, often they have the role of regression tests .


Integration tests


Integration tests are those tests that can affect several modules and functions. Such tests require more time and may require special environments. They are necessary to make sure that individual modules and functions can work with each other. Those. unit tests verify the conformity of real interfaces to the expected, and integration tests - that functions and modules interact correctly with each other.


System tests


This is the highest level of automatic testing. System tests verify that the whole system works, that its parts perform their tasks and are able to interact correctly.


Why keep track of types


Usually, with the growth of the project, the code base will also grow. The duration of automatic checks will increase, supporting a large number of integration and system tests will become more and more difficult. Therefore, the challenge for developers is to minimize the necessary tests. To do this, try to use unit tests where possible and reduce integration using "mocks" (mocks).


Reality


Typical API Test


deftest_user_reg(client):return json.loads(
        client.post(url, json=data, content_type='application/json').data
    )
    response = client.post('api/user.reg', json={
        'email': 'name@mail.ru',
        'password': 'password1',
        'first_name': 'Name',
        'last_name': 'Last Name'
    })
    data = json.loads(response.data)
    assert data['code'] == 0

From the official documentation of flask, we get a ready-made recipe for initializing the application and creating the database. Here comes the work with the database. This is not a unit test, but not a system test. This is an integration test that uses a database test application.


Why integration rather than modular? Because in the processing of requests, interaction with flask, with ORM, with our business logic is performed. Handlers act as a unifying element of other parts of the project, so writing unit tests for them is not too simple (you need to replace the database with mokami, internal logic) and not too practical (integration tests will check similar aspects - "were the necessary functions called?", " were the data correctly received? ", etc.).


Names and grouping of tests


deftest_not_empty_errors():assert validate_not_empty('email', '') == ('email is empty',)
    assert validate_not_empty('email', '  ') == ('email is empty',)
    assert validate_email_format('email', "") == ('email is empty',)
    assert validate_password_format('pass', "") == ('pass is empty',)
    assert validate_datetime('datetime', "") == ('datetime is empty',)

In this test, all conditions for the "small" tests are met - the behavior of the function without dependencies is checked for compliance with the expected. But the design raises questions.


It is good practice to write tests that focus on a specific aspect of the program. In this example, there are different functions - validate_password_format, validate_password_format, validate_datetime. Grouping checks is not based on the result, but on the test objects.


The test name ( test_not_empty_errors) does not describe the test object (which method is being tested), it describes only the result (errors are not empty). This method was worth calling test__validate_not_empty__error_on_empty. This name describes what is being tested and what result is expected. This applies to almost every test name in the project due to the fact that no time was taken to discuss the test naming conventions.


Regression tests


deftest_datetime_errors():assert validate_datetime('datetime', '0123-24-31T;431') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2018-10-18T20:21:21+-23:1') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-13-20T20:20:20+20:20') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-02-29T20:20:20+20:20') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-12-20T25:20:20+20:20') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-12-20T20:61:20+22:20') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-12-20T20:20:61+20:20') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-12-20T20:20:20+25:20') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-12-20T20:20:20+20:61') == ('datetime is invalid',)
    assert validate_datetime('datetime', '2015-13-35T25:61:61+61:61') == ('datetime is invalid',)

This test originally consisted of the first two assert. After that, a "bug" was discovered - instead of checking the date, only the regular expression was checked, i.e. 9999-99-99was considered a normal date. The developer fixed it. Naturally, after fixing the bug, you need to add tests to prevent future regression. Instead of adding a new test in which to write why this test exists, checks have been added to this test.


What should a new test be called in which to add verification? Probably test__validate_datetime__error_on_bad_datetime.


Ignoring tools


deftest_get_providers():classTmp:def__init__(self, id_external, token, username):
            self.id_external = id_external
            self.token = token
            self.username = username
    ...

Tmp? This is a substitution for an object that is not used in this test. The developer did not seem to know about the existence of @patchand MagicMockfrom unittest.mock. No need to complicate the code, solving problems naively when there are more adequate tools.


There is such a test that initializes the services (in the database), uses the application context.


deftest_get_posts(client):deffake_request(*args, **kwargs):return [one, two]
    handler = VKServiceHandler()
    handler.request = fake_request
    services_init()
    with app.app_context():
        posts = handler.get_posts(None)
    assert len(posts) == 2

You can exclude database and context work from the test by simply adding one @patch.


@patch("mobius.services.service_vk.Service")deftest_get_posts(mock):deffake_request(*args, **kwargs):return [one, two]
    handler = VKServiceHandler()
    handler.request = fake_request
    posts = handler.get_posts(None)
    assert len(posts) == 2

Summary


  • To develop quality software, you need to write tests. At a minimum, to make sure you write what you need.
  • For bulky information systems, tests are even more important - they allow you to avoid unwanted interface changes or return bugs.
  • So that the written tests do not turn into a lot of strange methods over time, you need to pay attention to the naming convention of the tests, adhere to good practices, and minimize the tests.
  • Unit tests can be a great tool during development. They can be run after every small change to make sure nothing is broken.

A very important point is that the tests do not guarantee the availability or absence of bugs. Tests ensure that the real result of the program (or part of it) is expected. In this case, verification only occurs for those aspects for which tests were written. Therefore, when creating a quality product, we should not forget about other types of testing.


Also popular now: