The problem of duplication and obsolescence of knowledge in mock-objects or Integration tests is good

    When choosing between integration and unit tests, many programmers prefer the unit test (or, in other words, the unit test). Some consider integration tests to be antipattern, some simply follow fashion trends. But let's see what this leads to. To implement the unit test, mock objects are hung not only on external services and data storages, but also on classes implemented directly inside the program. At the same time, if the mocked class is used in several other classes, then the mock-object will be contained in tests for several classes . And since it is customary to set the tested behavior inside the test (see given-when-then , arrange-act-assert , test builder), then the moka behavior is re-set each time in each test, and the DRY principle is violated (although code duplication may not exist). In addition, the class’s behavior is declared in the mock object, but this declaration itself is not checked, therefore, over time , the behavior declared in the mock can become outdated and begin to differ from the actual behavior of the class being mocked. This causes a number of difficulties:

    1) Firstly, when changing the functionality it’s hard to remember that in addition to the class and tests for it, you also need to change the mokeys of this class. Let's take a look at the development cycle within the framework of TDD: “creating / changing functional tests -> creating / changing functional -> refactoring”. Mock-objects are a declaration of the class’s behavior and are not related to any of these three categories (they are not functional tests, despite the fact that they are used in tests, and even less so are the functional itself). Thus, changing the mock-objects of classes implemented inside the program does not fit into the concept of TDD .

    2) Secondly, it is difficult to find all the places of this class. I have not seen a single tool for this. Here you can either write your bike, or watch all the places of use of this class and select those where mokas are created. But with a manual search, you can make a mistake and overlook something. Here, you probably have a question: if the problem is so fundamental, as the author describes, did it never occur to anyone to implement tools to simplify its solution? I have a hypothesis on this subject. A few years ago, I started writing a library that was supposed to build a mock object in the same way that an IOC container builds a regular class, and automatically create and run tests for the behavior described in moks. But then I abandoned this idea because I found a more elegantsolution to the mok problem: just don't create this problem . Probably, for a similar reason, a specialized tool for searching for moxes of a particular class is either not implemented or is little known.

    3) Thirdly, there can be many places for class mocking, and changing them all is a routine. If a programmer is forced to do a routine that cannot be automated, then this is a clear sign that something is wrong with the tools, architecture, or workflows.

    I hope the essence of the problem is clear. Next, I will describe ways to solve this problem and explain why, from my point of view, integration tests are preferable to unit tests.




    As a solution to the problem, I propose using mocking only for external services and data stores, and in other cases use real classes, i.e. write integration tests instead of unit tests. Some programmers are skeptical of integration tests, and they would not like this idea.

    Let's consider what arguments the opponents of integration tests give.

    Statement 1. Integration tests are less helpful in finding errors than unit tests.

    Proof:
    Let's imagine that in some class that is used everywhere, a mistake was made. After that, the tests of the class itself turned red, as well as all the integration tests in which this class was used. As a result, half of the tests in the project are red. How to understand what is the reason for the redness of the tests? What test to start with? But if a mock-object were used instead of a class, then only the tests of this class would blush.
    Disclaimer:
    Let's recall the workflow within the framework of TDD: “red” tests signaling an error -> creation / change of functionality -> “green” tests. Accordingly, when changing the functionality, the programmer first changes the tests so that they test the changed functionality. Since the code still contains outdated functionality, the tests fail. Then the programmer corrects the functional code, and the tests pass. If a programmer worked with classes, but not with their tests, then he did not act within the framework of TDD.
    But even if the programmer changed the code, but did not change the tests and did not check their progress, the crash of the tests can be tracked by the continuous integration server, which automatically runs the tests with each push to the version control system. The author of the changes will see a message about the crash of the tests, he will quickly remember which classes he rules, and first of all he will begin to deal with the tests of these particular classes. If a programmer inadvertently introduced a bug into a class, and then fixed it, then not only the tests of this class will turn green, but also all the tests in which this class was used. But what if they don't turn green? Then this is a signal that changes in the class have led to a change in the behavior of other classes where this class was used, and now either errors appeared in these classes, or their tests deviated from the application logic.
    Another case is possible. If for some reason the class in which the mistake was made was not well covered by tests, then unit tests on mokas would not have revealed the problem at all. Integration tests, however, at least signal a problem, although in order to identify the problem class one will have to resort to the good old trace.
    To summarize: if you follow TDD, then reddening the tests of those classes that you have not changed is an advantage because it signals problems. If you do not follow TDD, but use continuous integration, then reddening the "extra" tests is not such a problem for you. If you do not follow TDD and do not run tests on a regular basis, then the problem of identifying the “fallen test - problem class” correspondence is relevant for you. In this case, it is better to solve the problem of duplication of knowledge in mokas and the lack of tests for the behavior declared in mokas, not by using integration tests instead of unit ones, but by other means (we'll talk about them a bit later).

    Statement 2. Integration tests to a lesser extent help in design than modular

    Proof:
    Unit testing, unlike integration testing, forces programmers to inject dependencies through the constructor or properties. And if you use integration testing instead of unit testing, then the junior can instantiate dependencies directly in the class code. But I have no time to write architectural notes and review codes. Yes, and no one to charge. And I do not want to.
    Disclaimer:
    In fact, not only unit testing is able to force the programmer to inject dependencies. IOC-container does a great job of this. In fact, if you inject dependencies, then you are probably using an IOC-container. You can of course write the factory for creating the main class itself, where the entry point is located. But IOC-container solves many common problems and simplifies life. For example, you can make a class a singleton with one line of code without delving into the pitfalls of implementing a singleton. So if you inject dependencies but don’t use IOC-container, then I recommend starting to do this.
    In general, if you use unit testing, then you will almost certainly use IOC-container. If you use an IOC-container, then it encourages the programmer to inject dependencies. Of course, you can create an object without using IOC-container, but in the same way you can create a class without providing it with a unit test. So, I do not see any significant advantages in unit tests in terms of inducing the implementation of the Inversion of control principle.
    In addition, you can not force programmers to do what you need due to limitations in the architecture, but simply explain the benefits of dependency injection and using an IOC container. Coercion by force , like any violence, can cause counter resistance.

    Statement 3. To cover the same functionality with tests, integration tests will require much more than unit tests.

    Proof:
    The author of the article with the loud title “Integration Tests - the lot of crooks” writes that he passionately hates integration tests and considers them a virus that brings endless pain and suffering. He substantiates his thoughts as follows:
    You write integration tests because you are not able to write perfect unit tests. This problem is familiar to you: all your tests have passed, but the program still detects a defect. You decide to write an integration test to make sure that the entire program execution path works as it should. And everything seems to be going fine until you think: “And let's use integration tests everywhere.” Bad idea! The number of possible ways to execute a program non-linearly depends on the size of the program. You need at least 10,000 tests to cover a test application with 20 pages of tests. Maybe a million. When writing 50 tests per week, you write only 2,500 tests per year, which is 2.5% of the required amount. And after that, you wonder why you spend 70% of your time answering user calls ?! Integration tests are a waste of time. They must remain in the past.

    Disclaimer:
    The author of that article gives the following definition of an integration test:
    I use the term integrated test to mean any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior.
    An integration test is a test whose passing result depends on the correct implementation of more than one piece of non-trivial logic (method).

    As you can see, in this definition there is not a word that integration tests can only be written on the main class where the entry point is located, but the author of the above article implicitly relies on this condition in his reasoning.
    According to TDD, tests are designed to test functionality (feature), and not the ways of program execution. Follow the TDD and you will not encounter the problems that this author spoke about. Just write integration tests the same way you would write unit tests, but don’t wet the classes implemented in your program, and you will not encounter the problem of an exponential increase in the number of tests.

    Statement 4. Integration tests run longer than unit tests.

    Unfortunately, you can’t argue with this - integration tests are almost always performed longer than unit tests. Creating mocha, of course, is not free and takes some time, but the logic of the application, as a rule, takes longer. Hypothetically, it is quite possible that the tests run unsatisfactorily for a long time, and you are not going to optimize the tested logic in the near future. And a logical solution could be to optimize the tests. For example, the use of mok.

    Ways to combat duplication and obsolescence of knowledge in mokas


    The first way, as I have already said, is to use moki only to declare the behavior of external services and data storages.

    The second way is to automatically check the relevance of the behavior declared in moka. For example, you can automatically create and run the appropriate test. But then you need to take into account that the class being mocked can have its own dependencies, some of which can be external services. For performance, you can first test the unique behavior (indicated in the mokas) of the classes of the lowest layer, then the behavior of classes that use the previous classes, and so on. Then, if some identical behavior is declared in mokas in several places, then it can be checked only once.
    You can manually write a test for each unique case of moking and somehow set the correspondence between the moka and the test for it, and instruct the programmers to manually maintain this correspondence when changing the functionality.
    You can simply instruct programmers to manually maintain the relevance of mock objects. But then you will have to slightly change the workflow, moving away from the classic TDD, replace "Changing tests for functionality -> Changing functionality -> ..." with "Changing tests for functionality -> Changing declarations of this behavior (in moks) -> Changing functionality - > ... ".

    To eliminate the problem of code duplication during moking, you can put all moki in one class in a separate storage. This will simplify the “Changing Declarations of Behavior in Mocks” phase, but it can reduce the readability of the unit test - then decide for yourself based on your own priorities.

    Conclusion


    Martin Fowler has long noticed the formation of two different TDD schools - the classical school and the Moqists:
    Now I'm at the point where I can explore the second dichotomy: that between classical and mockist TDD. The big issue here is when to use a mock (or other double).

    The classical TDD style is to use real objects if possible and a double if it's awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn't really matter that much.

    A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service.

    Both of these schools have advantages and disadvantages. Personally, I believe that the flaws of a classic TDD are more acceptable and solvable than the flaws of a wet TDD. Well, someone can take it the other way around - he can cope perfectly with the consequences of using wet TDD and not consider acceptable the problems that arise with classic TDD. Why not? All people are different, and everyone has the right to their own style. I just made the reasons why I personally like the classic more, but the final choice is yours.

    PS I do not urge you to completely abandon unit tests. When using classical TDD, tests for those classes that do not access the methods and properties of other classes will be modular.

    Also popular now: