Misconceptions about automatic testing

Hello, my name is Dmitry Karlovsky and this is a continuation of the traditional heading "Why do we not like to write tests?" Short answer: because the bonuses received from them do not outweigh the efforts expended. If so, then we are doing something wrong. Let's see what could have gone wrong ..


A picture to attract attention


This note has grown from the chapter “Misconceptions” of the “Automated Testing Concepts ” longread , by adding new misconceptions and arguments.


Unit tests are faster than component tests.


Yes, mokas usually execute faster than real code. However, they hide some kinds of errors, which is why more tests have to be written. If the framework is not lazy and does a lot of unnecessary work to raise the component tree (such as web-components nailed to the DOM or TestBed in Angular, which creates everything in the world during initialization), then the tests will slow down significantly, but not so fatally. If the framework does not render until it is asked about it and the components are created until it is needed (such as $ mol_view ), component tests pass no slower than unit tests.


It’s hard to locate a bug with component tests


Yes, if they are executed in random order, then an error in the logic can drop a bunch of tests, from which it may not be clear where to start digging. Unfortunately, this is a common anti-pattern - to find all files with a given extension and execute them in random order, they say the tests are independent of each other. And this is true for unit tests.


However, performing component tests makes sense in order from less dependent components to more dependent ones. Then the first fallen test will show the source of the problem. The rest of the tests can usually not be performed anymore, which saves time while passing tests. Again, in the MAM architecture, all code (production, test) is serialized in a single order. This ensures that dependency tests are executed before the dependent tests, which means that he can safely rely on the dependence working correctly. If you use other tools, think about how to use them to build tests in the correct order.


No templates to test


You need to test the logic. A rare template engine ( mustache , view.tree ) prohibits embedding logic in templates, which means that they also need to be tested. Often unit tests are not suitable for this ( enzyme as a rare exception), so you still have to resort to component ones.


Tests must follow the Given / When / Then pattern.


Yes, sometimes in the test scenario you can highlight these steps, but you should not suck them out of your finger when they are not there. Often a script has a simpler (for example, only Then block) or complex (Given / Check / When / Then) structure. A few examples:


Pure functions often only have a Then block:


console.assert( Math.pow( 2 , 3 ) === 8 ) // Then

No less often, the action (When) consists precisely in preparing the state (Given):


component.setState({ name : 'Jin' }) // Given/When
console.assert( component.greeting === 'Hello, Jin!' ) // Then

And it happens that the check is not needed, because the fact of successful code execution is sufficient:


ensurePerson({ name : 'Jin' , age : 33 })

Similar code is completely meaningless:


const component = new MyComponent // Given
expect( component ).toBeTruthy() // Then

Just like a test that never crashes, it doesn’t test anything. So assert, which never threw an exception - does not check anything.


There should be only one assert in the right test


It is often necessary to check whether we have prepared the state correctly by verification in the middle:


wizard.nextStep().nextStep() // Given
console.assert( wizard.passport.isVisible === false ) // Check
wizard.toggleRegistration() // When
console.assert( wizard.passport.isVisible === true ) // Then

It is impossible to break this test into the following two, since the second one implicitly relies on the state created by the first:


wizard.nextStep().nextStep() // When
console.assert( wizard.passport.isVisible === false ) // Then

wizard.nextStep().nextStep() // Given
wizard.toggleRegistration() // When
console.assert( wizard.passport.isVisible === true ) // Then

Imagine that the requirements have changed and now we show the registration form by default:


wizard.nextStep().nextStep() // When
console.assert( wizard.passport.isVisible === true ) // Then

Now, if toggleRegistrationimplemented in such a way that, for example, it uses its state to speed up work, then it will pass the second test, still returning true and it turns out that the first application toggleRegistrationwill not change anything in the form:


isPassportVisible = false
toggleRegistration() {
     this.passport.isVisible = this.isPassportVisible = !this.isPassportVisible
}

In the variant with additional verification of the default state, we would catch a fallen test in this case. Moreover, do not be afraid to write longer scripts if the next step is based on the state of the previous one.


wizard.nextStep().nextStep() // When
console.assert( wizard.passport.isVisible === false ) // Then
wizard.toggleRegistration() // When
console.assert( wizard.passport.isVisible === true ) // Then
wizard.toggleRegistration() // When
console.assert( wizard.passport.isVisible === false ) // Then

Usually, an argument against this approach is the difficulty in understanding which assertion has fallen. But wait, no one forces you to use a testing tool that does not provide comprehensive information about where the test fell. A good tool (for example, $ mol_test ) will helpfully even stop the debugger in this place, allowing you to immediately start investigating the problem.


To summarize, we can recommend writing tests not according to the "Given / When / Then" pattern, but as a small adventure, starting from an absolute void and through a certain number of actions, going through a number of states, which we check.


Also popular now: