Disclaimer

Reading a book on automated testing, I found a rather interesting description of some of the ways that are used to automate the process. Since the book is written in English, I present my free translation of one part. When reading a book, he very often recalled his path to automation. This article discusses something that cannot be automated in any case.



Arbitrary, spontaneous (I think this is a more accurate translation of Ad hoc) testing means just sitting in front of a computer and trying out different things. A tester may or may not have a test plan or checklist. The tester thinks that it would be a test and just clicks here and there, trying different scenarios, values ​​and thinking "What will happen if you do like this." All the ideas and actions of the tester are not documented and are more spontaneous. And some scenarios cannot subsequently be reproduced.
This state of affairs usually happens when a software development project is launched late, little time is devoted to test planning. For such a situation, a lack of specifications is typical, requirements are still under development and constantly changing, and PM says: “There is no time for writing, just test”. Work in such a project is not very pleasant and deserves sympathy.

Often, automation of this type of testing occurs as follows:

1. Think what to do, what to test
2. We think over specific inputs.
3. Enter the data you just invented.
4. We check that the program works correctly, observing the answers that appear on the screen

It is hard to come up with the many benefits of spontaneous testing. The only argument that is mentioned is that it should save time (we do not waste time planning, designing tests, we just start testing). This is an analogue of the syndrome “Why is it not programming yet?”, Which characterizes the simplest software development. This approach is always more expensive for long-term development (both in development and in testing). a lot more mistakes will be made, and they will cost more to solve them.

Cons of this type of testing include:

1. Many parts that need to be checked can simply be skipped.
2. Some parts can be inspected more than necessary.
3. Tests are not repeatable, error resolution cannot be reliably verified (in some cases, errors may simply not be reproduced)
4. This is usually inefficient and unproductive.

Automation of such testing depends on the decision of the tester what to check. This means that the tester must have the necessary knowledge to make good tests. Otherwise, it depends on the individual design and implementation of the tests without any independent verification of the quality of the tests.