August 23, 2010 at 11:38Intuitive testing. How to find a tricky bug?
"Ad hoc testing is a commonly used term for software testing performed without planning and documentation"
I often hear from people close to software development, but far from testing phrases like: "all testers are nerds." Probably, this opinion was formed due to the fact that we, testers, have to spend most of our time doing routine work, for example, compiling and analyzing hundreds of similar test cases and then passing them, at best, before each release, or conducting an acceptance test every time a new build comes out. Such work is characterized by a clear definiteness of the testing process, i.e. took the case number 1, reproduced the procedure, put a mark on the passage, took the case number 2, etc. No creativity and by itself a step left-right = shooting.
Thus, this routine cannot but depress the healthy and young brain of the tester, who, getting a job, read on the Internet that “testers are creative people”, and here exploration comes to the rescue, namely ed hock testing, the randomness of which is explained by a free search for bugs , I want to click there, I want to come here, but I want to dabble with the database, and at first glance, here it is creativity, but it's not so simple. I think that any more or less experienced tester, before starting testing any functionality in his head, draws up a test plan and all the same test cases No. 1-100500, which in another case would be written, and only after that gets to work. This I explain the prefix "pseudo." Thus, the alternation of intuitive testing with the passage of test cases allows you to relieve stress from the brain.
Each tester should at least know his project. With the advent of a new tester, the question arises: "How to teach him to know the project?". In some companies, testers are immediately thrown into battle - they give a couple of hundred cases for execution. Indeed, why teach, if everything is written in the procedures. Thus, the company acquires the “monkey” for 3-5 months, until it passes most of the test kits. After that, the tester knows the project well enough to perform other functions besides playing steps. Let's imagine that the same tester instead of going through the cases, for a week (40-50 hours) will have to randomly search for bugs where he wants, while reading specs and dzdoks about objects of interest to him. Within a week, our tester will know 70% of what he would know by passing cases 3-5 months.
For example, a player writes that his client fell after studying the Habrahabr spell. The simplest reproduction procedure will look like this:
1. go into the tower of magic
2. learn the spell “Habrahabr”
Expected result: the client fell with a crash.
Actual result: the spell is learned.
What should I do if the bug did not play, but we trust the player? You can ask to remember everything that he did the last 15 minutes before the fall. 80% that he will not remember. Then only ed hock remains. Under the motto: “play at all costs” we open the client and storm the brain with all sorts of user behavior algorithms. Total: after 2-100 hours, the bug is reproduced, and the player’s ears burn in proportion to the number of hours spent. And the bug turned out to be simple: a stable client crash on computers where Skype is installed, while receiving a message in icq6.
Advantages of intuitive testing:
1. “Pseudo-chaotic” testing
I often hear from people close to software development, but far from testing phrases like: "all testers are nerds." Probably, this opinion was formed due to the fact that we, testers, have to spend most of our time doing routine work, for example, compiling and analyzing hundreds of similar test cases and then passing them, at best, before each release, or conducting an acceptance test every time a new build comes out. Such work is characterized by a clear definiteness of the testing process, i.e. took the case number 1, reproduced the procedure, put a mark on the passage, took the case number 2, etc. No creativity and by itself a step left-right = shooting.
Thus, this routine cannot but depress the healthy and young brain of the tester, who, getting a job, read on the Internet that “testers are creative people”, and here exploration comes to the rescue, namely ed hock testing, the randomness of which is explained by a free search for bugs , I want to click there, I want to come here, but I want to dabble with the database, and at first glance, here it is creativity, but it's not so simple. I think that any more or less experienced tester, before starting testing any functionality in his head, draws up a test plan and all the same test cases No. 1-100500, which in another case would be written, and only after that gets to work. This I explain the prefix "pseudo." Thus, the alternation of intuitive testing with the passage of test cases allows you to relieve stress from the brain.
2. Forced training of new employees
Each tester should at least know his project. With the advent of a new tester, the question arises: "How to teach him to know the project?". In some companies, testers are immediately thrown into battle - they give a couple of hundred cases for execution. Indeed, why teach, if everything is written in the procedures. Thus, the company acquires the “monkey” for 3-5 months, until it passes most of the test kits. After that, the tester knows the project well enough to perform other functions besides playing steps. Let's imagine that the same tester instead of going through the cases, for a week (40-50 hours) will have to randomly search for bugs where he wants, while reading specs and dzdoks about objects of interest to him. Within a week, our tester will know 70% of what he would know by passing cases 3-5 months.
3. The main plus of the method is the ability to search for "tricky, insidious" bugs that are not only difficult to reproduce, but also it is not known how to reproduce them
For example, a player writes that his client fell after studying the Habrahabr spell. The simplest reproduction procedure will look like this:
1. go into the tower of magic
2. learn the spell “Habrahabr”
Expected result: the client fell with a crash.
Actual result: the spell is learned.
What should I do if the bug did not play, but we trust the player? You can ask to remember everything that he did the last 15 minutes before the fall. 80% that he will not remember. Then only ed hock remains. Under the motto: “play at all costs” we open the client and storm the brain with all sorts of user behavior algorithms. Total: after 2-100 hours, the bug is reproduced, and the player’s ears burn in proportion to the number of hours spent. And the bug turned out to be simple: a stable client crash on computers where Skype is installed, while receiving a message in icq6.