Why study the mistakes of others?
Six months ago, we launched the Panbagon project , a collective blog designed to collect bugs and discuss them. As I wrote in the announcement published on the Habré , I have no goal of making either a public bug tracker or a board of shame. I wanted to create a kind of incubator where ideas could be formed from garbage that could be useful for finding bugs similar to those described in Panbagon. Therefore, I called not only to describe the bug itself, but also to state the thoughts that you had about this.
However, I am periodically asked a sacramental question in the headline - why study the mistakes of others? So I decided to explain why I find this extremely useful for testers.
However, I'll start a little from afar - with a story about morel morels.
Recently, Michael Bolton published a series of remarkable articles on StickyMinds.com in which he examines individual fragments of the history of the development of various fields of human knowledge, and discusses what lessons we can learn for our testing classes. I want to partially retell one of these articles, “Food for Thought,” it seems to me that the examples described there will allow me to better outline my intent.
In his article, Bolton mentions Michael Pollan’s book, The Omnivore's Dilemma: A Natural History of Four Meals. This book is devoted to a comprehensive review of food and its relationship with humans - how we produce it, how we get it, and how we eat it.
Among other things, Pollan talks about a very rare mushroom "fire morel", which grows on fires in the first spring after a forest fire. Not only is this morel rare, it is also quite difficult to detect - it is small in size and very similar to a little head that are scattered literally everywhere in the place where it grows. Specialists use special techniques to search for these mushrooms. One way is to look along the ground at a low angle, this allows you to notice protruding hats; another way is to look for dogwood near the bushes, which prefers the soil at the same temperature as morels; another way is to look at the same altitude where mushrooms were already found this week. “I understand why these theories are needed,” writes Pollan, “Theories tell us when to focus our attention, scrupulously examining the forest litter with his eyes, and when attention can be weakened. For the “morel hunter,” focused viewing is a highly accurate search tool, but it is a limited resource, and theories that accumulate knowledge accumulated in the past help to spend it efficiently. ”
During testing, we act in approximately the same way - we try to look for errors of a certain type, creating specific conditions in which they can occur, look at the system in the right places and “at the right angle”, submit specific data to the input, and so on, we know and we use many different heuristics in which the knowledge accumulated in the past is concentrated.
But here is what Pollan writes further: while hunting for morels, you should “be ready to discard all past theories and use something that seems to work in this particular place and at this particular time. Mushrooms behave unpredictably, and theories only work until they come across a new mystery. ”
But with bugs everything is exactly the same! All the techniques, models, strategies and heuristics that we use sometimes are beneficial, but in certain situations, on the contrary, they can lead us away. And you can never understand in advance whether this or that method will work or not, and which of them will be the most effective. Because bugs are unpredictable, like morel morels of fire (can be called bugs morels, eh?)
However, Pollan further writes that one can still learn to search for morels, for this we use the property of the human brain that is familiar to mushroom pickers, which psychologists call the "pop-out effect" - "... when we fix in our mind some visual characteristic of the object that we we want to notice or detect (it can be a color or shape or something else), the object seems to stand out, pop-out from the environment.Probably, people acquired such a wonderful ability in the process of evolution, collecting, it helped to find food .
Can we use this “built-in” skill to find bugs during testing? Apparently, to some extent, we can. Yes. Moreover, we can develop it through training. Examining and discussing bugs helps to “look up” for bug detection by fixing certain patterns in the brain, consisting of sets of signs of the presence of bugs of one kind or another.
We will return now to Panbagon.
Just describing a bug is the same as showing the morel you found. To the consumer, that is, the consumer of these morels, this result undoubtedly seems important. Your customers want to receive exactly that from you. But if you discuss the results of your hunt with other mushroom testers, the more important and interesting thing for them is not morel like food itself, but how you found it. Of course, the size, shape and color will be carefully studied, but they also would like to know what kind of soil there was, what grew nearby, whether the hat stuck out high, whether the sun shone brightly, it shone in your back or in the face and other signs of the environment.
If you found a dozen morels, analyzed related signs, formulated a theory, and then, using it found a dozen more, you can immediately state this theory. Perhaps it will be useful and someone else can use it to find several mushrooms as well.
But if your theory has not yet developed, you can at least just tell about all the signs that you thought were important, about what you thought and what you felt at that moment. Why did you notice this bug? Were there other bugs nearby, and if there were - why did you decide to tell you about this particular one? Do you often notice such bugs? What thoughts or emotions accompany the moment of "grasping" a bug? Do you have confidence that next time if a similar bug arises in a similar situation, you will also find it?
Let us study bugs together in their habitats, try to identify signs by which they can be found, and we hope that as a result of this there will be new theories and heuristics that will benefit some of us, and with it the honor and glory of the skilled tester.
However, I am periodically asked a sacramental question in the headline - why study the mistakes of others? So I decided to explain why I find this extremely useful for testers.
However, I'll start a little from afar - with a story about morel morels.
Recently, Michael Bolton published a series of remarkable articles on StickyMinds.com in which he examines individual fragments of the history of the development of various fields of human knowledge, and discusses what lessons we can learn for our testing classes. I want to partially retell one of these articles, “Food for Thought,” it seems to me that the examples described there will allow me to better outline my intent.
In his article, Bolton mentions Michael Pollan’s book, The Omnivore's Dilemma: A Natural History of Four Meals. This book is devoted to a comprehensive review of food and its relationship with humans - how we produce it, how we get it, and how we eat it.
Among other things, Pollan talks about a very rare mushroom "fire morel", which grows on fires in the first spring after a forest fire. Not only is this morel rare, it is also quite difficult to detect - it is small in size and very similar to a little head that are scattered literally everywhere in the place where it grows. Specialists use special techniques to search for these mushrooms. One way is to look along the ground at a low angle, this allows you to notice protruding hats; another way is to look for dogwood near the bushes, which prefers the soil at the same temperature as morels; another way is to look at the same altitude where mushrooms were already found this week. “I understand why these theories are needed,” writes Pollan, “Theories tell us when to focus our attention, scrupulously examining the forest litter with his eyes, and when attention can be weakened. For the “morel hunter,” focused viewing is a highly accurate search tool, but it is a limited resource, and theories that accumulate knowledge accumulated in the past help to spend it efficiently. ”
During testing, we act in approximately the same way - we try to look for errors of a certain type, creating specific conditions in which they can occur, look at the system in the right places and “at the right angle”, submit specific data to the input, and so on, we know and we use many different heuristics in which the knowledge accumulated in the past is concentrated.
But here is what Pollan writes further: while hunting for morels, you should “be ready to discard all past theories and use something that seems to work in this particular place and at this particular time. Mushrooms behave unpredictably, and theories only work until they come across a new mystery. ”
But with bugs everything is exactly the same! All the techniques, models, strategies and heuristics that we use sometimes are beneficial, but in certain situations, on the contrary, they can lead us away. And you can never understand in advance whether this or that method will work or not, and which of them will be the most effective. Because bugs are unpredictable, like morel morels of fire (can be called bugs morels, eh?)
However, Pollan further writes that one can still learn to search for morels, for this we use the property of the human brain that is familiar to mushroom pickers, which psychologists call the "pop-out effect" - "... when we fix in our mind some visual characteristic of the object that we we want to notice or detect (it can be a color or shape or something else), the object seems to stand out, pop-out from the environment.Probably, people acquired such a wonderful ability in the process of evolution, collecting, it helped to find food .
Can we use this “built-in” skill to find bugs during testing? Apparently, to some extent, we can. Yes. Moreover, we can develop it through training. Examining and discussing bugs helps to “look up” for bug detection by fixing certain patterns in the brain, consisting of sets of signs of the presence of bugs of one kind or another.
We will return now to Panbagon.
Just describing a bug is the same as showing the morel you found. To the consumer, that is, the consumer of these morels, this result undoubtedly seems important. Your customers want to receive exactly that from you. But if you discuss the results of your hunt with other mushroom testers, the more important and interesting thing for them is not morel like food itself, but how you found it. Of course, the size, shape and color will be carefully studied, but they also would like to know what kind of soil there was, what grew nearby, whether the hat stuck out high, whether the sun shone brightly, it shone in your back or in the face and other signs of the environment.
If you found a dozen morels, analyzed related signs, formulated a theory, and then, using it found a dozen more, you can immediately state this theory. Perhaps it will be useful and someone else can use it to find several mushrooms as well.
But if your theory has not yet developed, you can at least just tell about all the signs that you thought were important, about what you thought and what you felt at that moment. Why did you notice this bug? Were there other bugs nearby, and if there were - why did you decide to tell you about this particular one? Do you often notice such bugs? What thoughts or emotions accompany the moment of "grasping" a bug? Do you have confidence that next time if a similar bug arises in a similar situation, you will also find it?
Let us study bugs together in their habitats, try to identify signs by which they can be found, and we hope that as a result of this there will be new theories and heuristics that will benefit some of us, and with it the honor and glory of the skilled tester.