Once again about the unbearable ease of testing
We continue the conversation.
In my recent post, I gave several examples of how a tester can sometimes miss a defect that he practically had in his hands. By thoughtlessness, inexperience, naivety, fatigue, in a hurry - it does not matter. Itself. The programmer and the general development culture can, of course, help - they say that this is not a bug, but nevertheless, in the cases described, their role is auxiliary.
mtikhon in his article “An Easy Way to Pass Testing”perfectly supplemented that list with “external” problems affecting the test result. External - in the sense that they do not arise in the testing department, but in other units, and even more often - somewhere at the junctions of units, with the interaction of the departments. (I understand that a special department is not always formally allocated for testing. But this is a cosmetic difference, it doesn’t change the essence: here it’s more about separation of roles)
mtikhon 'slightly criticized in the comments that the list of problems was stated, but there was no easy way to get around them . He, in turn, has rightly noted that "the methods usually vary greatly." It is on this thought that I want to trample a little more in detail.
Perhaps I’ll go straight to the same points.
Actually, the solution is outlined in the title itself: find a way to notify employees about important changes in time, and the life of the project will become easier. I would even add: not only a tester, but also a programmer, and an analyst, and everyone else - they should also be in the know, yes. Especially when it comes to a distributed project, in which the work of one team significantly affects the work of another.
The first problem is to find a working way for such alerts.
On the one hand, all the information is already there - everything is recorded in the task tracker. In addition, the version control system also sends messages about changes - the most accurate and detailed. On the other hand, it’s clear, in general, that a bunch of junk messages go through the task tracker, and no one ever reads anything that is not related to his work. And the messages from the version control system are, well, all too detailed and atomic, and even in a not entirely human language. Finally, several related tasks and commits can correspond to one change that is interesting to all.
Okay, let's say we agreed when and what changes (in business logic, architecture, etc.) to notify, and even somehow decided how much detail these changes are described in the notification itself (who will be interested in the details - will specify).
The second problem is how to deliver these notifications.
You can get up and say it out loud. You can write a letter to everyone. You can put it on a wiki or some other internal resource. You can still come up with something exotic. Ideally, of course, it would be immediately put in the right part of the brain to the participants in the process, but so far it is impossible. In general, it’s worth a couple of minutes to think about how to be. There is no need to think long, it is better to start at least somehow; then you can adjust. Moreover, there is also a third problem.
As they say, we can bring a donkey to the river, but even Allah will not force him to drink water. That is, no matter how they inform me about the changes, there is always the opportunity not to know about them: to leave the room, not to read this spam, especially arriving from another city, etc.
That is, you need to somehow show the recipients the benefits of reading these messages. And senders, by the way, too, otherwise they will forget to inform.
In general, there are many opportunities to under-implement such a practice. Although if it works out - the benefits are enormous, up to the rejection of crazy changes in the early stages.
Politics is such a policy. Honestly, I didn’t come across such that a person was punished for the fact that someone found additional problems in “his” piece of work; but apparently this also happens.
Perhaps I in the original article slurred. In any case, it’s not about suddenly taking on someone else's work or climbing into someone else’s monastery. It’s just that if I’m working on my task and I see a problem in the area that another city is responsible for, I usually ask if I already know about it. Especially if I know that we use different test configurations. Moreover, if I suddenly know that testing of this feature has already been completed. Especially if the problem does not affect my own work in the best way.
It is not necessary to start a bug right away - for starters, you can just ask: is it here like this - on purpose? And listen to the answer, and explain your point of view, if that. And let that side start the bug, if this is suddenly important, I do not mind. At the same time, horizontal ties will be strengthened.
By the way, an important point, emphasized by Rascko in the comments: A person should know "whose it is", if it is not his. This helps a lot: you can contact a person directly, and not write to a general mailing list or contact through a hierarchy of managers.
It is unpleasant, very, I know from my own experience. On the one hand, it is our business to report on the problem, on the other, it’s true that the work is wasted.
Here is the question: who redirects and why? If a programmer is one thing, if a project manager is another (or several other things, the manager may have a lot of reasons why this defect is now uncritical; there was a recent remark on this topic in the article “Testing: from dirt to riches” )
And the solutions may be different. You can agree that the decision, whether or not to fix it, should be made by the project manager for all defects, or, for example, the analyst responsible for the feature. You can simply mask the problem and make a suggestion to a specific programmer. It’s possible to achieve a clear signal that we don’t get such specific defects (usability, productivity, localization) yet, because they won’t be fixed anyway - and this is not necessarily a disaster; Perhaps this is the best solution at the moment, allowing you to focus efforts. It is only necessary to determine to what point such an agreement is valid and explicitly wave it back when the deadline expires.
What to choose? Depends on the specific situation. This is where the most difficult part begins - to identify the problem and find out what kind of situation we have.
The test environment is important, there is nothing to argue with again. And the decision, in general, is correct: to look for a middle ground between the ideal and gross reality.
I just want to emphasize that the testers themselves should speak first of all about the shortcomings of the test environment and how to fix them. Because the rest do not know anything about this environment, especially the bosses. Until they came and said that there is a problem with the iron, there is no problem, and no one will solve it in any way.
They came once and calmed down - they will forget or ignore it with a high degree of probability.
Only regular reminders will deprive the leadership of peace and make you believe that the problem is not far-fetched, and also has a solution. Well and still distinct examples and figures in hands.
Passing part of the testing out to customers / users (alpha, beta) is an option, but there is enough underwater rake for a separate article. But you can try.
An eternally relevant problem is how to evaluate the work of testers, and even those working on tight deadlines. And programmers, by the way, too.
Either look at the number of bugs, or at criticality, or at user reviews, or just look at the peephole. There are many options in literature and life.
If we talk about the result / terms of work on the product - evaluate the work of programmers and testers, and the rest of the team, too, together. We worked on the product all together, the result is also general.
If we are talking about evaluating a specific tester, then everything sounds simple. We give him some tasks to fulfill. So we agree on the shore that from you, they say, this and that and that is expected. Performance criteria - such here. Timing ... Well, how much do you need? Ok, come on. Report problems, huh? And then - we look at what he did and evaluate.
In practice, this is getting a lot more complicated, especially from unaccustomed. Well, the point is that - these problems are not easily solved.
Returning to the deadlines: you see that the deadlines are unrealistic? Talk about it. To test everything we need, we need so much time. In the time that we have, we can test this much. Let's set priorities in order to have time to test the more important.
Here, in general, everything is about the same as in paragraph 3. Or am I missing something?
The problems are well described. In this case, the decision is so strongly tied to a specific project / product that it is difficult to give any general recommendations.
Only one thing: automation and manual testing. These concepts should not be at war; rather, the opposite is to complement and help each other. Moreover, automation is not necessarily about direct product testing. We can automate some actions / preparations for manual tests and perform the latter better and more efficiently. But by hand.
And here everything is again in the hands of the tester.
If the testers in the company are highly valued, and they are engaged in nonsense, then very soon they will be at the level of a cleaner.
If testers bring real benefits to the project and its participants, credibility will grow.
Of course, the upward movement is longer and more difficult to fall from the throne, but this is the case everywhere.
As for the set of commands from the “garbage” of programmers, show that such people are not suitable for the complex tasks facing the testing department. Prove that these tasks are complex and require high qualifications. Explain what the project is losing by giving up good staff.
Difficult.
In my recent post, I gave several examples of how a tester can sometimes miss a defect that he practically had in his hands. By thoughtlessness, inexperience, naivety, fatigue, in a hurry - it does not matter. Itself. The programmer and the general development culture can, of course, help - they say that this is not a bug, but nevertheless, in the cases described, their role is auxiliary.
mtikhon in his article “An Easy Way to Pass Testing”perfectly supplemented that list with “external” problems affecting the test result. External - in the sense that they do not arise in the testing department, but in other units, and even more often - somewhere at the junctions of units, with the interaction of the departments. (I understand that a special department is not always formally allocated for testing. But this is a cosmetic difference, it doesn’t change the essence: here it’s more about separation of roles)
mtikhon 'slightly criticized in the comments that the list of problems was stated, but there was no easy way to get around them . He, in turn, has rightly noted that "the methods usually vary greatly." It is on this thought that I want to trample a little more in detail.
Perhaps I’ll go straight to the same points.
1. The tester must be aware of changes in the product
Actually, the solution is outlined in the title itself: find a way to notify employees about important changes in time, and the life of the project will become easier. I would even add: not only a tester, but also a programmer, and an analyst, and everyone else - they should also be in the know, yes. Especially when it comes to a distributed project, in which the work of one team significantly affects the work of another.
The first problem is to find a working way for such alerts.
On the one hand, all the information is already there - everything is recorded in the task tracker. In addition, the version control system also sends messages about changes - the most accurate and detailed. On the other hand, it’s clear, in general, that a bunch of junk messages go through the task tracker, and no one ever reads anything that is not related to his work. And the messages from the version control system are, well, all too detailed and atomic, and even in a not entirely human language. Finally, several related tasks and commits can correspond to one change that is interesting to all.
Okay, let's say we agreed when and what changes (in business logic, architecture, etc.) to notify, and even somehow decided how much detail these changes are described in the notification itself (who will be interested in the details - will specify).
The second problem is how to deliver these notifications.
You can get up and say it out loud. You can write a letter to everyone. You can put it on a wiki or some other internal resource. You can still come up with something exotic. Ideally, of course, it would be immediately put in the right part of the brain to the participants in the process, but so far it is impossible. In general, it’s worth a couple of minutes to think about how to be. There is no need to think long, it is better to start at least somehow; then you can adjust. Moreover, there is also a third problem.
As they say, we can bring a donkey to the river, but even Allah will not force him to drink water. That is, no matter how they inform me about the changes, there is always the opportunity not to know about them: to leave the room, not to read this spam, especially arriving from another city, etc.
That is, you need to somehow show the recipients the benefits of reading these messages. And senders, by the way, too, otherwise they will forget to inform.
In general, there are many opportunities to under-implement such a practice. Although if it works out - the benefits are enormous, up to the rejection of crazy changes in the early stages.
2. Foreign bugs
Politics is such a policy. Honestly, I didn’t come across such that a person was punished for the fact that someone found additional problems in “his” piece of work; but apparently this also happens.
Perhaps I in the original article slurred. In any case, it’s not about suddenly taking on someone else's work or climbing into someone else’s monastery. It’s just that if I’m working on my task and I see a problem in the area that another city is responsible for, I usually ask if I already know about it. Especially if I know that we use different test configurations. Moreover, if I suddenly know that testing of this feature has already been completed. Especially if the problem does not affect my own work in the best way.
It is not necessary to start a bug right away - for starters, you can just ask: is it here like this - on purpose? And listen to the answer, and explain your point of view, if that. And let that side start the bug, if this is suddenly important, I do not mind. At the same time, horizontal ties will be strengthened.
By the way, an important point, emphasized by Rascko in the comments: A person should know "whose it is", if it is not his. This helps a lot: you can contact a person directly, and not write to a general mailing list or contact through a hierarchy of managers.
3. Minor bugs and interface bugs / imruments flying in the redject
It is unpleasant, very, I know from my own experience. On the one hand, it is our business to report on the problem, on the other, it’s true that the work is wasted.
Here is the question: who redirects and why? If a programmer is one thing, if a project manager is another (or several other things, the manager may have a lot of reasons why this defect is now uncritical; there was a recent remark on this topic in the article “Testing: from dirt to riches” )
And the solutions may be different. You can agree that the decision, whether or not to fix it, should be made by the project manager for all defects, or, for example, the analyst responsible for the feature. You can simply mask the problem and make a suggestion to a specific programmer. It’s possible to achieve a clear signal that we don’t get such specific defects (usability, productivity, localization) yet, because they won’t be fixed anyway - and this is not necessarily a disaster; Perhaps this is the best solution at the moment, allowing you to focus efforts. It is only necessary to determine to what point such an agreement is valid and explicitly wave it back when the deadline expires.
What to choose? Depends on the specific situation. This is where the most difficult part begins - to identify the problem and find out what kind of situation we have.
4. Laboratory
The test environment is important, there is nothing to argue with again. And the decision, in general, is correct: to look for a middle ground between the ideal and gross reality.
I just want to emphasize that the testers themselves should speak first of all about the shortcomings of the test environment and how to fix them. Because the rest do not know anything about this environment, especially the bosses. Until they came and said that there is a problem with the iron, there is no problem, and no one will solve it in any way.
They came once and calmed down - they will forget or ignore it with a high degree of probability.
Only regular reminders will deprive the leadership of peace and make you believe that the problem is not far-fetched, and also has a solution. Well and still distinct examples and figures in hands.
Passing part of the testing out to customers / users (alpha, beta) is an option, but there is enough underwater rake for a separate article. But you can try.
5. Timing and reporting
An eternally relevant problem is how to evaluate the work of testers, and even those working on tight deadlines. And programmers, by the way, too.
Either look at the number of bugs, or at criticality, or at user reviews, or just look at the peephole. There are many options in literature and life.
If we talk about the result / terms of work on the product - evaluate the work of programmers and testers, and the rest of the team, too, together. We worked on the product all together, the result is also general.
If we are talking about evaluating a specific tester, then everything sounds simple. We give him some tasks to fulfill. So we agree on the shore that from you, they say, this and that and that is expected. Performance criteria - such here. Timing ... Well, how much do you need? Ok, come on. Report problems, huh? And then - we look at what he did and evaluate.
In practice, this is getting a lot more complicated, especially from unaccustomed. Well, the point is that - these problems are not easily solved.
Returning to the deadlines: you see that the deadlines are unrealistic? Talk about it. To test everything we need, we need so much time. In the time that we have, we can test this much. Let's set priorities in order to have time to test the more important.
6. What happens with bugs of the form: not reproducible, observed and etc?
Here, in general, everything is about the same as in paragraph 3. Or am I missing something?
7. Automation vs manual testing
The problems are well described. In this case, the decision is so strongly tied to a specific project / product that it is difficult to give any general recommendations.
Only one thing: automation and manual testing. These concepts should not be at war; rather, the opposite is to complement and help each other. Moreover, automation is not necessarily about direct product testing. We can automate some actions / preparations for manual tests and perform the latter better and more efficiently. But by hand.
8. The position of the tester in the overall structure of the company
And here everything is again in the hands of the tester.
If the testers in the company are highly valued, and they are engaged in nonsense, then very soon they will be at the level of a cleaner.
If testers bring real benefits to the project and its participants, credibility will grow.
Of course, the upward movement is longer and more difficult to fall from the throne, but this is the case everywhere.
As for the set of commands from the “garbage” of programmers, show that such people are not suitable for the complex tasks facing the testing department. Prove that these tasks are complex and require high qualifications. Explain what the project is losing by giving up good staff.
Difficult.