Choose the appropriate bug tracking
I talked with dozens of QA engineers from different companies and each of them talked about the fact that they use different systems and tools for bug tracking. We also tried several of them, and I decided to share the solution we came to.
I will be banal. Errors appear and are detected at various stages of the development process. Therefore, you can divide the bugs into categories, depending on the time they were detected:
Two years ago, we had a dedicated team of testers who manually tested the product after integrating the code of all the teams. Up to this point, the code has been tested by developers at developer stands. Errors that the testers found were recorded in the backlog in Jira. Bugs were stored in a common backlog and moved from sprint to sprint with other tasks. Each sprint got two or three bugs and fixed them, but most remained at the bottom of the backlog:
I will give a simple example. When building reports in the date input fields, the date is substituted by default. When changing the date in the popup, you can again select today and the date input field will be cleared.
I have suspicions that over the entire life of our network, no one except testers has not reproduced this error. Such errors make up the majority of bugs that are not fixed.
With this approach, when all found bugs are entered, some of them are duplicated and most of these bugs are not repaired. Problems arise:
In the spring of 2018, we abandoned Jira and moved to Kaiten. The change of tools was caused by reasons about which Andrei Arefiev wrote in the article “Why Dodo Pizza began to use Kaiten instead of a bunch of Trello and Jira”. After moving to Kaiten, our approach to working with bugs has not changed:
We decided to experiment with the formats and created a separate board in Kaiten, on which we kept bugs and changed statuses. We simplified the establishment of a bug report in order to spend less time. When adding a card to Kaiten, testers marked developers. A notification was sent to their mail about this. We put this board on a monitor that hung in the aisle near our workplace, so that developers could see the progress and the testing process became transparent. This practice also did not take root, because the main communication channel is Slack. Our developers do not check mail often. Therefore, this solution quickly stopped working and we returned to Slack.
After a failed board experiment in Kaiten, some developers were still against the message format in Slack. And we began to think how to track bugs in slack and solve problems that prevented developers. As a result of searches, I came across an application for Slack, Workast, which helps to organize work with toddlers right in the messenger. We thought that this application will allow you to manage the process of working with bugs more flexibly. This application had its advantages: change of statuses and assignment to developers at the click of a button, completed tasks were hidden and the message did not grow to enormous size.
So the solved problems looked in the Todo application and requests to return "ants". After the trial period of the Workast application ended, we decided to abandon it. Because using this application, we came to the same thing as when we used Jira. There were some bugs that wandered from regress to regress on this list. And with each iteration, there were more of them. Perhaps it was worth finalizing the process of working with this extension, buy it and use it, but we did not.
At the end of 2018 - the beginning of 2019, a number of changes took place in our company that affected the process of working with bugs.
Firstly, we did not have a dedicated team of testers. All testers dispersed into development teams, to strengthen the competence of testing teams. Thanks to this, we began to find errors earlier, before the integration of the command code.
Secondly, we abandoned manual regression testing in favor of automatic.
Thirdly, we adopted a “zero bug policy”. In the article " #zerobugpolicy or how we fix bugs " bevzuk details how we select the bugs we fix.
Today the process of working with bugs is as follows:
In short, we basically abandoned the bug tracking system . Using this approach to working with bugs, we solved several pains:
I do not want to say that tracking bugs is useless. The bugs that we take to work are tracked like any other tasks. But a bug tracking system is not a required testing attribute. It does not need to be used only because most companies use it and so is customary in the industry. You need to “think with your head” and try on tools for your processes and needs. It’s ideal for us to work without a bug tracking system now. For half a year of such work, we never thought about returning to it and again getting all the bugs there.
And how is the process of working with bugs arranged in your company?
Intro
I will be banal. Errors appear and are detected at various stages of the development process. Therefore, you can divide the bugs into categories, depending on the time they were detected:
- Deficiencies . These are the mistakes that the developers made while sawing new functionality. Such errors are found during research or acceptance testing of new features on development stands of teams.
- Bugs in regression . These are defects that find manual regression tests or automatic UI and API tests on the stand for code integration.
- Bugs with a sale . These are problems that employees or customers found and contacted technical support.
Where did we start, or Jira
Two years ago, we had a dedicated team of testers who manually tested the product after integrating the code of all the teams. Up to this point, the code has been tested by developers at developer stands. Errors that the testers found were recorded in the backlog in Jira. Bugs were stored in a common backlog and moved from sprint to sprint with other tasks. Each sprint got two or three bugs and fixed them, but most remained at the bottom of the backlog:
- One of the reasons why bugs are accumulated in the backlog is that they do not interfere with users. Such bugs have a low priority and will not be fixed.
- Also, if the company does not have clear and understandable rules for establishing bugs, testers can add the same problem several times because they could not find the already added bug report in this list.
- Another reason may be that inexperienced testers are involved in the project. It is a mistake for beginning testers to enter in the bug tracking all the bugs found during operation. Inexperienced testers consider the purpose of testing is to search for bugs, rather than providing information about the quality of the product and preventing the appearance of defects.
I will give a simple example. When building reports in the date input fields, the date is substituted by default. When changing the date in the popup, you can again select today and the date input field will be cleared.
I have suspicions that over the entire life of our network, no one except testers has not reproduced this error. Such errors make up the majority of bugs that are not fixed.
With this approach, when all found bugs are entered, some of them are duplicated and most of these bugs are not repaired. Problems arise:
- Testers are demotivated, because the errors that they find are not fixed by the developers. One gets the feeling that the work does not make sense.
- It’s difficult for the product owner to manage a backlog with a lot of bugs.
Goodbye Jira, Long live Kaiten
In the spring of 2018, we abandoned Jira and moved to Kaiten. The change of tools was caused by reasons about which Andrei Arefiev wrote in the article “Why Dodo Pizza began to use Kaiten instead of a bunch of Trello and Jira”. After moving to Kaiten, our approach to working with bugs has not changed:
- Imperfections were recorded on the command boards and the developers independently decided to repair them or not.
- The bugs that were found in the regression (it was carried out by a dedicated team of testers) were repaired in the release branch and did not release the code until all the problems were fixed. We decided that it would be more logical to keep and collect information about these problems in the testers channel in Slack. The testers wrote a message that contained a legend, a list of bugs with logs and the names of the developers who took the task to work. With the help of emoji, they changed the status, and in trades they discussed, applied screenshots, synchronized. This format suited testers. Some developers did not like this method, because in the chat there was another correspondence in parallel and this message went up and was not visible. We fixed it, but it did not greatly simplify life.
- The bugs that were found on the prod were recorded in the backlog, Product Owner, prioritized and selected those that we will repair.
Experiment time or not
We decided to experiment with the formats and created a separate board in Kaiten, on which we kept bugs and changed statuses. We simplified the establishment of a bug report in order to spend less time. When adding a card to Kaiten, testers marked developers. A notification was sent to their mail about this. We put this board on a monitor that hung in the aisle near our workplace, so that developers could see the progress and the testing process became transparent. This practice also did not take root, because the main communication channel is Slack. Our developers do not check mail often. Therefore, this solution quickly stopped working and we returned to Slack.
Bring the ants back
After a failed board experiment in Kaiten, some developers were still against the message format in Slack. And we began to think how to track bugs in slack and solve problems that prevented developers. As a result of searches, I came across an application for Slack, Workast, which helps to organize work with toddlers right in the messenger. We thought that this application will allow you to manage the process of working with bugs more flexibly. This application had its advantages: change of statuses and assignment to developers at the click of a button, completed tasks were hidden and the message did not grow to enormous size.
So the solved problems looked in the Todo application and requests to return "ants". After the trial period of the Workast application ended, we decided to abandon it. Because using this application, we came to the same thing as when we used Jira. There were some bugs that wandered from regress to regress on this list. And with each iteration, there were more of them. Perhaps it was worth finalizing the process of working with this extension, buy it and use it, but we did not.
Our perfect way to deal with bugs now
At the end of 2018 - the beginning of 2019, a number of changes took place in our company that affected the process of working with bugs.
Firstly, we did not have a dedicated team of testers. All testers dispersed into development teams, to strengthen the competence of testing teams. Thanks to this, we began to find errors earlier, before the integration of the command code.
Secondly, we abandoned manual regression testing in favor of automatic.
Thirdly, we adopted a “zero bug policy”. In the article " #zerobugpolicy or how we fix bugs " bevzuk details how we select the bugs we fix.
Today the process of working with bugs is as follows:
- Deficiencies . Testers report the problem to analysts or product managers. They walk, show, reproduce, explain how it will affect customers and decide whether it needs to be repaired before release, or can be repaired later, or maybe it’s not worth repairing at all.
- Bugs in the regression that auto-tests found are repaired by the team that touched this part of the system and we will not release this code until we solve the problem. Testers fix these bugs in an arbitrary format on the release channel in Slack.
- Bugs with a sale . Such bugs go directly to the task owners. Analysts run errors on the Priority Matrix of the bug and add it to the backlog or fix it on their own, accumulating statistics of hits on this issue.
Summary
In short, we basically abandoned the bug tracking system . Using this approach to working with bugs, we solved several pains:
- Testers are not upset because the errors that they find and trigger in bug tracking are not fixed.
- Testers do not spend time on an institution and a full description of bugs that no one will even read.
- PO is easier to manage backlog in which there is no dead load.
I do not want to say that tracking bugs is useless. The bugs that we take to work are tracked like any other tasks. But a bug tracking system is not a required testing attribute. It does not need to be used only because most companies use it and so is customary in the industry. You need to “think with your head” and try on tools for your processes and needs. It’s ideal for us to work without a bug tracking system now. For half a year of such work, we never thought about returning to it and again getting all the bugs there.
And how is the process of working with bugs arranged in your company?