Quality is the responsibility of the team. Our QA experience

    I work as a QA engineer at Miro . I’ll tell you about our experiment in transferring to developers part of the tasks of testing and transforming the role of the tester into the role of QA (Quality assurance).

    First, briefly about our development process. We have daily client releases and 3 to 5 server releases per week. The development team has 60+ people who are divided into 10 functional scrum teams.

    I work in the Integration team, whose task is to integrate our service into external products and integrate external products into our service. For example, we integrated the Jira task tracker . Jira Cards - a visual display of tasks that you can conveniently work on the board without going into Jira.



    How did the experiment begin?


    It all started with a commonplace problem. When one of the testers went to the sick-list, the team's performance was greatly reduced. The team continued to work on the tasks, but the code reached the testing stage and the task paused. As a result, the new functionality did not reach production on time.

    Departure of the tester on vacation is another story. He needs to find in advance one of the testers who is ready to take his tasks in addition to his own, to agree, to immerse in tasks. At the same time, two testers go on vacation is an impermissible luxury.

    We began to think how to solve the problem, and realized that the real problem is that the narrow neck of development processes is testing. I will show you an example of several stories.

    The first story: endless task rollback


    There is also a developer. Each has its own tasks. The developer completed one of the tasks and gave it to me for testing. Since this task has a higher priority than my current ones, I switch to it. I find bugs, I get everything in Jira and give it back to the developer for revision.

    I return to the tasks that had to be postponed. The developer switches from new tasks to the task that I returned. He corrects bugs and returns me for testing. I find bugs again and return the task again. This process can go on for a very long time.



    As a result, the total time of work on the task increases several times, followed by the increase of Time to market, and this is critical for us as a food company. There are several reasons for increasing the time spent on the task:

    1. The task is constantly shifting between development and testing.
    2. The task is idle waiting for a tester or developer.
    3. The developer and tester have to regularly switch between tasks, which requires additional time and energy.

    The second story: a growing line of tasks


    Our scrum team has several developers. They regularly submit their tasks to me for testing. A queue of tasks forms for which I undertake, based on priorities.

    So we worked for about a year, but during this time the company has grown significantly: the number of projects and developers has increased, and hence the number of tasks in testing. At the same time, I was still the only tester in our team and could not multiply my speed. As a result, more and more tasks were accumulating in the testing queue, and the process of transferring tasks between the developer and the tester increased the waiting time.

    Suddenly I got sick. Delivery of new features from our production team has completely stopped. What should a team do in such a situation? You can ask the tester for help from another team. But, with a high degree of probability, he will not be immersed in our specifics, which will negatively affect the quality and timing of both teams.

    The conclusion from both stories is the same - the teams depend too much on the tester:

    • The performance of the entire team is limited by the performance of the tester.
    • The number of developers cannot be increased without increasing the staff of testers.
    • An increase in the development speed does not make sense if all tasks fall into the queue for testing.
    • While the work of the developer is checked by the tester, the developer’s sense of responsibility for quality is reduced.
    • If there is no tester, then the process of outputting new functionality suffers.

    Let's increase the staff of testers?


    The most obvious thought is to increase the staff of testers. This will work, but only until a certain point: the number of tasks will constantly grow, and it is impossible to infinitely increase the number of testers - at some point it will become expensive and inefficient. Fedor Ovchinnikov , CEO of Dodo Pizza, wrote well on the topic of “resource thinking” problems (can’t you solve the problem? Hire another employee) .

    It is much more efficient to maintain the speed and quality of development within current resources. Therefore, we decided to launch an experiment that will help teams create functionality immediately, taking into account all risks and border situations. They called it Transform tester to QA, because it is about the transformation of one of the roles in the team: from a monkey tester who identifies errors after the developer to a QA engineer who consciously ensures quality at all stages of the development process.

    Let's improve development processes


    Objectives of the experiment:

    • To remove the team’s dependence on testers, but without loss of quality and time.
    • Improve quality assurance by QA engineers in teams.

    The first step was to change the thinking of the team. Everyone is used to the fact that the tester is responsible for the quality of the team.

    Our hypothesis was that increasing the time to prepare and test tasks would help reduce the number of iterations in working on one task. This, in turn, will allow bringing new functionality to production without loss of speed and quality level, and maybe faster and with better quality.

    Together with the team and with testers from other teams, we developed a new working scheme. For half a year, as the team works on it, we took into account the difficulties and problems that arose along the way, and today it looks like this:

    1. Presentation of the production.
    2. Technical solution and test scenario.
    3. Development and verification.
    4. Release

    Formulation of the problem


    Product Owner presents the statement of the problem to the team. The team analyzes the formulation in order to identify borderline situations from the technical and product sides. If there are questions that need to be further investigated - a separate task is set, for which time is allocated in the sprint.



    Technical solution


    The result of the analysis of the formulation of the problem is a technical solution, which is one or more developers. All relevant company employees, including QA, must be familiar with and agree with the technical solution. The technical solution describes the problem that we are solving, the functional blocks of the product that will be affected, and the proposed plan for making changes to the code.

    This solution allows you to identify a simpler, better solution based on the experience of relevant participants in the development process. Having a detailed technical description on hand - it is easier to see and avoid blocking problems, it is easier to conduct a code review.

    Here is an example of some blocks from a technical solution:
    Description of the task

    Are new entities or approaches added to the system?
    If so, they are described and explained why it is impossible to solve the problem within the framework of existing approaches.
    Is server interaction changing within the cluster? Are new cluster roles being added?

    Is the data model changing?

    For the server we are talking about objects and models.
    If the data model is complex, you can present it in the form of a UML diagram, or as a text description.

    Is the interaction between the client and server changing?

    Description of changes. If this is an API, can it be given to external users? Do not forget about error handling - i.e. indicate the correct reason.

    Test scenario


    In parallel with this, the developer or QA draws up a test scenario that takes into account all possible places where problems have occurred. If the developer does this, he will necessarily give the script to QA, which checks it for completeness and sufficiency. If the script is QA, the developer must confirm that he understands how each of its items can be completed. The team also evaluates the script for correctness.

    For compilation and storage of scripts we use HipTest.





    Development and Validation


    The developer starts writing code based on a technical solution and taking into account all the possible situations described in the test documentation. Passes a code review and checks the code against test scenarios. Produces merge in the master.

    At this stage, QA provides the necessary support to the developer. For example, it helps with playing complex cases, setting up a test environment, conducting load testing, and suggests the possible nuances of outputting big features to production.

    Release of ready functionality


    This is the final stage. Here, the team may be required to pre \ post-release actions, for example, the inclusion of new functionality for beta users.

    Documentation and Tools


    The new scheme of work required developers more immersion in the testing process. Therefore, as a QA, it was important for me to provide them with all the necessary tools and teach the basics of functional testing. Together with QA from other teams, we made a list of the necessary documentation, created the missing testing instructions and finalized the existing ones taking into account things that were not obvious to developers.

    Then we gave developers access to all testing tools and test environments and began to conduct joint testing. At first, the developers did not want to take responsibility for the test results, because nobody else checked anything for them, it was unusual for them. Each developer had only a test script, the functionality he developed and the Merge button. But gradually they got used to it.

    Experiment Results


    Six months have passed since the start of the experiment. The graph displays the statistics of the number of bugs by week in our team. Red color shows the number of all bugs found by the team, green - the number of fixed ones.



    It can be seen that the level of the red line remained at the same level, except for the burst at the beginning of the experiment. There are no more bugs, which means we have maintained the current level of quality.

    At the same time, the average time for working on tasks decreased by only 2%: before the start of the experiment, it was 12 hours 40 minutes, after - 12 hours 25 minutes. So we were able to maintain the current speed of work on tasks.

    As a result, our team no longer has a strong dependence on QA. If I get sick or go on vacation - the team will continue to work without loss in speed and quality.

    Do we consider the experiment successful, despite the fact that the indicators of time and quality remained the same? Yes, we believe, because we maintained the speed and quality of work and freed up QA time for more conscious work on product quality and development processes. For example, to conduct research testing, increase the coverage of functional tests and describe the principles and rules for the development of test documentation in all teams.

    In other teams, we sowed the seed of the experiment. For example, in one of the commands, QA now does not test what is described by the developer in the pull request, because the developer can see it on his own. In another team, QA analyzes the request changes and checks only complex and non-obvious cases.

    Things to remember before starting an experiment


    This is a complex and lengthy experiment. It is not implemented at the click of a finger, you need to carefully prepare for it and not wait for quick results.

    Be prepared for the negativity of the team. You can’t just take it and say that now the developers are testing the functionality. It is necessary to prepare them for this, develop approaches, explain the value of the experiment for the team and the product.

    The loss of speed at the start is inevitable. Time to train developers, write the missing documentation, and rebuild processes takes time, which will have to be donated to the experiment.

    There is no ready-made solution. Similar processes are implemented, for example, in Atlassian, but that doesn’t mean that you can also implement them as is. Adaptation to the culture of the company and the specifics of the teams is important.

    Also popular now: