Test group in a Scrum project

We have four Scrum teams in our department. Sprints, timeboxing, rallies, demos, Product Owner, customer, etc. Over the years, a list of core values ​​has been formulated:

• team spirit and atmosphere in teams - the convenience of processes is more important than the rules and habits that have developed over the years;
• routine automation is more important than mass crushing;
• quick decision making is more important than consultations, boards, long meetings;
• trust in people, the ability to learn (and make mistakes!) is more important than centralization of management;
• openness for everyone (from top management to a specific project participant) and involvement is more important than momentary saving of time and appearance of consent;
• democracy in the discussion of issues, but dictatorship in the implementation of decisions;
• the necessary minimum of rules, but exactingness in their implementation;
• when everyone thinks the same way - no one thinks;
• “Man said - man did”, everyone is responsible for their decisions.

Everything is fine, but for several years we had thoughts on how to organize testing. We experimented a lot before we came to the current state of things.

At first we tried to test by the developers themselves. So that developers do not just write unit tests, but fully work as testers - check complex cases, test with their hands. Frankly speaking, testers from developers are not very good. Checking your own code is a torment: "It works anyway, I just wrote it."
We tried to include testers in development teams. This brings together developers and testers, makes them understand each other's problems. Sometimes it is so close that the tester becomes another developer on the team, and the developer, as I said above, is already a bad tester. But there is a positive point - developers are starting to think about how to test the product.
We tried to attract testing teams external to the project (there is a separate testing department that is not involved in the project). For a long time they explained to the members of the external team what and how, they deployed the infrastructure, for a long time they explained what we needed from them. We spent a lot of time, but did not get much profit.
As a result, the idea of ​​what we, in fact, need to organize testing appeared.

Permanent Testing Team

What we needed was a team — people who worked together, who knew each other, trusted each other, and had similar knowledge and skills. A team that, in addition to testing itself, could develop technology and develop itself. Now we have such a dedicated team of 7 people.
Initially, they wanted to get a team of generalists who can do everything. But the station wagon, as a rule, knows vast areas, but superficially. And I wanted to dive into the area was great. As a result, this idea was abandoned and two areas were identified: functional and automated testing.
Functional testing is basic for all testers. This includes the allocation of requirements, and the design of tests, and their immediate implementation. Functional testing, in turn, is divided into testing basic functionality and testing applied logic.
Automated testing involves the automation of manual tests, the organization of specialized types of testing.
Testers who are assigned to any one area can dive deep there, this improves the quality of testing.

Maximum focus on automation

Automatic deployment, automatic test run, simulation of real user actions, automatic fixing of results, reasons for failed tests, environment.
Now we have about 800 GUI tests that cover the lion's share of the system’s functionality. They are launched without any participation of the tester. The environment rises automatically, the data necessary for testing is automatically created, tests are automatically run.


Auto test run table
(oops, it looks like someone has failed the last run)

Tests work through the API and UI. The API is used to prepare some kind of test data, and testing through the UI allows you to check real user scenarios.
In addition to the tests themselves, we automatically remove non-functional indicators of the system: the time of operations, the number of requests between the server and the client, traffic. The measured characteristics are available to everyone on the project in the form of graphs in dynamics.


Traffic and lead time graphs

Trust in Testing

At any time, we can ask the testing team what condition the product is in. A testing team based on a run of automatic tests, express testing can quickly answer whether it is possible to put a product into production, and if not, why.
We trust this opinion. We know that if the testing team gave the go-ahead, then we will not receive critical errors in production. We know that the team forbids us to release the version for a reason, but because there are reasons for that.

Conducting specialized testing

First of all, we are talking about load testing. All other species are somehow covered by it. Tools for such testing are quite expensive. We wrote our tool to conduct various types of special testing.
The main task of the utility is to simulate the work of many users in the system. You can write various scenarios for users with minimal time and effort.

Utility to simulate user actions

Testing for completeness, functional convenience

Testers cannot verify that the idea itself, the product idea is correct, that it was necessary to do just that and not otherwise. We close this problem with corridor testing, but this is a separate topic of a separate article ...
However, testers may well verify that the functionality is convenient to use. It often happens that formally the functional requirements were met, but the user is simply tormented by the use of the product.
Convenience in this case is not only the user interface. This is both an external API and the ability to quickly and efficiently implement the tasks provided by the system.

As close to developers as possible

The convergence of testers and developers allows you to get the test result earlier. If the bugs reach the developer in a week, then he again needs to dive into the code, dig into it again, and if in a day, it’s easier to remember what you wrote yesterday.
The developers are trying to give out features in parts that can be tested directly during the sprint, tell what is now ready and what is still in work. This allows you to quickly start testing and find problems on still crude functionality.

Graphs of the duration of development and testing;
the second option reduces the time by 1/4.

The idea of ​​a dedicated testing team fits well with the basic principles of our work. The team has been working on projects for 3 years and has shown its effectiveness.
We found the option seems optimal. But maybe you can tell us how you organized testing in your Scrum projects, do you have a dedicated department or testing group, if not, why did you refuse?