Create a testing department

Software development is not possible without quality control, and the testing process plays a key role in this. It should be noted that testing is not the only and especially not sufficient measure to create high-quality software, but absolutely necessary.



What is testing? Simplified, this is the process of verifying that the program meets all the requirements. Even more simplified - testing is a search for errors. In this case, the program is usually considered as a “black box”, and the check is performed by multiple launches with different initial data and in different conditions.
We are convinced that only a separate subdivision can perform full-fledged testing of a software product in a company - in fact, the testing department. Shifting the functions of testers to developers, business analysts, or even managers is an inefficient way. In this article, we will describe how you can build a testing department.

Glossary

Test case - steps to achieve the expected result.
Test plan - planned and assigned test cases.
Run - execution of a test plan.
Regression test - a test of the old functionality of the program, the performance of which could be impaired after the introduction of new functions.

People

The most important thing is to find the right people. The first thing you will encounter is that we are not taught testers, and finding a person is not so simple. You have two ways: to “make a tester” one of your employees or to hire a person in your work experience.
Definitely, a person from the company already knows the features of your product, the requirements of customers and the team (this is also important). It is likely (if you are already developing) that some of the programmers still tested their code and someone did it better and more thoroughly than others?
However, although the transition from programmers to testers is possible, there is no way to be sure that your employees who like their work will agree to devote themselves completely to testing. I would suggest that they will not be delighted with such an offer at all - these are too different professions. Therefore, hiring from the outside also consider.
A good tester is not just a “test runner”, it’s the person who will use your product every day and be the most dedicated user. And having such a person on the team is worth a lot.

Time planning

Testing is a time-consuming and time-consuming process. One run of regression tests is worth it! Therefore, when testing, it is extremely important to plan time. For example, you have many projects, and the testing department is only one. There are many problems with simultaneous releases.
It’s very convenient to use a google calendar for the solution (I already wrote about the use of google services ), that is, create a calendar in which you can make planned releases, and share it with managers and testers. You can view the release dates in the project management system, for example, Redmine, but most clearly use a single calendar for planning.

Testing is accounting and control

Than to have a very “angry” test case, it is much more important to “run it” regularly and record the results. You can, of course, keep everything in mind or enter the results of the tests in a notebook, but this is not effective. Use TMS (Test Management System). They allow you to use a systematic approach to testing - when we initially sit down and describe our test plan so that it doesn’t turn out like this: we tested it here, there and, as a result, we can’t say what the state of the project / product is.
The most important thing when using TMS is the analysis of the results obtained and the identification of the dynamics of project development. We are starting to see from release to release that, for example:

• The number of bugs is increasing, and you should understand what the problem is.
• One of the testers finds more bugs than the rest. How to raise the skills of testers?
• The time to run the regression tests is growing steadily with the development of the project, but we clearly see this and can plan it.
• ... ... in general, the information for analysis is quite enough.

We use TMS Testlink. You can choose other systems. One of my colleagues described his experience and advice in an article . Take note - it will help in difficult times. In TMS, we submit all of our tests, and then review the reports.

Manual testing

Manual testing is where it all starts. Tests are performed manually, without the use of special automation tools (with the exception of TMS). The tester usually simply follows a pre-written action plan, recording the result - whether the behavior was expected or not (in this case, it is recorded what went wrong).
In most cases, manual testing is a functional test, as well as software and hardware compatibility testing. Manual testing is most common for testing user interfaces - here automation is sometimes economically inefficient or not possible at all.
An employee for manual testing should look for a painstaking, inquisitive and patient person who is “on you” with test objects - sites, mobile devices, SmartTV, etc.

Automated Testing

The need for automated testing arises when the project is already large and its active development is underway. To avoid wasting time manually running tests, which have to be performed very often, you can use automated testing.
Often, autotests understand only unit tests, although almost any type of testing can be automated. We believe that any testing of the code on the basis of the “white” or “gray box” is the task of the developer, not the tester, therefore, we will not consider the organization of unit testing here.
The tools for automating manual functional testing are quite wide. We ourselves use Selenium to test the web.

It often happens that the cost of maintaining autotests far exceeds the cost of manual testing. Therefore, we must act without fanaticism.
The requirements for an employee are almost the same as for an administrator programmer. Why to the administrator? Because autotesting is closely related to release engineering.
That is, we, as a rule, either collect packages for testing, or roll out the release into a test environment (staging environment or “quarantine” in our terminology). At some point, we come to the conclusion that we need to minimize the testing time before release.

Stress Testing

About stress testing will be discussed after the first peak of users. If everything slowed down and fell, then as soon as the sparks ceased to pour from the eyes, raising the project, the developers themselves will begin to pronounce this mantra, and you will be lucky if in the TOR for the project you find the load numbers that your project / product must withstand. Oddly enough, the customer himself often does not know what to expect from users. Typically, these numbers will be knocked out of the manager by the developers, and the manager by the otherworldly manager, that is, by the customer.
What tool to test? We use yandex-tank.
Requirements for the employee conducting the test:

• knowledge of the technical structure of the project,
• good administration level in linux (I'm talking about large projects),
• programming skills.

Fault tolerance testing

You will need this type of testing as soon as you start building fault-tolerant systems (for example, with element redundancy or with graceful degradation). Sad experience shows that in the event of a failure of one element in such systems, in general, all the behavior of other elements is difficult to predict.
How do we conduct a fault tolerance test? We analyze the technical structure of the project, carefully plan the actions, and during the hours of least load we bring some elements of the system out of work, study the consequences, make improvements and adjustments. Then we repeat.

Conclusion

To summarize, what needs to be done to create a testing department:

• Choose the right people - from the company or from the outside;
• Get a time planning tool for testing releases;
• Organize work with TMS to form test cases and record test results;
• Actually, regularly conduct tests and analyze the results;
• Start with manual testing and, if necessary, implement automation; as projects develop, add stress testing and fault tolerance testing.