What a project manager should know about testing

    I’ve been testing for 8 years. Manual and automated, in the role of tester and test manager, as an employee of the company and as a representative of outsourcing. And on almost all projects, I face the same problem: project managers do not understand why they need testing.

    If you ask the average RM a simple question: “Why is testing on this project?”, The answer will most often be “You're a test manager, you have to answer this question.”

    But when you come to the hairdresser you don’t tell the master “you yourself know what I need”? And at the grocery store you do not ask the seller to throw in your basket what younecessary? You can consult, you can find out “how can you?”, Ask for options, but the decision is always yours. What is the difference between testing? Maybe in the fact that too few project managers understand why they need it?

    In this article I will try to play the role of a seller who shows the client: “what happens at all?” Many things will be described, perhaps too detailed, too simple ... Don’t be angry, I just really want to be understood :)

    Why formulate any expectations from testing?


    Camon! Testing is all kinds of button presses, well, if automated. And so it is clear what goals! You need to find bugs, and the more the better. What other goals might testing have?

    Oops ... Based on this approach, then in all companies with testing everything is in order. Bugs are found, fixed, checked ... And what, testing is always useful? Always unleashes its full potential?

    Something tells me not. Then what are the differences between those rare cases when “everything is fine with testing” and the overwhelming majority?

    I will try to give another illustration. You have never gone for a massage in your life. You go into the salon (and why, everyone goes, why don't you go?) And order a half-hour session. You have a fixed budget and there are requirements: "massage me." An indistinct look, the lady is indistinctly messing around with your quite distinct shoulders, no buzz, you leave the salon and think "oh, massage - it's complete garbage." But your demand “massage me” was fulfilled! Maybe the point is the incorrectness of the requirement? Did your back hurt and you want the pain to go away? Or was there not enough stretching? Or did you want relaxation? The more precise the requirements, the more chances the contractor has to satisfy them. But if you cannot formulate what you want, then the performer will not be able to do what is needed, and you will not be able to objectively evaluate the result.

    What can be expected from testing?


    Testing is necessary in order to improve the quality of the product. Well this is obvious!

    Oops ... A simple example from practice. The testing team finds defects in a timely manner, the development team does not have time to fix them, the product invariably enters the market, customers say “sucks” ... What, was the testing bad?
    Or let's vice versa. Testers thoroughly examine the product, using it in the tail and in the mane, do not find a single error, it is released to the market, users are happy, caps are flying in the air, flowers are sent to the address of the development company ... Are the testers well done or not?

    In general, there are many examples. But the conclusion is almost always the same: quality testing has no effect. Just like your haircut does not affect your success. “Hey, hairdresser, cut my hair so that I pass the exam!” No logic, right? In testing, the same thing. Testing provides information, in a certain budget, in a certain format, with a certain speed ... But testing does not affect the quality, even if it really wants to.

    What then can provide testing?

    All our results, all our work is described by a formula of four variables: test coverage, information provided, speed and budget.

    Let's first look at these points, and then move on to the formulas themselves.

    1. Test coverage


    This is the% of possible user scenarios, conditions, settings, input parameters, etc., which was tested by the testing group. The higher this percentage, the more bugs found, the less missed. The more bugs can be fixed if desired and possible.

    At the same time, test coverage and the number of tests, as well as the resources spent on testing, are connected very indirectly. Qualified testers have a whole arsenal of techniques and tools that allow you to “shove in the non-pushable”, aka optimize the test suite, aka provide maximum coverage with a minimum number of tests.

    Coverage can and should be measured: traceability matrix, code coverage,% of missed defects and a host of other metrics to help you.

    2. Information provided


    For years, a meme about a tester lives, with a panic in his eyes saying "nothing works!". And if you ask me to explain in more detail, he replies: "Well, absolutely nothing!". Perhaps he provided good test coverage, but, alas, did not provide adequate information. What information may a project need from testing?
    • Qualitatively described bugs in the bug tracker
    • Product statistics for forecasting releases and making decisions about processes
    • Quality Metrics for Release Decisions
    • Assessment of the testing itself: what coverage, what is checked, how much testing is enough, etc.

    If the testing group does not provide clear information about the product and the project (and this is its main function!), Then making decisions is very difficult.

    3. Testing speed


    OK, the coverage is good, the information is clear ... It's just too late. Or for a long time. Or long, and therefore late.
    All of you obviously faced with the situation: before the release of 3 days, there is a critical bug. The developer gets on the head, begins to fix it, and it turns out that this bug has been missing for six months, and actually it could be found much earlier. And that this bug 2 months ago would be fixed in half an hour, and now half a product is tied to it, and it will take a week to mess around.

    Few people undertake to analyze TOC projects , and few people notice that testing sometimes lengthens releases several times. Here, replace the testing with a timely one, and get the release twice as fast.

    Usually this is invisible, and it seems that bugs are the source of all troubles ... But sometimes finding them earlier = reduce costs for the entire development cycle!

    Or another aspect of testing speed: the time it takes to validate an assembly. For example, we are preparing for the release. We get RC (release candidate). Testing needs a week to make sure it is working, RM agrees, allocates a week, on the fifth day there is a critical bug ... 10 minutes on a fix, and again 5 days, and again on the fifth there is a critical bug ... Faced? Sounds familiar?

    4. Budget


    Perhaps this is the easiest item of all. Naturally, it is relative. Is 1000 rubles a lot or a little? There is probably a lot of bread for a loaf of bread, perhaps not enough for a trip to Bali. Therefore, the budget is the money that you are willing to pay, but not for “testing”, but for specific and understandable results of this testing itself.

    So what's next?


    And then this. To bring the project maximum benefit, you must know exactly what it needs now. To know what you need, you need to analyze its indicators, which is outside the norm: deadlines, quality or budget. And after that, create your unique formula with testing requirements. At the same time, we will be honest: “reduce the time” or “increase the coverage” - these are the same “massage me”.

    We need accurate indicators. What for? To evaluate changes. To keep track of whether they really affect the performance of the entire project as a whole (let's be honest, you don’t need coverage in itself!). To control the process.

    Notice, I never said “unit tests”, “automation”, “test design”, etc. All actions are introduced into the process only according to the results of certain goals! Need to increase assembly testing speed? We introduce automation. Need to increase the speed of finding defects? Unit tests. Reporting quality? Moderation of defects. In testing, hundreds of approaches, tools, solutions that can be combined based on your goals. But the tools themselves are secondary.

    Therefore, AS a tester / test manager / test outsourcer will achieve your goals - this is his headache. But no one better than RM, who is attentive to their project, will say what this project needs!

    Also popular now: