Test Maturity Model: how can a tester evaluate a project and plan processes
Hello! My name is Misha and I am Senior QA with commercial experience of more than 6 years. Now I work in Provectus, but I started my tester path as a student when I took part in alpha and beta tests of various games. At some point I thought: “Why not start doing this professionally?” And off we go. During this time, I managed to participate in various projects: from start-ups to enterprises, from small educational unit games to huge applications with strong business logic.
But often I was introduced into small teams, in which there were from 5 developers for 1-2 testers and, as a rule, a lot of heat in the project. Actually, I want to share with you how to learn to understand where you found yourself and how to begin to advance in the formulation of QA processes.
Having gained experience from participating in projects, I would conditionally divide them into 3 types:
At the time of my involvement, each of them was at its stage of development. I began to observe these situations and began to develop my vision - how to promote testing processes on them. After some time, I stumbled upon the Maturity Model, and she lay down one on one on my groundwork. This strengthened my belief that this is the place to be.
And here I am very cleverly insert a quote from Wikipedia :
In short and simple, this is a set of models that show how well organized processes are in the organization according to certain criteria. Having such an assessment, one can soberly assess whether it is worth giving one or another order to a particular contractor.
Actually, from this the development of the Maturity Model began. In the 80s, the US Department of Defense realized that it could not accurately assess the quality of the work of software development contractors. And since this is a state structure of such a level, this is unacceptable. The money is state-owned, the deadlines are clearly delineated, and reliable software for weapons will allow everyone to sleep more peacefully. Based on this, the Ministry instructed the Software Engineering Institute to create such a rating system and a year later the first questionnaire was created, and 4 years later the first version of the model was created.
These are 5 levels, within which the work / reliability / stability of the enterprise is evaluated.
For testers, there is a special MM, it is called TMMi. It also contains 5 levels, on which I would like to dwell in more detail and consider what is typical for each of them.
The first level is “Initial”.
At the first level, testing is not structured and chaotic. There is no stable environment to support testing processes. The team does not pay attention to planning and standards.
The main goal is to deliver the product on time without critical problems, because testing here is used only to show that the application works without serious glitches. Often the success of such projects depends solely on the heroism and skills of individuals, rather than established processes.
As a result:
The second level is “Repeatable.”
At the second level, testing becomes a process-driven process. Discipline and progress made ensure that these practices are maintained during stress. Testing is still regarded as the activity that follows development. Test plans are developed and implemented, with the help of which they clearly determine what kind of testing should be done, when and by whom.
The main goal is to ensure that the product meets the specified requirements.
As a result:
Third level - “Defined”
At the third level, testing is no longer regarded as activity following programming. Testing is fully integrated into the project. Test planning is carried out at earlier stages and is fixed in the master plan. Non-functional testing (e.g. usability) is being introduced.
As a result:
Fourth level - “Measurable”
At the fourth level, testing is a well-defined, well-established and measurable process. Testing is perceived as an assessment and consists of all operations of the software development life cycle. The practice of measuring the quality of testing is being introduced. These measurements are used to predict the performance and cost of testing.
As a result:
Fifth level - “Innovative”
At the fifth level, all approaches and processes are well established. The team does not stop at this stage, but continues to systematically improve processes, constantly trying to reduce the time of the testing cycle and time-to-market without reducing the quality of the project. Testing focuses on preventiveness. Automation is added for more efficient use of resources.
As a result:
Case No. 1. Unfair testing
Once I got to a small project where testing was carried out by a customer, his friends or a cat. Having assessed the situation and looking at the model, we can understand that we are at level 1 and that we should focus on level 2 during activity planning. After putting Jira in order, where there were a huge number of bugs (most of which were duplicates or not reproducible), I started compiling test documentation in the form of checklists and arranging the basic testing processes. Such as regression testing and sanity checks during major changes.
Case No. 2. Without testing
In my opinion, projects of this type can be in three cases:
Once in any of these projects, you can often see the benefit of the fact that it is at a secret zero level. We can jump over the first level and immediately do everything qualitatively with a good sense, feeling, arrangement, guided by the goals of the second. Often we can implement test plans, on the basis of which all the necessary types of testing will be performed. You can immediately identify the same workflow with the development team that is missing on projects from the first case.
Case No. 3. Testing was
In truth, the rarest case: a person, by virtue of certain events, decided to change his team / department / work and gives you his best practices. In this situation, you already have an established system of interaction with developers, a certain base of test documentation. Further development paths may include improving the established process or moving to the third level - introduce testing at earlier stages or add non-functional testing.
Case number 4. Three in one
When I came to my project in Provectus, I was faced with a situation in which there were only a little bit from the three cases described above. As in case No. 1, the first thing to do was to put Jira in order. As in the second case, it was decided to do everything immediately with high quality in order to be immediately guided by the second level. Therefore, I immediately began to develop test documentation and implement test management tools. I also tried to squeeze the most out of the artifacts of past iterations, as was the case in the third case.
After a very short time, I can safely say that we are already deliberately going to the third level - we even connected testing at the business requirements stage :)
In my experience, most Ukrainian projects, as well as projects that are not designed for a long time, can successfully be brought to Go live at level 3. But if you have a long-term project, the possibilities to improve it are endless and you can safely be guided by the achievements of the highest levels.
In conclusion, I would like to say that the purpose of this article was to show not so much how to work with the classic MM or TMM itself, but that with its help you can clearly understand at what stage the project you are involved in, and which movements to take, and which are not worth it. Moreover, taking its principles as a basis, you can create your own model, which can be applied not only in development, but also in various areas of life. And most importantly - it is tested and works :)
But often I was introduced into small teams, in which there were from 5 developers for 1-2 testers and, as a rule, a lot of heat in the project. Actually, I want to share with you how to learn to understand where you found yourself and how to begin to advance in the formulation of QA processes.
First of all, you should understand what project you are on.
Having gained experience from participating in projects, I would conditionally divide them into 3 types:
- projects without testing;
- projects with unfair testing;
- testing projects.
At the time of my involvement, each of them was at its stage of development. I began to observe these situations and began to develop my vision - how to promote testing processes on them. After some time, I stumbled upon the Maturity Model, and she lay down one on one on my groundwork. This strengthened my belief that this is the place to be.
What is Maturity Model
And here I am very cleverly insert a quote from Wikipedia :
Capability Maturity Model Integration (CMMI) - a set of models (methodologies) for improving processes in organizations of different sizes and activities. CMMI contains a set of recommendations in the form of practices, the implementation of which, according to the developers of the model, allows you to realize the goals necessary for the full implementation of certain areas of activity.
In short and simple, this is a set of models that show how well organized processes are in the organization according to certain criteria. Having such an assessment, one can soberly assess whether it is worth giving one or another order to a particular contractor.
Actually, from this the development of the Maturity Model began. In the 80s, the US Department of Defense realized that it could not accurately assess the quality of the work of software development contractors. And since this is a state structure of such a level, this is unacceptable. The money is state-owned, the deadlines are clearly delineated, and reliable software for weapons will allow everyone to sleep more peacefully. Based on this, the Ministry instructed the Software Engineering Institute to create such a rating system and a year later the first questionnaire was created, and 4 years later the first version of the model was created.
Maturity Model Levels
These are 5 levels, within which the work / reliability / stability of the enterprise is evaluated.
Where in the Maturity Model is testing
For testers, there is a special MM, it is called TMMi. It also contains 5 levels, on which I would like to dwell in more detail and consider what is typical for each of them.
The first level is “Initial”.
At the first level, testing is not structured and chaotic. There is no stable environment to support testing processes. The team does not pay attention to planning and standards.
The main goal is to deliver the product on time without critical problems, because testing here is used only to show that the application works without serious glitches. Often the success of such projects depends solely on the heroism and skills of individuals, rather than established processes.
As a result:
- no test documentation;
- the product is unstable;
- refusal of processes during problem situations;
- testing is nothing more than an aid in debugging.
The second level is “Repeatable.”
At the second level, testing becomes a process-driven process. Discipline and progress made ensure that these practices are maintained during stress. Testing is still regarded as the activity that follows development. Test plans are developed and implemented, with the help of which they clearly determine what kind of testing should be done, when and by whom.
The main goal is to ensure that the product meets the specified requirements.
As a result:
- there are the main types and types of testing (integration, modular, regression, acceptance);
- test plans implemented;
- test activities are monitored and controlled;
- the process is documented and can be repeated.
Third level - “Defined”
At the third level, testing is no longer regarded as activity following programming. Testing is fully integrated into the project. Test planning is carried out at earlier stages and is fixed in the master plan. Non-functional testing (e.g. usability) is being introduced.
As a result:
- testing begins with the requirements stage;
- activities are added that allow you to work more efficiently (internal trainings, additional reviews);
- non-functional testing is being introduced.
Fourth level - “Measurable”
At the fourth level, testing is a well-defined, well-established and measurable process. Testing is perceived as an assessment and consists of all operations of the software development life cycle. The practice of measuring the quality of testing is being introduced. These measurements are used to predict the performance and cost of testing.
As a result:
- testing as a measurable process;
- measurements are used for forecasting;
- the team is looking for ways to make the testing process more efficient.
Fifth level - “Innovative”
At the fifth level, all approaches and processes are well established. The team does not stop at this stage, but continues to systematically improve processes, constantly trying to reduce the time of the testing cycle and time-to-market without reducing the quality of the project. Testing focuses on preventiveness. Automation is added for more efficient use of resources.
As a result:
- continued process improvement;
- focus on preventiveness and optimization.
How knowledge of this structure helps
Case No. 1. Unfair testing
Once I got to a small project where testing was carried out by a customer, his friends or a cat. Having assessed the situation and looking at the model, we can understand that we are at level 1 and that we should focus on level 2 during activity planning. After putting Jira in order, where there were a huge number of bugs (most of which were duplicates or not reproducible), I started compiling test documentation in the form of checklists and arranging the basic testing processes. Such as regression testing and sanity checks during major changes.
Case No. 2. Without testing
In my opinion, projects of this type can be in three cases:
- the project is just starting;
- there was no need for a tester on the project while there was a little functionality;
- The full-stack team closed the shortage of testers with quality code-review and dev-testing.
Once in any of these projects, you can often see the benefit of the fact that it is at a secret zero level. We can jump over the first level and immediately do everything qualitatively with a good sense, feeling, arrangement, guided by the goals of the second. Often we can implement test plans, on the basis of which all the necessary types of testing will be performed. You can immediately identify the same workflow with the development team that is missing on projects from the first case.
Case No. 3. Testing was
In truth, the rarest case: a person, by virtue of certain events, decided to change his team / department / work and gives you his best practices. In this situation, you already have an established system of interaction with developers, a certain base of test documentation. Further development paths may include improving the established process or moving to the third level - introduce testing at earlier stages or add non-functional testing.
Case number 4. Three in one
When I came to my project in Provectus, I was faced with a situation in which there were only a little bit from the three cases described above. As in case No. 1, the first thing to do was to put Jira in order. As in the second case, it was decided to do everything immediately with high quality in order to be immediately guided by the second level. Therefore, I immediately began to develop test documentation and implement test management tools. I also tried to squeeze the most out of the artifacts of past iterations, as was the case in the third case.
After a very short time, I can safely say that we are already deliberately going to the third level - we even connected testing at the business requirements stage :)
findings
In my experience, most Ukrainian projects, as well as projects that are not designed for a long time, can successfully be brought to Go live at level 3. But if you have a long-term project, the possibilities to improve it are endless and you can safely be guided by the achievements of the highest levels.
In conclusion, I would like to say that the purpose of this article was to show not so much how to work with the classic MM or TMM itself, but that with its help you can clearly understand at what stage the project you are involved in, and which movements to take, and which are not worth it. Moreover, taking its principles as a basis, you can create your own model, which can be applied not only in development, but also in various areas of life. And most importantly - it is tested and works :)