What does the DevOps methodology mean for testers?

Original author: Paul Gerrard
  • Transfer
We suggest you get acquainted with the translation of Paul Gerrard's article “What Does DevOps Mean for Testers?”.
Paul Gerrard is CTO of TestOpera Limited and director of the UK Test Management Forum, winner of the EuroSTAR European Testing Excellence and European Software Testing Awards (TESTA).
Translation and publication made with permission


Background


In this article I want to discuss the implementation of the DevOps methodology in terms of testing and tester. The “DevOps Movement” (there is no better name yet) is developing rapidly. The speed of introducing innovations in this methodology, like many other changes that are taking place in this industry, is very high. However, there is still no clearly defined name for the movement itself.

Depending on the structure of your team, on who you are talking to, the DevOps methodology can be both a solution to a problem and an independent goal. In some enterprises, the goal is the transition to digital technology, and the DevOps methodology is part of a common approach to ensuring regular and high-quality deployments. It is in this sense that I consider DevOps in this article. However, when marketing technologies and services related to DevOps methodology, this goal may not be fully understood. The issue of cultural change (or behavior change) required to succeed is often underestimated. Another assumption, from which I build, is that for testers working within the DevOps methodology and under its influence,

I built this article as an introduction to the DevOps methodology for such testers, as well as a discussion of its impact on testing methods. If you are an experienced practitioner using the DevOps methodology, this article may also be helpful. If you are not a tester, you will at least look at the question from the point of view of the tester.

For the lay reader: what is the DevOps methodology?


In simple words: DevOps is a concept that designates a development group and a system operation group that work more closely together. In the so-called deployment pipeline (from source program code to operation in a production environment), developers automate any actions of the operation group. The operations team has the ability to track developer actions and exert some influence on them. The goal is to accelerate the deployment and implementation of software. The combination of the Operations Team (Ops) and the Developers (Dev) (actually in the form of a team using Agile methods) allows us to implement an approach that can be called “Agile Operations Based Operations” (Agile Operations).

The obvious result of the successful implementation of the DevOps methodology is to reduce the amount of time it takes for a software change to go through all the stages from idea to operation in a production environment. When a developer says that a software change is “done”, the transition to using this change in the production environment is done through comprehensive automation. Automated tools and processes are used during system configuration, during the assembly process, during testing, deployment in a test, intermediate and production environment, during monitoring after deployment is completed, during evaluation and operation.

So, are DevOps just tools?

On the one hand, the goal of the DevOps methodology is to eliminate bottlenecks in the deployment pipeline through the use of automation. But automation of multi-stage processes still needs to be managed. Many automated processes are not really autonomous - they cannot be performed without human intervention (to perform maintenance and process non-standard situations). The fully automated DevOps process does not make sense without considering the influence of the human factor. Despite the fact that automation tools do a lot of rough work, it is the people who manage the process that ensure its successful (or unsuccessful) execution.

So, are DevOps just developers (Dev) and an Operations Team (Ops) that work closely with automation?

No, wrong again. The transfer of control between automated processes often includes other processes - usually testing of one kind or another. Automated tests are created by developers and testers. The results of these tests are aimed at providing sufficient information needed to perform other processes or, more often, necessary for people who are moving from one stage of the conveyor to another. Testers and developers who test ensure the successful and reliable execution of the DevOps process.

Head spin! So what does DevOps mean? This is a progressive latest discipline. This issue is discussed in detail in the article “What Is DevOps?”that appeared a few weeks before this article was written. As you already understood, the definition of the concept of DevOps has not yet been formulated completely. Perhaps it never will be.

What does this mean for testers? This means that so far there is no “only right way” and that your role in the DevOps mode, which is constantly evolving (and each mode is constantly evolving), has not yet been finally established. There are two points that you can implement:

1) pay attention to pain points and make efforts to reduce their negative impact;
2) identify opportunities that allow you to optimize the DevOps process.

If there is a single mantra that best describes the movement towards the DevOps methodology, then it sounds like this: “If this is unpleasant, do it more often.” This may seem like a cliche, but I will use this expression as a context for introducing and improving practical testing skills using the DevOps methodology.

If it’s unpleasant, do it more often.


Difficulties or discomforts that we experience when doing a certain job affect us negatively. If we do not like any task, then we seek to postpone it. When we finally embark on this unpleasant business - it becomes even more unbearable. A visit to the dentist, cleaning in the garage, software integration, testing, etc. Experience suggests that the less often we perform such tasks, the more unpleasant they are. Martin Fowler suggested three reasons why frequent (or even continuous) tasks can reduce discomfort.

The first reason is that large and complex tasks are difficult to plan, difficult to manage and difficult to control. Dividing large tasks into several small ones makes it easier to complete them and reduce the degree of risk, and if something goes wrong, it will be easier to return everything to its previous state. The second reason is that many tasks (and testing in this regard is the best example) provide feedback. This feedback, if it arrives at an early stage and often enough, allows you to quickly respond to problems that arise before time is wasted. And the third reason: if we carry out any activity more often, we better cope with it. We learn to do this work efficiently. In addition, we find ways to automate it in one way or another.

From the point of view of the tester, this mantra forces a more serious approach to the concept of automation in the implementation of the testing process. If manual intervention is performed (usually between automated stages in the DevOps process), then it should be considered a pain point: bottlenecks, causes of delays and potentially unreliable and error prone aspects of the process.

Manual testing is unpleasant. Yes, it can be fun to conduct research testing, you can assume that only as a person can you find those intricate errors that automation will never find, that only you, as a tester, can be trusted to prevent a catastrophe.

You as a tester may be concerned about whether the testing work will be performed properly if it is entrusted to developers or automated tools. If this is unpleasant for you, do it more often.

Tests, Automation and Trust


There is a lot of controversy surrounding the meaning of, for example, testing and testing , and around how much we can rely on testers, on checks and on automated tools ( The New Model and Testing v Checking ).

I am not saying that we should fully rely on automated checks. We will need to apply a more complex approach. However, for the purposes of this article, we can at least divide the tests and testing activities into four components:

1) checks that can be automated by developers as part of component tests and continuous integration processes;

2) checks that can be automated (usually by system testers) to perform transactions at the level of API, communication lines or the entire system;

3) tests, including compatibility checks and allowing to demonstrate compatibility with browsers, operating systems and platforms;

4) tests that can only be performed by a person.

In this article I can only make an assumption how to make such a separation - each environment has its own characteristics. The most appropriate question for this article is how can a tester “free himself” from late manual checks? Earlier, I wrote about the exclusion of late manual checks. This requires forethought and trust.

It is necessary to focus on the following points:

1. If possible, manual checks that can be performed at the component level should be assigned to developers. As a tester, you can offer to perform these tests during a pair or team session. You can write them yourself and enable them in continuous integration mode.

2. Testing the user interface and complex testing may require the use of automated tools. Such testing should be minimized, as it takes a long time, is unstable, and often requires maintenance. Think about whether you need to perform it each time you check the program code or whether it can be delayed for use only on large, less frequent releases.

3. What non-automated tests can be performed on components that are not yet integrated into the pre-release version? Can I do manual testing in pairing sessions with developers? Are there alternatives to such testing? Can working with archival documents, prototyping using the BDD approach help? Can I perform user interface checks on layouts or wireframes?

4. What tests need to be performed only manually, unlike the checks that need to be left for regression testing (they are candidates for automated testing)?

I mentioned trust above. Another question is how to reliably test the system if there is no late manual testing. Imagine an environment in which all testing is done by automated tools. Do you believe that developers will perform the test properly? Shifting the worries of testing to the left should dispel your doubts. If you, as a tester, act like a pathfinder: in order to identify and evaluate risks, select tests and ensure that they are integrated into development and automation processes, then your concerns can be minimized.

Naturally, you should stop considering yourself a quality guard, the last frontier of defense, the only person who cares. You have to think more as a visionary, risk identifier, risk manager, ranger, coordinator and instructor or mentor.

Practice, monitor and improve


With all desire, if you stop relying on late manual checks, there may still be errors. When software is released into production, problems are discovered. One of the key disciplines of the DevOps methodology in terms of operation is deeper monitoring.
Monitoring at every level - from components and simple transactions in applications to integration and messaging and, of course, the infrastructure itself. One of the goals of monitoring is to generate failure messages before users experience their consequences. This is a rather ambitious statement, but it defines the ultimate goal.

When problems occur in a production environment, the task is to use the analytical information obtained during monitoring in order to not only track the cause and eliminate it, but also to clarify the testing process (automated or manual) and reduce the likelihood of such problems in the future. The role of testing and analytical information in the operation of the entire pipeline is defined and discussed in the article “Thinking Big: Introducing Test Analytics” .

We can call automated testing of the DevOps process monitoring. When monitoring is combined with the production environment, it can be said that monitoring throughout the entire DevOps process with entry into production expands the scope of testing. Thus, the DevOps methodology does not reduce the role of testers.

Conclusion


Recently I was asked: "In which case, you should not try to apply the DevOps methodology in the organization?" This is a good question. I think it is based on the fear of whether the DevOps methodology should become the norm and whether testers should pay attention to this. My answer is simple.
Why don't you let developers and maintenance professionals talk to each other? Why don't you start getting more reliable builds and deployments in a test and production environment? Why don't you use the best technology for a more accurate, efficient and informative conveyor? The DevOps methodology is a good thing, but it is not always easy to achieve the required level of its use. Not to mention the fact that it requires a change in culture, but it is not so simple.

From the point of view of the tester, the DevOps methodology influences to a greater extent in the early stages of projects, forces us to think more seriously about testing automation, providing information and making decisions. Testers should strive to use the DevOps methodology, as it provides opportunities to carry out actions in a proactive manner, to gain greater authority and respect in the project team.

On November 11, Paul Gerrard will give a master class in Moscow on the topic “Testing in the Digital Age: DevOps, ChatOps, and Automation to Support Testing .

Also popular now: