Container-based integration testing
Integration testing remains an important part of the CI / CD production cycle, including when developing container applications. Integration tests, as a rule, are not very long, but very resource-intensive workloads. Let's see how you can combine integration testing tools and tools with container orchestration tools (in particular, with Red Hat OpenShift ) to speed up testing, increase its dynamism, and more efficiently use resources.
Let's create integration BDD tests ( behavior-driven development - development through behavior) using Cucumber , Protractor and Selenium and run them on the OpenShift platform using Zalenium.
When developing integration tests, BDD allows you to create definitions of integration tests for business analysts (BA), and not only for programmers. Thanks to BDD, the development process can be organized in such a way that the functional requirements and definitions of integration tests are prepared simultaneously, and they are created by business analysts.
This is much better than traditional approaches, when the business functionality of the application is first determined, and then the QA department creates integration tests, as shown in the diagram below:
And this is what it looks like when using BDD:
In addition, in this new scheme, each iteration, as a rule, takes less time.
Business analysts can create definitions of integration tests because in BDD these tests are described in a simple and understandable language Gherkin , the main keywords of which are Given (Provided), When (When) and Then (Then). Each statement in the Gherkin language must necessarily begin with one of these words.
For example:
Given the user navigated to the login page (Provided that the user entered the login page)
When the user enters the username and password
When the username and password are correct (When the username and password are correct )
Then the system logs them in. (Then the system allows it to log in).
One of the popular execution environments that can interpret Gherkin tests is Cucumber . To use Cucumber, the developer must implement certain functions so that any Gherkin directives can be executed. Cucumber has bindings to many programming languages. Tests are recommended (but not required) to write in the same language as the application under test.
Let us analyze the testing procedure on the example of the TodoMVC web application in the implementation of AngularJS . AngularJS is a popular single-page application framework (Single-Page Applications, SPA).
Since AngularJS is JavaScript, we will use Cumcumber.js , the binding of Cucumber to JavaScript.
To emulate user experience in the browser, use Selenium . Selenium is a process that can launch a browser and replay users' actions on commands received via the API.
Finally, we will use Protractorto take into account the nuances of emulating SPA applications written in AngularJS. Protractor will take on the challenge of waiting for views inside the page to load correctly.
So, our testing stack looks like this:
The process presented in this diagram is described as follows:
Deploying such a stack in a non-containerized infrastructure is not easy. And not only because of the large number of processes and frameworks used, but also because it was always difficult to launch a browser on a server without a monitor. Fortunately, in a containerized world, all this can be automated.
Corporate web applications need to be tested for various combinations of client OS and browser. Usually this is limited to a certain basic set of options that reflects the situation on the end-user machines of the application. But even in this case, the number of test configurations for each application rarely drops below half a dozen.
If you consistently deploy the stack for each test configuration and run the corresponding tests on it, then it is too expensive in terms of time and resources.
Ideally, I would like to run the tests in parallel. Selenium-Grid
can help us with this - a solution that includes the Selenium Hub query broker and one or several nodes on which these queries are executed.
Each Selenium-node, which usually works on a separate server, can be configured for a certain combination of client OS and browser (in Selenium, these and other characteristics of the node are called capabilities). At the same time, Selenium Hub is smart enough to send requests for which certain Selenium properties are required to those nodes that have these properties.
Selenium-Grid clusters are quite difficult to install and manage, and so much so that even companies with relevant services appeared on the market. In particular, among the main players are the companies SauceLabs and BrowserStack.
Very often, I would like to be able to create a Selenium-Grid cluster, which would provide the necessary Selenium-properties for our tests and perform the tests themselves with a high degree of parallelization. Then, after the completion of testing, I would like to be able to completely destroy this cluster. In other words, to learn how to work in the way that the service providers of integration test farms work with.
This area of technology is still at an early stage of formation, however, one promising open source project - Zalenium - offers some of what we need.
Zalenium uses a modified Hub that can create nodes on demand and destroy them when they are no longer needed. At the moment, Zalenium only supports Chrome and Firefox browsers on Linux. But with the advent of Windows sites for Kubernetes, Explorer and Edge support on Windows may appear.
If you put it all together, the technological stack looks like this:
Each oval in the diagram is a separate pod in Kubernetes. The test player and emulator pods are ephemeral and are destroyed at the end of the test.
Create a simple Jenkins pipeline to show how to embed this type of integration testing into the rest of the release management process. It looks like this:
Your pipeline may be different, but you still have the opportunity to reuse the integration testing phase without significant code refactoring.
Since most of the pods are ephemeral, one of the tasks of the pipeline is to collect test results. In Jenkins, this can be done with the help of the archive and publishHTML primitives.
And this is an example of a report on the results of tests (note that the tests were run on two browsers):
Thus, despite the complexity of the organization of the end-to-end integration testing infrastructure, the “infrastructure as code” approach allows to simplify the process. Performing integration tests on various combinations of operating systems and browsers takes a lot of time and resources, but the means of container orchestration and ephemeral workloads help to cope with this problem.
Even in the absence of established tools for organizing container-based integration testing, today you can perform integration testing on the basis of container platforms and take advantage of the container approach.
If you are developing container applications, try using this approach as part of the CI / CD pipeline and see if it helps simplify integration testing.
Примеры кода из этой статьи можно найти на сайте GitHub по адресу redhat-cop/container-pipelinesh.
Let's create integration BDD tests ( behavior-driven development - development through behavior) using Cucumber , Protractor and Selenium and run them on the OpenShift platform using Zalenium.
BDD testing
When developing integration tests, BDD allows you to create definitions of integration tests for business analysts (BA), and not only for programmers. Thanks to BDD, the development process can be organized in such a way that the functional requirements and definitions of integration tests are prepared simultaneously, and they are created by business analysts.
This is much better than traditional approaches, when the business functionality of the application is first determined, and then the QA department creates integration tests, as shown in the diagram below:
And this is what it looks like when using BDD:
In addition, in this new scheme, each iteration, as a rule, takes less time.
Business analysts can create definitions of integration tests because in BDD these tests are described in a simple and understandable language Gherkin , the main keywords of which are Given (Provided), When (When) and Then (Then). Each statement in the Gherkin language must necessarily begin with one of these words.
For example:
Given the user navigated to the login page (Provided that the user entered the login page)
When the user enters the username and password
When the username and password are correct (When the username and password are correct )
Then the system logs them in. (Then the system allows it to log in).
One of the popular execution environments that can interpret Gherkin tests is Cucumber . To use Cucumber, the developer must implement certain functions so that any Gherkin directives can be executed. Cucumber has bindings to many programming languages. Tests are recommended (but not required) to write in the same language as the application under test.
Testing Technology Stack
Let us analyze the testing procedure on the example of the TodoMVC web application in the implementation of AngularJS . AngularJS is a popular single-page application framework (Single-Page Applications, SPA).
Since AngularJS is JavaScript, we will use Cumcumber.js , the binding of Cucumber to JavaScript.
To emulate user experience in the browser, use Selenium . Selenium is a process that can launch a browser and replay users' actions on commands received via the API.
Finally, we will use Protractorto take into account the nuances of emulating SPA applications written in AngularJS. Protractor will take on the challenge of waiting for views inside the page to load correctly.
So, our testing stack looks like this:
The process presented in this diagram is described as follows:
- When the Cucumber test runs, Cucumber reads the test definition from the Gherkin file.
- Then it starts a call to the test script implementation code.
- The test case implementation code uses Protractor to perform actions on a web page.
- When this happens, Protractor connects to the Protractor server and issues commands to Selenium via the API.
- Selenium executes these commands in a browser instance.
- The browser, if necessary, connects to the web server (s). In our example, a SPA application is used, so this connection does not occur, because the application was already loaded when downloading the first page from the server.
Deploying such a stack in a non-containerized infrastructure is not easy. And not only because of the large number of processes and frameworks used, but also because it was always difficult to launch a browser on a server without a monitor. Fortunately, in a containerized world, all this can be automated.
Integration testing farm
Corporate web applications need to be tested for various combinations of client OS and browser. Usually this is limited to a certain basic set of options that reflects the situation on the end-user machines of the application. But even in this case, the number of test configurations for each application rarely drops below half a dozen.
If you consistently deploy the stack for each test configuration and run the corresponding tests on it, then it is too expensive in terms of time and resources.
Ideally, I would like to run the tests in parallel. Selenium-Grid
can help us with this - a solution that includes the Selenium Hub query broker and one or several nodes on which these queries are executed.
Each Selenium-node, which usually works on a separate server, can be configured for a certain combination of client OS and browser (in Selenium, these and other characteristics of the node are called capabilities). At the same time, Selenium Hub is smart enough to send requests for which certain Selenium properties are required to those nodes that have these properties.
Selenium-Grid clusters are quite difficult to install and manage, and so much so that even companies with relevant services appeared on the market. In particular, among the main players are the companies SauceLabs and BrowserStack.
Container-based integration testing
Very often, I would like to be able to create a Selenium-Grid cluster, which would provide the necessary Selenium-properties for our tests and perform the tests themselves with a high degree of parallelization. Then, after the completion of testing, I would like to be able to completely destroy this cluster. In other words, to learn how to work in the way that the service providers of integration test farms work with.
This area of technology is still at an early stage of formation, however, one promising open source project - Zalenium - offers some of what we need.
Zalenium uses a modified Hub that can create nodes on demand and destroy them when they are no longer needed. At the moment, Zalenium only supports Chrome and Firefox browsers on Linux. But with the advent of Windows sites for Kubernetes, Explorer and Edge support on Windows may appear.
If you put it all together, the technological stack looks like this:
Each oval in the diagram is a separate pod in Kubernetes. The test player and emulator pods are ephemeral and are destroyed at the end of the test.
Performing integration tests within the CI / CD pipeline
Create a simple Jenkins pipeline to show how to embed this type of integration testing into the rest of the release management process. It looks like this:
Your pipeline may be different, but you still have the opportunity to reuse the integration testing phase without significant code refactoring.
Since most of the pods are ephemeral, one of the tasks of the pipeline is to collect test results. In Jenkins, this can be done with the help of the archive and publishHTML primitives.
And this is an example of a report on the results of tests (note that the tests were run on two browsers):
Conclusion
Thus, despite the complexity of the organization of the end-to-end integration testing infrastructure, the “infrastructure as code” approach allows to simplify the process. Performing integration tests on various combinations of operating systems and browsers takes a lot of time and resources, but the means of container orchestration and ephemeral workloads help to cope with this problem.
Even in the absence of established tools for organizing container-based integration testing, today you can perform integration testing on the basis of container platforms and take advantage of the container approach.
If you are developing container applications, try using this approach as part of the CI / CD pipeline and see if it helps simplify integration testing.
Примеры кода из этой статьи можно найти на сайте GitHub по адресу redhat-cop/container-pipelinesh.