Automatic testing of web interfaces in Virto Commerce

    In the early stages of development, you can get by manually testing a given test plan. But with the advent of the modular architecture, when several development teams can make their changes simultaneously, there is an exponential increase in the number of test scenarios. Manually chasing them away becomes harder and harder.

    In this article we will talk about how we at Virto Commerce automate the testing of important business scenarios.

    Why all this?

    I will give an example from our experience - a small edit in the CSS-style led to the fact that the “Add product to cart” button was no longer visible on the mobile device. Unfortunately, in this form, it was tested and hit the release. There are many such examples when new functionality, minor changes and corrections break important business scenarios. Unfortunately, we often find out about them after the releases and from customers. To avoid this, more than two years ago, we began to implement E2E testing as part of the development process. After that, most of the errors were revealed at the development stage, and not in the combat environment.

    E2E is testing a business scenario from start to finish. The E2E test simulates the user's real actions in a real browser, precisely as the real user will use the solution.

    We use E2E tests:

    1. To control the regression
    2. To describe the system
    3. For integration into CI / CD

    This includes ensuring that all modules function and work together correctly and predictably.

    E2E tests allow us to cover application sections that are not checked by Unit and integration tests. This is due to the fact that Unit tests and integration tests cover individual parts of the application and test the isolated part of the functionality. Even if these parts work well on their own, one cannot be completely sure whether they will work together. Thus, the presence of a set of E2E tests on top of the Unit and integration allows us to test the entire solution most fully.

    It takes time and resources to write and run E2E tests. Google, for example, offers a 70/20/10 division: 70% unit tests, 20% integration tests, and 10% E2E tests. The exact combination will be different for each project, but in general it should retain the shape of the pyramid.

    E2E tests are not simple and at first seem expensive to develop and maintain. In Virto Commerce, we are constantly working to simplify development and reduce the cost of supporting E2E tests when new releases are released. We hope that our solutions will help you to simplify the use of E2E in your projects.

    A good user story is an almost complete E2E test.

    You can often hear that writing E2E tests takes time and is difficult to maintain. Yes, this is so, they are not easy to maintain in the same way as the documentation. Why not make them part of the development process and writing documentation, thereby fixing the business scenarios made.

    Creating an E2E begins with a user story description. As thoroughly as we prepare the description at this stage, it will be as easy for the development team to write an E2E test, which will demonstrate that all systems work correctly in the implementation of this scenario.

    An example of a user history for the add product button to the cart:

    GIVEN I am signedinto the storefront 
    AND Some product page isopen (/product-name) 
    AND my cart is empty 
    WHEN I click to the "Add" button on the item with the following parameters:
    THEN I should see a dropdown menu where I can choose from 1to10+

    which then turns into the next test:

    After some time, technical writers or a new development team, instead of reading the documentation, can use the tests in order to run and see the implemented scripts in a real browser or automatically record video tutorials.

    It's easy - install and configure Protractor

    As a testing tool, we selected Protractor, an open source E2E test automation system designed specifically for AngularJS web applications.

    Protractor is a Node.js program built on top of WebDriverJS. Protractor works as a solution integrator that combines powerful technologies such as Node.js, Jasmine, Selenium, Mocha, Cucumber, and Web.

    Protractor knows about AngularJS, which allows to simplify the writing of tests, without spending attention on waiting for the launch of the AngularJS project, updating the page, etc. ... Experience shows that the threshold for a developer to enter is very low.
    Use npm to install Protractor globally:

    npm install -g protractor

    webdriver-manager is a utility that allows you to easily configure an instance of Selenium Server. Selenium Server - will be used to transfer commands between the test and the real environment.

    Run this command to install or download:

    webdriver-manager update

    And run:

    webdriver-manager start

    This command will launch Selenium Server and open a window for log output. Protractor will send requests to this server to control the browser.

    Protractor is ready to go. A more detailed description of the basic steps for writing tests can be found on the main site .

    Proper project structure

    E2E tests are necessarily in the same repository, where the application or theme.
    We have abandoned the use of external E2E testing services, because, with seeming simplicity, services complicate the launch of tests on a local machine and subsequent maintenance. The code and tests are in different physical locations, which causes the developers to forget to update them.

    Finding the application code and test in one repository allows you to simplify the maintenance and support of the project.

    A basic example can be found here .

    The project creates an E2E folder of the following structure, in which Page Objects are used to organize tests.

    |	|--cart.pageObject.js
    |	|--*.spec.js
    |	|--home.pageObject.js
    |	|--*.spec.js

    • conf.js - the configuration file is in the root
    • for each object of the interface, a folder is created in which is located
      • pageObject file to describe the elements on the page
      • several spec files - in which there are tests

    The project must use:

    • baseUrl - which allows you to use the test to test different environments
    • params - which are used to find key elements on the page or fill out forms .js # L29

    These parameters can then be changed by the IT administrator when setting up startup in Jenkins, in automatic mode for DEV and QA environments.

    protractor conf.js --baseUrl=

    Optimization of finding elements on the page

    Protractor offers very flexible methods for finding elements on the page:

    • by.binding
    • by.model
    • by.repeater
    • by.className
    • by.css
    • ()
    • by.partialButtonText ()
    • ...

    But in recent projects we have come to the model that the elements that participate in the test are marked with a special class prefixed with protractor-. In the test, these elements are found by the method of.className.

    This simplifies test maintenance and modification, as the programmer or automated tools can determine that an element that participates in the E2E test has been modified.

    What browsers are we testing

    By default, all processes are configured to run tests in the Chrome browser. As our experience of using E2E tests in Chrome shows, it allows us to identify most of the problems and at the same time not complicate the writing of tests.

    If you need to test in several browsers, this is done on the one hand very simple, on the other hand there are difficulties with setting up and the presence of errors in the implementation of drivers for different browsers.

    In various client projects, we additionally carried out testing in Firefox, Safari and IE. Firefox tests virtually duplicate results in Chrome. But the launch of E2E in Safari and IE required setting up the system, opening ports, disabling security and editing the registry, but this allowed to automatically detect a lot of problems with the display and incompatibility of scripts in the application.

    To add testing in the browser, you need to download and install the desired WebDriver.

    And add a new section multiCapabilities in configuration file:

    exports.config = {
      multiCapabilities: [
    	  'browserName' : 'chrome'
    	  'browserName' : 'firefox'

    Since modern web applications support the work at different resolutions, it is necessary to take this into account when writing tests.

    Protractor allows you to run tests for different screen resolutions.

    To do this, there is the option of setting the screen size through browser.driver.manage (). Window (). SetSize. In this example, we set the screen size to 1600 by 800.

    exports.config = {
      capabilities: {
        'browserName': 'chrome'
     onPrepare: function() {
         browser.driver.manage().window().setSize(1600, 800);

    Or through the multiCapabilities section in the configuration file. In this example, we will run tests in the Chrome browser, for three different permissions.

    multiCapabilities: [
       'browserName': 'chrome',
       'chromeOptions' : {
           args: ['--lang=en',
       specs: ['specs.js']
       'browserName': 'chrome',
       'chromeOptions' : {
        args: ['--lang=en',
       specs: ['specs.js']
       'browserName': 'chrome',
       'chromeOptions' : {
        args: ['--lang=en',
       specs: ['specs.js']

    E2E is part of the documentation.

    Once again, E2E is good as an element of documentation, demonstrating what was done by the development team. A new development team or technical writer can run the tests and see the realized scripts live. Therefore, document how a new employee can run these tests.

    CI / CD Integration

    Any tests should be current and run periodically. If this is not done, then it is not worth spending time writing tests, they will become unusable after a couple of weeks, and it will be easier to rewrite them from scratch.

    We have taken an important step in integrating E2E tests into the CI / CD and in becoming E2E part of the deployment process.

    Integration into the CI / CD allows you to run E2E tests automatically for DEV and QA environments to provide feedback and add insight that the system remains working even after a small change. And if the decision is broken, then give information when and with what change it happened.

    First, tests are run when one of the modules is updated and the new version falls on the QA environment.

    Secondly, E2E tests run at night according to the schedule on the DEV and QA environments in automatic mode.

    Thirdly, if necessary, the developers themselves can run the tests required.
    As a result of the launch, all project participants from the development team to the client receive a letter with an HTML report on the test results.

    It is important to note that the availability of information on the test result to the entire project team is mandatory. This gives confidence to the development team on the one hand and manage customer expectations on the other. The development team can make changes faster, while realizing that the core business scenarios are working. And the client receives information about the quality of the project, that most of the problems are identified before the release. And even if the test reveals a problem and the release is postponed, then information appears about what exactly broke and how the process of work on the correction proceeds.

    The report version of the plugin protractor-jasmine2-html-reporter ( is used for generating reports. This plugin generates an HTML report and inserts screenshots of inline elements.

    Installing it is very simple:

    1. Installing protractor-jasmine2-html-reporter
    2. Adding protractor-jasmine2-html-reporter to the project
    3. Connection report onPrepare method

    var HtmlScreenshotReporter = require('protractor-jasmine2-html-reporter');
    var reporter = new HtmlScreenshotReporter(self.options);
    var name = browser.suite;
    self.options.reportTitle = name + '-report.html';

    Continually improve the process

    Constantly work in a team to reduce the cost and increase the speed of development and support of E2E tests.

    Conduct training and review of writing tests. See what new components and services can be used.

    For example, we are considering the Cucumber connection option. Cucumber is a tool for running automated tests written in simple language.


    E2E tests are not so simple and at first seem expensive to maintain, but they are very valuable because they are indicative of the health of important business scenarios.

    From the experience of using E2E tests in the Virto Commerce project, we can draw the following conclusions:

    • Writing an E2E is part of the development process. And it is very important, as it tests the work of business processes and the entire solution. A huge number of errors were revealed at the DEV and QA stages, and not in the combat environment.
    • Tests and code should be in the same repository and be accessible to the developer.
    • The quality of the E2E test depends on the description of the user history; if it is written correctly, it is easier for the development team to create an E2E test.
    • E2E tests can act as documentation and capture made scripts.
    • E2E tests should be run continuously and automatically on the DEV and QA environments. If you have not automated the launch, then you should not waste time on tests, they will become unusable in a couple of weeks.
    • Test results should be available to all project participants. It gives a picture of what is happening.
    • E2E is not a 100% solution to all problems. Follow the 70/20/10 rule: 70% unit tests, 20% integration tests, and 10% E2E tests
    • Constantly work in a team to reduce the cost and increase the speed of development and support of E2E tests.


    Protractor for AngularJS - Writing end-to-end tests has never been so fun tree / master / VirtoCommerce.Storefront.Test / e2e

    End-to-End Tests html

    Also popular now: