There are no autotests in the team - what should I do?
Some of you may have heard that the “ Code Hour ” educational event took place throughout Russia in the first week of December : open classes were held for students in grades 5–11 to demonstrate that programming is simple and fun, and everyone can learn it.
We at Acronis decided to share our own experience: how we try to train our employees in programming without taking them away from work.The
task
As you know, product acceptance testing, by and large, comes down to repeating the same work from build to build. Rovers run standard scripts on each build - sometimes everything in a row, sometimes selectively. One way or another, there is a list of actions that need to be performed periodically. Naturally, there is a desire to automate this process.
It would seem that this is not difficult at all: specific steps and results are given that should ultimately be obtained — however, there is one big “BUT”. Most testers do not speak programming languages. When we encountered this problem at Acronis, we had the idea to find a tool that could run test scripts written by people without developer qualifications. Testers will write scripts as before (maybe a little more formalized), but these scripts will not be executed by a person, but by a machine.
Tool
Several generally accepted truths have long been established in automation: for example, keyword-driven or data-driven approaches, BDD (Behavior Driven Development) and ATTD (Acceptance Test Driven Development), writing test libraries for your application, generating reports, etc. As a tool for our task, we chose the Robot Framework , which brings all these things together. This is an open-source framework that is primarily designed to automate acceptance testing and ATTD. At the same time, its capabilities go far beyond this framework and allow you to create a powerful framework for automating software testing with an orientation to a wide variety of needs.
We focused on this framework for several reasons. Firstly, because it has tools for testing web pages and web applications. Secondly, there is a wide range of packages and extensions: for example, we use the add-on for controlling machines through SSH. Finally, the Robot Framework is based on the principle of keyword-driven testing, that is, on the development of keywords that can then be used even by people who are not very knowledgeable in programming.
This approach is very convenient to use for building automatic tests. Tests are written in a high-level language and can, in theory, be written by anyone who knows how to test a product.
For example :
Based on keywords, you can easily build tests with various input data, or turn tests into readable and executable text. Libraries are integrated in the Robot Framework for automating web applications, databases, file system actions, SSH, Swing, SWT, Windows GUIs, etc. Also, this framework has a test editor and many additional plug-ins for integration into any projects. In general, the architecture of the framework is designed in such a way that you can easily expand the original functionality and write your own libraries in Python or Java.
Implementation
By the beginning of the experiment, most of our services were already covered by functional tests that the developers themselves wrote on the python. At the same time, the web console and the integration points between it and the server were not covered by tests. We decided, so to speak, to “kill two birds with one stone”: to teach testers to write automatic tests and to attract front-end developers to write basic functionality and keywords at the lower level. For example, the code for the phrase “open browser” in the Robot Framework is as follows:
In general, the framework is very easy to use, so our front-end developers quickly mastered it and began to cover the most painful sections of the code with tests. In our opinion, it is worth noting separately the good reporting system that is provided by the robot out of the box. Although we didn’t have to use it much, because we already had our own reporting system for testing.
The integration of the Robot Framework with our reporting system did not cause any special problems. The robot supports any kind of add-ons for reports: for this you just need to write a Python class with a couple of methods:
Another feature that we evaluated in this case is the ability to group tests using tags. Each test can be assigned one or another attribute, and then select only those that have it to run. For example, flag tests that take a long time to run, and do not run them on every run. We use this feature mainly to monitor our battle servers. We select a part of the tests that check the availability of the main entry points into the system, write the corresponding plug-in (so that the result is in a format that is understandable to zabbix-agent) and as a result we have a constant monitoring system for the accessibility of our site.
By this link you can see the demo version of the robot framework: click
As for attracting our testers to automation, we are moving in the right direction. Now, for each bug or task in the bug tracker, developers write scripts “how to check”: moreover, they write so that they can be easily used in work. So, step by step, we convince our testers that programming and testing is easy :)
We at Acronis decided to share our own experience: how we try to train our employees in programming without taking them away from work.The
task
As you know, product acceptance testing, by and large, comes down to repeating the same work from build to build. Rovers run standard scripts on each build - sometimes everything in a row, sometimes selectively. One way or another, there is a list of actions that need to be performed periodically. Naturally, there is a desire to automate this process.
It would seem that this is not difficult at all: specific steps and results are given that should ultimately be obtained — however, there is one big “BUT”. Most testers do not speak programming languages. When we encountered this problem at Acronis, we had the idea to find a tool that could run test scripts written by people without developer qualifications. Testers will write scripts as before (maybe a little more formalized), but these scripts will not be executed by a person, but by a machine.
Tool
Several generally accepted truths have long been established in automation: for example, keyword-driven or data-driven approaches, BDD (Behavior Driven Development) and ATTD (Acceptance Test Driven Development), writing test libraries for your application, generating reports, etc. As a tool for our task, we chose the Robot Framework , which brings all these things together. This is an open-source framework that is primarily designed to automate acceptance testing and ATTD. At the same time, its capabilities go far beyond this framework and allow you to create a powerful framework for automating software testing with an orientation to a wide variety of needs.
We focused on this framework for several reasons. Firstly, because it has tools for testing web pages and web applications. Secondly, there is a wide range of packages and extensions: for example, we use the add-on for controlling machines through SSH. Finally, the Robot Framework is based on the principle of keyword-driven testing, that is, on the development of keywords that can then be used even by people who are not very knowledgeable in programming.
This approach is very convenient to use for building automatic tests. Tests are written in a high-level language and can, in theory, be written by anyone who knows how to test a product.
For example :
open browser
welcome page should be opened
username field should be visible
password field should be visible
login button should be visible
input username $username
input password $password
press login buttonBased on keywords, you can easily build tests with various input data, or turn tests into readable and executable text. Libraries are integrated in the Robot Framework for automating web applications, databases, file system actions, SSH, Swing, SWT, Windows GUIs, etc. Also, this framework has a test editor and many additional plug-ins for integration into any projects. In general, the architecture of the framework is designed in such a way that you can easily expand the original functionality and write your own libraries in Python or Java.
Implementation
By the beginning of the experiment, most of our services were already covered by functional tests that the developers themselves wrote on the python. At the same time, the web console and the integration points between it and the server were not covered by tests. We decided, so to speak, to “kill two birds with one stone”: to teach testers to write automatic tests and to attract front-end developers to write basic functionality and keywords at the lower level. For example, the code for the phrase “open browser” in the Robot Framework is as follows:
*** Keywords ***
open browser
[Arguments] ${url}
run keyword if '${DEBUG}' == 'false' set log level debugging
Browser.open browser ${url} ${BROWSER} desired_capabilities=service_args:--ignore-ssl-errors=true;--ssl-protocol=tlsv1;--debug=${DEBUG};--disk-cache=true;--max-disk-cache-size=204800
set browser implicit wait ${IMPLICIT WAIT}
set selenium speed 0.2s
delete all cookies
set window size 1400 1280 # for PhantomJS
maximize browser windowIn general, the framework is very easy to use, so our front-end developers quickly mastered it and began to cover the most painful sections of the code with tests. In our opinion, it is worth noting separately the good reporting system that is provided by the robot out of the box. Although we didn’t have to use it much, because we already had our own reporting system for testing.
The integration of the Robot Framework with our reporting system did not cause any special problems. The robot supports any kind of add-ons for reports: for this you just need to write a Python class with a couple of methods:
class CustomReporter:
def start_suite(self, name, attributes):
# put your code here
def end_suite(self, name, attributes):
# put your code here
def start_test(self, name, attributes):
# put your code here
def end_test(self, name, attributes):
# put your code hereAnother feature that we evaluated in this case is the ability to group tests using tags. Each test can be assigned one or another attribute, and then select only those that have it to run. For example, flag tests that take a long time to run, and do not run them on every run. We use this feature mainly to monitor our battle servers. We select a part of the tests that check the availability of the main entry points into the system, write the corresponding plug-in (so that the result is in a format that is understandable to zabbix-agent) and as a result we have a constant monitoring system for the accessibility of our site.
By this link you can see the demo version of the robot framework: click
As for attracting our testers to automation, we are moving in the right direction. Now, for each bug or task in the bug tracker, developers write scripts “how to check”: moreover, they write so that they can be easily used in work. So, step by step, we convince our testers that programming and testing is easy :)