Translation of Hans Boovalda's article “Action-Based Testing”

Original author: Hans Buwalda
  • Transfer
We have prepared a translation of an article by Hans Bouvalda, CTO of LogiGear, in which he dispels myths and fears related to test automation in Agile projects.

Hans Buwalda during his professional career has gained vast experience as a software developer, manager and chief consultant in leading companies and organizations around the world. The testing methods he proposed (based on actions and in the style of a soap opera) helped many customers develop scalable and easily supported solutions for a large volume of complex testing tasks. Hans often serves as a speaker at international conferences. He is also a co-author of Integrated Test Design and Automation .


How does functional testing automation fit into Agile projects? Today we are constantly faced with this issue. The Agile strategy is becoming more widespread, but functional testing, as a rule, is still carried out manually, because Agile iterations and sprints simply do not have time to automate it. Unlike functional testing, effective automation of unit testing has become commonplace. You can answer this question by solving the following tasks:

  • well-planned and organized processes for designing tests and automation are needed;
  • these design and automation processes should be organized based on separate life cycles.

In this article, I will show how an action-based testing method can help with these tasks. To do this, I will briefly introduce the action-based testing method, and then I will tell you how to use it to bring the design of tests and their automation in accordance with the requirements of Agile projects.

Action Based Testing


There are many sources where you can learn more about the action-based testing methodology. Here I am only summarizing the basic principles underlying this method.

1. Not one, but three life cycles


Typically, testing and automation tasks are included in the life cycle of a system’s development, regardless of whether cascaded or Agile is used. However, in ABT, three life cycles are distinguished, which, although interdependent, are planned and managed in ABT projects as separate entities:

  1. System development: carried out in accordance with any traditional or flexible model of the life cycle of system development;
  2. Test development - includes test design, test execution, analysis of test results and test support;
  3. Automation - the main attention is paid to keywords of actions, interpretation of actions, comparison of user and other interfaces, research of technological problems, etc.

2. Test Design


The main place is the design of tests. This is the main factor in successful automation, much more significant than the automation technology used. In ABT, it is important to create a high-quality, high-level test design in which so-called test modules are defined. Each test module is developed as a separate miniproject and should have a clearly defined content that is different from other modules.

The test module includes test objectives and action lines. Testing objectives specify the content of the test module in the form of separate verbal provisions that determine what should be tested within the framework of this module.

Tests in the test module (which is arranged in the form of a table) are defined by a series of action lines, which are usually organized in the form of one or more test scenarios. Each line of action defines an action and includes a working word that defines the action, as well as arguments defining data for the action, including input values ​​and expected results.

It should be noted that in ABT the test scenario does not have the same value as in some other testing methods. We believe that the test scenario is a small separate component that is not crucial for the development of tests. Instead of creating a predefined list of test scripts, we create a list of test modules in which test scripts are included as a result of test design, and not as its original elements.

Efficiency is provided by varying various test scenarios. In addition, each test case can fulfill the precondition for the next case, which gives a good flow of test execution.

3. Automation


At ABT, automation work is separate from test development. For designing tests and for automation, different skills and abilities of developers are required. Perhaps there are specialists who are involved in both of them, but, judging from my experience, they are not often found. In addition, these two areas assume different responsibilities for ensuring that tests are performed.

At ABT, automation engineers are involved in automating actions and creating “interface definitions” to control the interaction of the system under test with various interfaces (user and other). To perform these automation tasks, special skills and work experience are required.

Flexible test development


When using ABT with two separate life cycles - test development and test automation - you should pay attention to two topics that are important for including automated testing in Agile projects:

  1. Design and development of tests;
  2. Automation.

When using the Scrum methodology with sprints, testing in Agile projects is divided into three processes:

  1. Testing in regular system development sprints;
  2. Test development before development sprints;
  3. Testing after completion of development.

1. Testing in regular sprints


The most common practice is the development and execution of tests as part of sprints. In the sprint, functionality is determined based on conversations with the customer and user stories to identify what exactly needs to be tested. Testing can be carried out through developed tests similar to the ABT test modules, as well as through research and interactive testing. In addition, it will be very useful to conduct at least some of the most “interesting” interactive tests in test modules for future use.

Unit testing is extremely important in itself, but in ABT it makes sense to consider the possibilities of reusing such tests and their application in various directions for testing individual functions.

By defining test modules for unit testing and assigning them to selected actions, you can easily combine them to test a wider range of values ​​and various elements of the system under test during or after the sprint.

2. Test development before development sprints


When using the ABT method, the use of actions, in particular, high-level business actions, allows you to develop tests that focus not on technical aspects, but on business functions. Such tests are commonly called high-level tests. They do not relate to user interface elements, but are aimed at business transactions, for example, filling out a request for a mortgage loan or renting a car.

Tests of a higher level can be developed at the initial stage of the project. For their development, you do not need to wait for the completion of the system development sprint, since in this case there is little time left to determine business functions and create suitable tests to test them.

The quantity and necessity of business-level tests depends on the specific situation. In general, I would recommend the following:

  • The number of business-level tests should be as large as possible, as they ensure the overall quality of the product and are not dependent on system changes that do not affect the functions being tested.
  • It is necessary to use the design stage of high-level tests in ABT (where test modules are allocated) to determine which functions can be checked first of all when conducting business-level tests and what needs can be covered in detailed tests as part of development sprints.

3. Testing after sprints


After completing sprints for individual parts of the system and after combining these parts, additional testing is usually required to ensure the quality of the entire system and its compliance with the requirements. In addition, additional tests may be needed to retest system elements that were not affected by the changes to confirm that the new elements interact well with the old system elements. This may be required, for example, in regression sprints or fault tolerance sprints.

In my opinion, such “follow-up testing” is the main area in which it is advisable to have pre-developed test modules and fully automated actions. This will save valuable time, especially when the release dates are approaching.

Flexible test automation


Agile projects typically use the term “timely automation” to describe test automation. When using the ABT method, this concept changes and sounds like “timely test development”. In any case, high-level automation plays an important role in increasing productivity and accelerating sprints.

To quickly and timely provide automation, the following rules should be observed:

  • Creating a foundation at the initial stage;
  • Improving automation reliability;
  • Ensuring testability of the system under test;
  • Automation Testing.

1. Creating the foundation at the initial stage


Successful automation begins by creating a solid foundation for the development of follow-up actions. Such a framework should provide various capabilities: performing all necessary operations on all classes of user interfaces, access to APIs, the ability to perform database queries, compiling and processing messages using the appropriate protocols, etc.

Although many technical features are available using LogiGear's TestArchitect tool, most of our projects begin with research and development related to specific technical problems of the customer, for example, modeling devices for cash registers, working with three-dimensional graphics for oil exploration, testing mobile devices, access to firmware in diagnostic equipment, etc.

It is important to start creating such a technical basis as soon as possible and pay close attention to it, identify and solve all technical problems. This will help to implement low-level actions, which can then be used for higher-level actions, for example, in development sprints. In addition, the creation of a technical basis at the initial stage reduces subsequent risks.

2. Improving automation reliability


The essence of Agile projects is that many elements of the system under test are clarified only when they are implemented in the program as part of iterations, such as sprints in Scrum. This primarily applies to areas in which automation is particularly important, for example, when testing the user interface. These elements can undergo many changes during the development process. Moreover, automation should not become a bottleneck. In this case, the main thing is flexibility.

An action-based testing model can provide the necessary flexibility, because it allows you to hide details in individual actions, which can then be quickly reconfigured if necessary. However, other points must be taken into account. Most often in our projects, the problem of “timing” arises. Often, to perform automated actions, you have to wait until the system under test will respond to a particular action and will be ready to perform the following.

We came to the conclusion that to solve this problem, an automation engineer should make the most of the possibilities of active timing. For its implementation, it is necessary to establish a criterion in the system under test - a predetermined maximum value. Upon reaching this value, an automatic transition to the next action occurs immediately. By paying due attention to these and similar issues, you will ensure the reliability and flexibility of automation.

3. Ensuring testability of the system under test


In preparation for automation, system developers must indicate which elements of the system under test provide convenient access for automation tools. If such elements are identified at the initial stage and defined as requirements, they can be easily incorporated into sprints.

A good example is the use of values ​​for certain identifying properties that are available on various platforms for screen controls or HTML elements. The user does not see these properties, but they are available for automation tools. Providing such values ​​will make it easy to automate testing of controls or other elements, moreover, as a rule, regardless of any design changes.

If these values ​​are defined in the early stages of a project, then using, for example, a tool such as TestArchitect, you can create “interface definitions” and use them before building the system under test.

Examples of such useful properties are the id attribute in HTML elements, name in Java / Swing, and accessibility name in .Net and WPF. They do not affect the user experience, but are available for automation tools. In addition, the use of such properties helps to solve localization problems: the “OK” button can always be found, even if the inscription on it is in another language.

4. Automation testing


Automation must be tested. In ABT, this means that you need to test the actions and definitions of interfaces. They are a kind of product that automation engineers provide to testers, this product must be high quality. Each project must have at least one folder (in the TestArchitect tree of tests) with test modules designed to test actions and interface definitions, and not the system under test.

Like the test development process, automation work should be well planned and organized with the participation of experienced professionals. In this case, a combination of careful test development planning and automation planning will help you successfully complete all the requirements of your Agile projects.

On November 22, Hans Bouvalda arrives in Moscow with a master class " Five Key Factors for Successful Test Automation ."

More articles from Hans Boowaldy can be found here .

Also popular now: