Open webinar "Pairwise Testing Method in Black Box Testing"

    Good day to all!

    We offer you a detailed description of the open lesson dedicated to pair testing. The event took place only a couple of days ago and was timed to the launch of the “QA-specialist” course .

    Lecturer - Nina Devayeva - Senior Tester, Team Leader and ISTQB-certified tester, an expert in the direction of Quality Assurance.

    At the open lesson we talked about the need for this type of test-design technique, as pairwise testing. We studied case studies in practice and examined in detail the tools available for work.

    Before the start of the webinar set the following goals:

    • find out what pair testing is like;
    • find out when it is permissible to use pairwise testing;
    • learn how to work with pairwise testing tools.

    A few words about test design

    Test design is a stage in the software testing process where test cases (test cases) are designed and created in accordance with previously defined quality criteria and testing objectives. Some have difficulty asking them why they need a test design. Meanwhile, the answer lies in one of the principles of testing, namely: exhaustive testing is unattainable . And this is true with the exception of any trivial cases where input data is very small. And, by and large, few budgets will “pull” all possible checks that can cover our product.

    Black Box Testing

    A well-known method that does not require lengthy explanations. In short, black box testing is functional or non-functional testing that is performed without knowing the internal structure of the component or system. The method is based on working exclusively with the external interfaces of the system under test.

    Techniques of test design using the black box method include:

    • equivalence classes;
    • boundary value analysis;
    • decision tables;
    • state change diagrams;
    • pairwise testing.

    If we talk about pairwise testing, then there are studies that show that most of the bugs occur when a combination of 2 of any parameters. That is why in conditions of limited resources and with a huge amount of input data it makes sense to use pairwise testing.

    So what is pairwise testing?

    Pairwise testing is the development of black box tests, in which test scripts are designed to perform all possible separate combinations of each pair of input parameters. Strictly speaking, pair-wise testing allows us to save a lot of time.

    For pairwise testing, algorithms based on the construction of orthogonal arrays or the All-Pairs algorithm are used, which are based on theoretical studies in the field of combinatorial algorithms, discrete mathematics algorithms and, in particular, Latin squares. Let's look at these algorithms in more detail.

    Testing using an orthogonal array

    Orthogonal array testing is a systematic approach to testing all paired combinations of variables using orthogonal arrays. This approach significantly reduces the number of combinations of variables when checking all paired combinations.

    An orthogonal array is essentially a table, where m is the number of rows, n is the number of columns, which corresponds to the number of input parameters, k is the number of options for the values ​​of table elements. The table has the following properties:

    1. any two columns of the table contain all combinations of values ​​of these columns;
    2. if any pair of values ​​of two columns occurs several times, then all possible pairwise combinations of the values ​​of these columns should occur as many times.

    For example: - this is an orthogonal array with four rows and three columns (by the number of variables). The number 2 means that all variables take only two values ​​- 1 and 2.

    For example, our application has 3 input parameters, each binary (takes the value "1" or "2"). Thus, all possible combinations of input data can be represented as follows:

    For clarity, let's assume that we have the Flashlight application, which:

    • works with iOS and Android;
    • has a night and daylight mode;
    • Allows you to shine constantly or flash in strobe mode.

    In general, we have three parameters that take binary values.

    Now let's see what our sample will look like after transferring to an orthogonal array:

    As you can see, we have reduced the number of cases from 8 to 4 with 3 different parameters taking on a binary value. This is a real gain, which will positively affect both the budget and the resources that we use.

    All-Pairs Algorithm

    All-Pairs Algorithm (an algorithm for all pairs) is a combinatorial technique that was specially created for pair testing. It is based on the choice of possible combinations of the values ​​of all variables, which contain all possible values ​​for each pair of variables. Based on the definition, the number of combinations will be less than when using orthogonal arrays.

    When testing using the All-Pairs algorithm, perform the following steps:

    1. similarly as for orthogonal arrays, they define the table of all variables and their values;
    2. leave in the table only all possible unique combinations of pairs of variable values.

    It is also necessary to note the fact that orthogonal arrays and All-Pairs Algorithm are used with correction for preliminary sampling of the input data.

    Tails for pairwise testing

    In order not to waste time on reducing all data, you can and should use tools for pairwise testing. Here are some of them:

    1. is a site with a clear interface that does not require specific knowledge.
    2. PICT is a free tool developed by Microsoft for Pairwise Testing. Downloaded on the following link .

    Of course, there are Allpairs and VPTag, but talking about them went beyond the last webinar.

    Practice and practice again.

    Now let's try how these tools work in practice. So, try to complete two simple tasks:

    1. Using pairwiseTool, select the combinations of initial parameters and drop the link to the screenshot of the result in the comment. It takes a conditional site that should open on Win 7, Win 8 and Win 10. Supported browsers are Google Chrome, Opera, Microsoft Edge, Mozilla Firefox, Yandex Browser. Users may or may not use AdBlock.
    2. Using the PICT program, make a selection of combinations of initial parameters and discard the link to the screenshot of the result in the comments. It is necessary to carry out configuration testing with the following components:

    • video cards: GeForce GT 730, GeForce GT 1030, GeForce GTX 1080, GeForce RTX 2070;
    • Processors: Intel Core i5, Intel Core i7, AMD Ryzen 7, Intel Core i9;
    • Memory: 8GB, 16GB.

    On this, perhaps, everything. You can get acquainted with the nuances of pairwise testing by looking at the webinar in full. And, yes, do not miss the Open Day of the QA Specialist course!

    Also popular now: