How to help a manual tester. Automators rush to the rescue

In any team that devotes proper time to testing, the moment comes when the question is asked about the automation of this process. How does this happen? There are several ways for development: either testers themselves begin to automate, or a specially trained person is hired who, as a panacea, must solve all problems. Regardless of how this happens, in the end we all come across what needs to be shown somehow, what is happening, what is reality - what has been done. As one of my colleagues said, “automation for the sake of automation is akin to the cult of Cargo,” as it happens that the automation department exists, but there is no result.

So, the main task of an automation engineer is to make life easier. This time we are going to simplify our life by the manual testing department (if there is one) or by ourselves if the entire testing process is on our shoulders.

Some structural data


Our company has a manual testing department and automation. As a system for organizing tests, a fairly popular tool is used - TestRail. From my point of view, convenient and quite functional.
Automation is also built on a fairly standard set of Ruby + Cucumber + Watir / Selenium (you can mention the Page Object pattern) + TeamCity.

What happens when a new build is given for testing (in our case, each time we deal with regression)? The tester creates a new wound test, which includes all the test cases from a specific test suite and - the fun went off. I’m sure everyone knows that feeling when you manually run a regression or smoke on a machine for the 4th time by clicking / tapping on all the elements and setting the status for the next test. At this moment, everything is surely floating before the red eyes and the picture in the head repeats the famous:

image

And sometimes:

image

Right now we come to the rescue. It just so happened that an idea arose. If we have automation, then why are we still not using the results of our work to make life easier? The idea is to use a report with TestRail instead of bulky and obscure reports with Cucumber. A rather interesting task is to make the tests in TestRail themselves change their status, depending on how the autotest passed.

Through the search for omnipotent Google, documentation was found on the TestRail API to implement this very good purpose. To begin with, we decided to make sure that by launching our autotests all information about the current state of tests would be displayed in TestRail. Actually, the goal was achieved - by clicking on the test launch button in RubyMine we automatically created a new testran and sent the results to the server. Pretty simple, given the existing information on the TestRail website.

As it turned out, this was just the beginning.

In the end, we managed to make pretty good functionality, namely, we configured the integration of TestRail + TeamCity + Automation Framework.

Now the details, ladies and gentlemen.

Testral


The first stop we will have is TestRail.

TestRail is a software for managing data from testing. This tool helps you track processes, manage software, and organize a team.

With TestRail, you can create test cases, manage test suites, and coordinate the entire software testing process. TestRail provides an opportunity to increase productivity and get a complete overview of the testing process.

To begin with, the first thing our user / manual tester should do is to create a testran itself, which we will use later for updates.

Requirements:
Feature: Running autotests.
To use automation in real life.
As a user, I want a magic button “run wounds with autotests”.

image

No sooner said than done. The benefit of the functionality of TestRail allows us to integrate our own code.

As a result, we have just such a button:

image

Yes, yes - Start Tests.

Actually the code for this button.
name: Trigger tests
for run
description: Triggers automated tests
for a test run
author: Gurock Software
version: 1.0
includes: ^ runs / view
excludes:
    js:
    $(document).ready(
        function() {
            /* Create the button */
            var button = $('');
            /* Add it to the toolbar */
            //if ($('.toolbar-button.content-header-button.button-edit').length > 0) {
            $("#content-header .content-header-inner").prepend(button);
            //}
            /* Disable test run button */
            if (uiscripts.context.run.name.indexOf('in progress') >= 0 || (uiscripts.context.plan != undefined && uiscripts.context.plan.name.indexOf('in progress') >= 0)) {
                $("a", button).addClass('toolbar-button-disabled button-add-disabled');
            }
            /* Bind the click event to trigger the automated tests */
            $("a", button).click(
                function() {
                    if (!$("a", button).hasClass("button-add-disabled")) {
                        App.Dialogs.message(
                            'The tests are being processed in the background and the results are automatically posted back to TestRail.',
                            'Confirmation'
                        );
                        platform = uiscripts.context.run.name.split(" ")[0];
                        ventures = uiscripts.context.run.name.split(" ")[1];
                        if (platform == 'OMS') {
                            var teamcity_oms_build_trigger_url = 'http://TeamCityServer/httpAuth/action.html?add2Queue=BuildName&name=reverse.dep.*.test_run_id&value=' + uiscripts.context.run.id;
                        }
                        popup = window.open(teamcity_build_trigger_url, "windowName", "height=200,width=200");
                        setTimeout(function() {
                            popup.close();
                        }, 1000);
                        /* Change the test run/test plan name to disable button */
                        var api_url, new_data;
                        if (uiscripts.context.plan == undefined) {
                            api_url = uiscripts.env.page_base + '/api/v2/update_run/' + uiscripts.context.run.id;
                            new_data = JSON.stringify({
                                "name": uiscripts.context.run.name + ' (in progress)'
                            });
                        } else {
                            var entries = [];
                            $.ajax({
                                url: uiscripts.env.page_base + '/api/v2/get_plan/' + uiscripts.context.plan.id,
                                type: 'GET',
                                dataType: 'json',
                                contentType: "application/json; charset=utf-8",
                                data: new_data,
                                success: function(data) {
                                    entries = data.entries;
                                    var selected_entry;
                                    $.each(entries, function(index, entry) {
                                        if (entry.name == uiscripts.context.run.name) {
                                            return selected_entry = entry;
                                        }
                                    });
                                    api_url = uiscripts.env.page_base + '/api/v2/update_plan_entry/' + uiscripts.context.plan.id + '/' + selected_entry.id;
                                    new_data = JSON.stringify({
                                        "name": uiscripts.context.run.name + ' (in progress)'
                                    });
                                    $.ajax({
                                        url: api_url,
                                        type: 'POST',
                                        dataType: 'json',
                                        contentType: "application/json; charset=utf-8",
                                        data: new_data,
                                        success: function(data) {
                                            $("a", button).addClass('toolbar-button-disabled button-add-disabled');
                                        }
                                    });
                                }
                            });
                        };
                        $.ajax({
                            url: api_url,
                            type: 'POST',
                            dataType: 'json',
                            contentType: "application/json; charset=utf-8",
                            data: new_data,
                            success: function(data) {
                                $("a", button).addClass('toolbar-button-disabled button-add-disabled');
                            }
                        });
                    }
                }
            );
        }
    );


Since for one suite we have test cases for web and mobile, in the first stage we check by the name of the testran which framework is used. Depending on what the testran is intended for, we launch a build on TeamCity.

So that the user does not click too often on the button, we have organized protection against the fool - after clicking on the button we add the key “in progress” to the name of the testran, this blocks the magic button until our autotests finish their job.

Automation framework


At the final stage, a gem / library was created, which when deployed gives us integration with TestRail on any of our sub-projects.

Creating a gem is a completely different story, worthy of a separate article.

In short, our test_rail_integration library has a little functionality that we use at home, but it seems to me that someone can also be useful.

First you need to install it:

gem install test_rail_integration

Next add:

TestRail::Hook.update_test_rail(scenario)

In after | scenario | hooks. And in the env.rb file:


if ENV['TESTRAIL'] == '1'
  puts 'Option : TestRail [ON]'
  require "test_rail_integration"
  require 'test_rail_integration/generator/test_rail_hooks'
end

Here are the commands to run:

test_rail_integration check_test_run_and_update  

In our case, tests were run for 6 different locations and all results were displayed in 1 testran. Sometimes a situation arose when two updates came at the same time. It turned out that they did not see each other and after the fail came pass status. This option changed the overall picture of the status of the test. In general, this team passes all tests and checks that it matches that there are no red results in green tests. If there is, then it changes the status of the test to red.

test_rail_integration create_test_run         

Everything is simple, with this command we create a testran with the specified name in the project suite, which we recorded in the config file. The command returns the number of the created tesan:

test_rail_integration perform  

It is necessary to write at the first start, since this command generates a configuration file, to which you need to specify the necessary information about TestRail:

test_rail_integration shoot 

This is the core functional with flags:

--test_run_id

Here, and so everything is clear, you need to specify the test wound number:

--venture
and
--env

The specific attributes that will be used when compiling the command to run (if, for example, you need to run your tests through a specific command, then in the config file you need to write it using these variables, then at startup we must specify them):

--showroom

This flag is also specific and, I think, will no longer be useful to anyone:

--command

Here you can specify a new command for the current run:

--auto

Since we use all the integration for 6 localizations, by the testran number we look for the name and parse it on the parameters so that the testran should have a name, say, DT SG staging. Here we find the necessary information to run.

--simple

Running without searching and checking for any parameters, this is the most useful command for you, dear reader.

--type

You can also determine the type number of the tests that will be run.

The complete command to run tests with an existing testran will look like this:

test_rail_integration shoot --test_run_id 1 --simple

Or else:

test_rail_integration shoot --test_run_id 1 --simple --command  --type 3

Teamcity



Here, in fact, everything is simple. We simply write down the command in the same way as we would run it locally, unless you can add several variables for convenience and combine the order in which the commands are launched (in case we start the build automatically, you must first indicate the command to create the test, then run the tests with the number that came to us earlier).

That's all. Since this is my first experience writing an article, criticism is welcome. I hope that I wrote all this quite understandably and this manual will be useful to anyone.

Time for professional thanks
I would like to thank the following people for their help and guidance: Lyubov Shishova, Evgeny Pustovit, Cream Vasily, Dmitry Konovalov, Nikita Nikon, Igor Rozdobudko, Andrey Tolpeev and Alexis Grohar as well as my favorite team 24/7 Entertainment and IT Labs.

Link to the repository: github.com/Kirikami/test_rail_integration
Link to the library: rubygems.org/gems/test_rail_integration

Also popular now: