November 19, 2015 at 11:08

Winium: now for Windows Phone


    There are no convenient and open tools for testing automation under Windows Phone and Windows that can be easily adapted to your needs. Those that are are closed, limited and offer their own approach, which differs from generally accepted standards like Selenium WebDriver.

    My colleague skyline-gleb recently wrote in Habré how we developed our selenium-like tool for automating the functional testing of desktop applications for Windows. In parallel, we developed a similar tool, only for Microsoft's mobile platforms.

    About why they started writing it, what benefits have come from using a single platform for testing automation for all mobile platforms and how to implement it on your project, read this article.

    A bit of history


    • October 2010. Windows Phone 7 was released. A year later, Expensify is posting WindowsPhoneTestFramework , a BDD testing tool for native applications, in open source ;
    • October 2012, Windows Phone 8 is released. Microsoft has not yet released a testing tool through the UI;
    • February - March 2014, we post the first prototype of WinphoneDrvier, the first open-source implementation of Selenium for native Silverlight-applications for Windows Phone;
    • In April 2014, Microsoft released Windows Phone 8.1 and finally, almost 4 years later, released the official toolkit for testing applications for Windows Phone through the UI - CodedUI. But, of course, this is not a Selenium-compatible tool and is available only in the most expensive Visual Studio subscriptions;
    • In May 2014, Salesforce.com released open source windowsphonedriver , a Selenium implementation for testing web applications on Windows Phone. Around the same time, we updated our driver to support Windows 8.1;
    • In February 2015, we launched Winium.StoreApps, an updated version of winphonedriver, which implements a significant part of the protocol commands and supports native StoreApps applications for Windows Phone 8.1. This is the driver we use in our processes.

    Just after this, we presented our toolkit at Codefest 2015 , where from the discussion on the sidelines with Sathish Gogineni from Badoo the idea of ​​Winium CodedUi was born - an implementation of the Selenium driver based on CodedUI that supports native and hybrid applications and, most importantly, testing on devices.

    At the beginning of the project, there was one open tool - Expensify / WindowsPhoneTestFramework, which did not suit us because it was incompatible with Selenium and had a non-standard API. Moreover, he was imprisoned under BDD. During the development of the project, Microsoft managed to release its version of the toolkit - CodedUI, which again had its own non-standard API, was sharpened for writing tests in Visual Studio in C #, was closed and paid (which does not scale well).

    So, we remembered how events developed. Back to Winium. Since the above tools did not suit us, we decided to write our own. So the Winium project was born , which began as a testing automation tool for Windows Phone Silverlight applications and soon turned into a whole set of testing automation tools for the Windows platform:


    About Winium.Desktop and Winium.Cruciatus we already talked about Today we will talk about Winium for Store Apps (the successor to Windows Phone Driver) and Winium.StoreApps.CodedUi.

    Winium.StoreApps


    Key features and driver restrictions

    Winium.StoreApps is the main driver implementation for mobile devices. We use it and develop it. Source code is open and available at Github .

    Key features:

    • implements Selenium protocol for testing native StoreApps applications for Windows Phone platform;
    • works with Json Wire Protocol. You can use selenium or appium binders and write tests in the same way as for iOS or Android;
    • supports installation and launch of the tested application, as well as downloading files to the local application storage;
    • supports single-touch gestures;
    • a basic inspector is provided to view the UI tree of the application under test;
    • supports Selenium Grid, which allows you to parallelize the execution of tests.

    Limitations:

    • only emulators are supported (although with minor changes, the driver can install the application on the device and work with it, but so far we do not know how to fully simulate gestures and input on devices);
    • you need to embed the automation server in your application (that is, connect the nuget package to the application and add one line of code that starts the server on a separate thread. Yes, this violates the first rule of Appium, but so far it has not been found better than CodedUI );
    • only one session is supported, but the driver can be connected to the Grid to distribute tests and run them in parallel.

    Winium.StoreApps supports all major Selenium commands and can be integrated into an existing testing infrastructure built on the basis of Selenium / Appium. In general, it can already be actively used in a continuous process, which we are doing.

    How does it all work

    In essence, Winium.StoreApps.Driver.exe is an HTTP server that runs on the JsonWire / WebDriver REST protocol. Driver commands are proxied if necessary to the InnerServer test server, which is embedded in the application under test.


    The structure of the interaction between the tests, the driver and the application under test.

    How to prepare an application and write tests

    To run tests against our application, you need to perform three simple steps:

    • prepare the application;
    • write tests (Ok, this is not so easy.);
    • run tests.

    Preparing the application

    Here everything is simple: connect nuget-pack Winium.StoreApps.InnerServer and initialize automation server on the main stream after the established UI. For example, in the MainPageOnLoaded method. The server is initialized on the main thread only to get the correct dispatcher. The server will operate on a different thread, except for direct access to the UI.

    After that, it would be nice to ensure testability of the application by writing down names and identifiers for those elements that you plan to use in tests, but this is not necessary.

    That's all, it remains only to collect the appx-package with your application.

    Writing tests

    Tests are written in the same way as for the web or mobile devices using selenium or appium binders.

    The first thing we need to do is create a new session. When creating a session, we can specify various desired properties. Here are a few basic ones supported by the driver (a complete list is available on the wiki ).

    dc = {
    'deviceName': 'Emulator 8.1 WVGA 4 inch 512MB',
    'app': r'C:\YorAppUnderTest.appx',
    'files': {
        {
            'C:\AppFiles\file1.png': 'download\file1.png',
            'C:\AppFiles\file2.png': 'download\file2.png'
        }
    },
    'debugConnectToRunningApp': False
}

    • deviceName is the partial name of the device on which we want to run our tests. If empty, the first emulator from the list will be selected.
    • app - the full path to the appx package with the application under test in which you integrated the automation server.
    • files - a dictionary of files that will be downloaded from the local disk to the local storage of the application.
    • debugConnectToRunningApp - allows you to skip all the steps of installing and downloading files to the application and connect to an already running application. This is very convenient when you run the application from Visual Studio, set breakpoints there and want to debug the error that occurs in the application when running one of the tests.

    So, the session was created, the application was launched. Now we need to somehow find the elements if we want to interact with them. The driver supports the following locators:

    • id - AutomationProperties.AutomationId;
    • name - AutomationProperties.Name;
    • class name - the full name of the class (Windows.UI.Xaml.Controls.TextBlock);
    • tag name - same as class name;
    • xname - x: Name, a legacy locator, is usually not supported by default binders and requires refinement of binders for use, but it allows you to search by name, which is usually assigned to an element for access from code.

    To facilitate the search for locators and a general understanding of the structure of the UI of the application under test, we made an inspector who can connect to the application and display the current state of the UI, as the driver sees it.


    The inspector's main window The inspector

    can do a little, but you can already see the screenshot, the UI tree, as the driver sees it, find out the element locators and their basic properties: position, visibility, text.

    So, we found the element, now we can do anything with it.

    # можно текст элемента запросить
element.text
# а можно и кликнуть в элемент
element.click()
# можно ещё ввести что-то в элемент
element.send_keys('Hello!'+Keys.ENTER)
# можно запросить публичное свойство по имени
element.get_attribute('Width')
# можно запросить вложенное свойство
element.get_attribute('DesiredSize.Width')
# а можно запросить сложные свойста, драйвер попробует сериализовать их в JSON
element.get_attribute('DesiredSize')
# '{"Width":300.0,"Height":114.0,"IsEmpty":false}'

    In the world of cell phones, an ordinary click can not do, gestures are needed here. We support the old and proven API from JSWP (but will add support for the new Mobile WebDriver API soon). Already now you can make flicks and scrolls.

    TouchActions(driver).flick_element(element, 0, 500, 100).perform()
TouchActions(driver).scroll(200,200).perform()
# можно даже создавать ваши собственные жесты
ActionChains(driver) \
.click_and_hold() \
.move_by_offset(100, 100) \
.release().perform()

    And since we are implementing the automation server in the application, we can do more interesting things. For example, invoke the MS Acessability API commands:

    # direct use of Windows Phone automation APIs
app_bar_button = driver.find_element_by_id('GoAppBarButton')
driver.execute_script('automation: invoke', app_bar_button)
list_box = driver.find_element_by_id('MyListBox')
si = {'v': 'smallIncrement', 'count': 10}
driver.execute_script('automation: scroll', list_box, si)

    This allows you to use precise means of scrolling elements, if necessary, instead of simulating gestures.
    You can also set the value of a public property, although we do not recommend doing this in tests:

    text_box = driver.find_element_by_id('MyTextBox')
driver.execute_script('attribute: set', text_box, 'Width', 10)
driver.execute_script('attribute: set', text_box, 'Background.Opacity', 0.3)

    But there are situations when it is justified. For example, in our tests - in order to quickly and accurately move the map to the desired position, we just use this API instead of moving the map with gestures.

    This is not a complete list of commands supported by the driver. For a more detailed list and notes on commands, see the wiki .

    So, let's put together a simple test based on all these commands:

    Simple test code
    # coding: utf-8
import os
from selenium.webdriver import Remote
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
class TestAutoSuggestBox(object):
    def setup_method(self, _):
        executor = "http://localhost:{}".format(os.environ.get('WINIUM_PORT', 9999))
        self.driver = Remote(command_executor=executor,
                             desired_capabilities={'app': 'aut.appx'})
    def test_select_suggest(self, waiter):
        self.driver.execute_script("mobile: OnScreenKeyboard.Disable")
        pivots = self.driver.find_elements_by_class_name("Windows.UI.Xaml.Controls.Primitives.PivotHeaderItem")
        pivots[1].click()
        autosuggestion_box = waiter.until(EC.presence_of_element_located((By.ID, 'MySuggestBox')))
        autosuggestion_input = autosuggestion_box.find_element_by_class_name('Windows.UI.Xaml.Controls.TextBox')
        autosuggestion_input.send_keys('A')
        suggestions_list = waiter.until(EC.presence_of_element_located((By.XNAME, 'SuggestionsList')))
        suggestions = suggestions_list.find_elements_by_class_name('Windows.UI.Xaml.Controls.TextBlock')
        expected_text = 'A2'
        for suggest in suggestions:
            if suggest.text == expected_text:
                suggest.click()
                break
        assert expected_text == autosuggestion_input.text
    def teardown_method(self, _):
        self.driver.quit()


    In this example, we are creating a new session. We hide the on-screen keyboard (for demonstration and just not to interfere).

    Switch to the second tab in the pivot element. We find there an input field with prompts and begin to enter text. After that, select one of the prompts in the list and check that the value in the input field matches the prompt value. And finally, close the session.

    Running tests

    The simplest remains. We run Winium.StoreApps.Driver.exe (which can be downloaded from Github ), run the tests with your favorite tester, and enjoy the magic.

    Demo .

    Winium codedui


    The main features and limitations of the driver. After Codefest 2015, we got the idea to create a prototype selenium driver that wraps CodedUI. The idea was brought to life and is now available on Github .

    Key features:

    • does not require modification of the tested application. You can even test pre-installed or downloaded from the store applications;
    • It works on emulators and devices;
    • compatible with Json Wire Protocol;
    • supports native applications;
    • already has limited support for hybrid applications.

    Limitations:

    • early prototype, so some stability issues are possible;
    • a license is required for Visual Studio Premium or higher (for 2013, for 2015 - Business);
    • one session (but we port multi-session from StoreApps whenever possible).

    How does it all work

    Everything works on the same principles as in StoreApps, but now instead of embedding the server in the application, we start the server as a separate background process through vs.test.console and CodedUI. This test server has access to the phone and UI of running applications directly through the Accessibility API (for example, to search for items) and through the CodedUI API (for example, for gestures).


    The structure of the interaction between the tests, the driver and the application under test.

    How to prepare an application and write tests

    Since in this approach it is not necessary to modify the tested application, it is possible to test both release versions of applications and pre-installed applications. Those. This version of the driver is as close as possible to the philosophy of Appium . This is both a plus and a minus - because it imposes restrictions on access to certain internals of the application.
    No special application preparation is required. Tests are written and run the same way as for Winium.StoreApps.

    Watch the demo video in which we automate the creation of events in the standard predefined Calendar application.

    Example code
    from time import sleep
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.wait import WebDriverWait
def find_element(driver, by, value):
    """
    :rtype: selenium.webdriver.remote.webelement.WebElement
    """
    return WebDriverWait(driver, 5).until(expected_conditions.presence_of_element_located((by, value)))
winium_driver = webdriver.Remote(
    command_executor='http://localhost:9999',
    desired_capabilities={
        'deviceName': 'Emulator',
        'locale': 'en-US',
    })
# AutomationId for tiles can not be used to find tile directly,
# but can be used to launch apps by switching to window
# Actula tile_id is very very very long
# {36F9FA1C-FDAD-4CF0-99EC-C03771ED741A}:x36f9fa1cyfdady4cf0y99ecyc03771ed741ax:Microsoft.MSCalendar_8wekyb3d8bbwe!x36f9fa1cyfdady4cf0y99ecyc03771ed741ax
# but all we care about is part after last colon
winium_driver.switch_to.window('_:_:Microsoft.MSCalendar_8wekyb3d8bbwe!x36f9fa1cyfdady4cf0y99ecyc03771ed741ax')
# accept permisson alert if any
try:
    accept_btn = winium_driver.find_element_by_name("allow")
    accept_btn.click()
except NoSuchElementException:
    pass
# now we are in calendar app
new_btn = find_element(winium_driver, By.NAME, "new")
new_btn.click()
sleep(1)  # it all happens fast, lets add sleeps
subject = find_element(winium_driver, By.ID, "EditCardSubjectFieldSimplified")
subject.send_keys(u'Winium Coded UI Demo')
sleep(1)
# we should have searched for LocationFiled using name or something, but Calendar app uses slightly different
# classes for location filed in 8.1 and 8.1 Update, searching by class works on both
location = winium_driver.find_elements_by_class_name('TextBox')[1]
location.send_keys(u'Your computer')
sleep(1)
save_btn = find_element(winium_driver, By.NAME, "save")
save_btn.click()
sleep(2)
winium_driver.close()
winium_driver.quit()


    What Winium Gave Us


    What gave us the creation of this tool instead of using platform-specific?

    Now all our mobile teams use uniform tools, which allows us to easily share our experience and build a common toolkit and infrastructure to run our tests and not to scatter the attention on the platforms. In particular, one experienced engineer was able to easily switch from iOS automation to Windows Phone, share experience and train testing engineers, which significantly raised the level of the project.

    From an infrastructure point of view, this allowed us to focus on creating one tool (vmmaster) to provide a repeatable environment for on-demand testing, but this is a topic for another article.

    But most importantly, this allowed us to begin combining our tests for different platforms into one project ( demo ).

    In short, now we have ...

    • fewer bicycles;
    • less code duplication;
    • support is easier;
    • Better code quality
    • “Sharing” of knowledge;
    • systems approach;
    • faster development.

    And of course, this is all opensource, so you can also use this tool to automate testing of your Windows Phone applications. Moreover, Winium.StoreApps can be used completely free of charge by downloading the latest release and installing emulators or Visual Studio Community with a mobile SDK. But for Winium.CodedUi you will need a paid version of Visual Studio Premium or higher.

    Once again, a link to the repository and other opensource 2GIS products .
    Tags:

    Also popular now: