Automate Flex applications with Java + Selenium + FlexMonkium
In this post I will describe my research, which I had to do when the question about test automation was raised on the project. The project is a website, the client part of which is written in Flex, and the server part in Python (Django).
To begin, consider the path from the beginning, which I went before I settled on a bunch of Java + Selenium + FlexMonkium. And this:
The first thing I tried to use as automation tools was Sfapi (Selenium Flex-API). It works in conjunction with Selenium RC and some programming language for which there are drivers (Java, C #, Python, Ruby). Of the benefits, it should be noted that this is a free solution that really works. Of the shortcomings - a lot of code, but not enough result. It works poorly with dynamic elements of the application. The reason for the further search for solutions was large amounts of code that performed simple operations.
The next Flex automation software I tried was Ranorex Studio. This is a very user-friendly software package with an intuitive interface. Of the pluses, it is worth noting the rapid creation of tests without knowing the programming language using the built-in functionality and Spy agent (which records the sequence of user actions), doping up already created tests by converting them to any of the proposed programming languages (the list of supported languages depends on the type of license purchased ) and good interaction with almost all Flex UI-elements (on the forms I met complaints about incorrect work with the DataGrid element, but during the work I didn’t have a chance to work with it ). Among the disadvantages is the high cost of the license, if you need to test the application on different domains (for example, test and main server), it is necessary to make some transformations with the tests first so that they correctly start interacting with the application interface. It was the high cost of the license that served as the reason for the further search for solutions.
Without waiting for the end of the Ranorex license, I continued to search for free software and at one of the forums I came across advice on using FlexMonkey. On their website there is short, but understandable documentation, there is also a forum with active discussions and live support. The application itself is written in Adobe Air, has a clear and user-friendly interface.
As it turned out, FlexMonkey is very fast, convenient and free. It correctly supports work with almost all UI components (at least everything worked for me with a bang), but it also has one drawback - tests cannot be improved with any programming language. In fact, you can use only basic functions, such as checking the properties of components, comparing the values of these properties, etc. But, for example, writing messages in the mail user interface without programming tools will not work. This is where the next product from the creators of FlexMonkey comes to the rescue, which works in conjunction with Selenium and is called FlexMonkium .
In order to start testing automation of your flex application, you need to configure the project to interact with Selenium RC + FlexMonkium, as well as prepare the programming environment (for example, I chose the Java language with the Eclipse IDE). Further, all actions are described in steps:
1. Download Selenium RC and Selenium Client Driver for the programming language in which the tests will be written (in my case, Java).
2. Download FlexMonkium(the example used version 4.1.8). The archive contains three components - a folder with .swc libraries, one of which will need to be connected to the Flex project, “user-extensions.js”, which is an extension for the Selenium server, thanks to which it will “understand” the flex-application control commands, specified in tests and .xpi plugin that extends the functionality of Selenium IDE for Firefox.
3. Open the Flex project in Flash Builder and connect the .swc library from the downloaded archive to the project. There are several such files in the archive with the same name and number at the end. The number means the version number of the Flex-SDK (in my case, this is the fourth version, that is, you need to connect automation_monkey4.x.swc). After that, we recompile the project and voila - it is ready to interact with the Selenium server.
4. Now you need to start the Selenium server with the command extension file, which will allow it to "understand" the commands specific to managing the flex application. This file is contained in the downloaded FlexMonkium archive and is called "user-extensions.js". Passed as a parameter on the command line at server startup. For these purposes, I created a separate bat-file with the following contents:
After that, you can safely start the server. If everything was done correctly, after starting the bat'nik, the following window will hang on the screen:
So, the application under test is ready to interact with the testing automation tools, and Selenium-RC is launched and understands the flex component management commands.
As a language for writing tests, I chose Java with the Eclipse development environment in conjunction with the JUnit library. With this combination, the tests worked fine, but JUnit restarted the browser after each test, which was not advisable for me, as many of my tests were a continuation of the previous ones, and resetting the application state did not suit me. After some forum searches, I read that this behavior cannot be disabled and I had to abandon the use of JUnit. In the report, I only needed to know which tests failed, at what steps this happened and how many such errors occurred. For these purposes, I wrote a special function that displays this information to the console. It is called when the test fails (when the test fails, it has a timeout) and Java throws an exception.
In this example, sending mail by the application’s internal mail is tested, as well as checking its display among outgoing emails.
And now in more detail. The seleniumStart and seleniumStop functions start and end a session. The first function initializes an instance of the Selenium class, which sends commands to the server. Parameters such as the host and port (on which the Selenium server is running), the browser (in which testing will be carried out), and the url (under which the tested application is available) are transferred to it.
Next comes the “printError” function, which takes the test name and step number as parameters and displays them in the console. This function is called when a timeout is reached at any step in the test.
In order not to write a block of repeating code in each step, it was moved to a separate function called “testExec”. Its parameters are the name and number of the step to be performed (which are transmitted to “printError” after the timeout), the command for the server, the path (locator) to the element in the application that we are trying to “reach out to”, and the timeout. Please note that the application path in the above example has 2 types: “VerifyProperty” and “UIEvent”. The first is used to get \ compare the attribute values of an element, and the second to perform actions on the specified element, for example, click on the button. You can specify the UI element of the application not only by its name or id, but also by the combination of several of its properties, which are additionally indicated in the "arg" tags. More detailed documentation is available on the website.Gorilla Logic .
In order not to write the locators manually, I wrote down the sequence of steps through FlexMonkey, exported the tests to an xml file and inserted them into Java code. The result is this result:
Or you can install the flexmonkium.xpi plugin in addition to the Selenium IDE and record tests directly from the Firefox browser.
But in any case, these 2 methods do not guarantee 100% accurate test performance, especially when application components have a dynamically generated ID. In this case, you will have to manually add additional arguments that are explicitly inherent in the required component. Component properties can be viewed in FlexMonkey or FlexMonkium Console.
At the moment, most of the regression tests are automated on the project. Their number was> 350, the runtime of which is about 3-4 hours.
I apologize for such a large piece of code in the example that inflated the length of the article.
The choice of tools for automation
To begin, consider the path from the beginning, which I went before I settled on a bunch of Java + Selenium + FlexMonkium. And this:
- Selenium + sfapi
- Ranorex
- Flexmonkey
- Selenium + FlexMonkium
The first thing I tried to use as automation tools was Sfapi (Selenium Flex-API). It works in conjunction with Selenium RC and some programming language for which there are drivers (Java, C #, Python, Ruby). Of the benefits, it should be noted that this is a free solution that really works. Of the shortcomings - a lot of code, but not enough result. It works poorly with dynamic elements of the application. The reason for the further search for solutions was large amounts of code that performed simple operations.
The next Flex automation software I tried was Ranorex Studio. This is a very user-friendly software package with an intuitive interface. Of the pluses, it is worth noting the rapid creation of tests without knowing the programming language using the built-in functionality and Spy agent (which records the sequence of user actions), doping up already created tests by converting them to any of the proposed programming languages (the list of supported languages depends on the type of license purchased ) and good interaction with almost all Flex UI-elements (on the forms I met complaints about incorrect work with the DataGrid element, but during the work I didn’t have a chance to work with it ). Among the disadvantages is the high cost of the license, if you need to test the application on different domains (for example, test and main server), it is necessary to make some transformations with the tests first so that they correctly start interacting with the application interface. It was the high cost of the license that served as the reason for the further search for solutions.
Without waiting for the end of the Ranorex license, I continued to search for free software and at one of the forums I came across advice on using FlexMonkey. On their website there is short, but understandable documentation, there is also a forum with active discussions and live support. The application itself is written in Adobe Air, has a clear and user-friendly interface.
As it turned out, FlexMonkey is very fast, convenient and free. It correctly supports work with almost all UI components (at least everything worked for me with a bang), but it also has one drawback - tests cannot be improved with any programming language. In fact, you can use only basic functions, such as checking the properties of components, comparing the values of these properties, etc. But, for example, writing messages in the mail user interface without programming tools will not work. This is where the next product from the creators of FlexMonkey comes to the rescue, which works in conjunction with Selenium and is called FlexMonkium .
Setting up the work environment
In order to start testing automation of your flex application, you need to configure the project to interact with Selenium RC + FlexMonkium, as well as prepare the programming environment (for example, I chose the Java language with the Eclipse IDE). Further, all actions are described in steps:
1. Download Selenium RC and Selenium Client Driver for the programming language in which the tests will be written (in my case, Java).
2. Download FlexMonkium(the example used version 4.1.8). The archive contains three components - a folder with .swc libraries, one of which will need to be connected to the Flex project, “user-extensions.js”, which is an extension for the Selenium server, thanks to which it will “understand” the flex-application control commands, specified in tests and .xpi plugin that extends the functionality of Selenium IDE for Firefox.
3. Open the Flex project in Flash Builder and connect the .swc library from the downloaded archive to the project. There are several such files in the archive with the same name and number at the end. The number means the version number of the Flex-SDK (in my case, this is the fourth version, that is, you need to connect automation_monkey4.x.swc). After that, we recompile the project and voila - it is ready to interact with the Selenium server.
4. Now you need to start the Selenium server with the command extension file, which will allow it to "understand" the commands specific to managing the flex application. This file is contained in the downloaded FlexMonkium archive and is called "user-extensions.js". Passed as a parameter on the command line at server startup. For these purposes, I created a separate bat-file with the following contents:
java -jar C:\Automation\selenium-server-standalone-2.0rc2.jar -userExtensions C:\user-extensions.jsAfter that, you can safely start the server. If everything was done correctly, after starting the bat'nik, the following window will hang on the screen:
So, the application under test is ready to interact with the testing automation tools, and Selenium-RC is launched and understands the flex component management commands.
Create the first test
As a language for writing tests, I chose Java with the Eclipse development environment in conjunction with the JUnit library. With this combination, the tests worked fine, but JUnit restarted the browser after each test, which was not advisable for me, as many of my tests were a continuation of the previous ones, and resetting the application state did not suit me. After some forum searches, I read that this behavior cannot be disabled and I had to abandon the use of JUnit. In the report, I only needed to know which tests failed, at what steps this happened and how many such errors occurred. For these purposes, I wrote a special function that displays this information to the console. It is called when the test fails (when the test fails, it has a timeout) and Java throws an exception.
import java.awt.event.KeyEvent;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.HttpCommandProcessor;
import com.thoughtworks.selenium.Selenium;
public class TestRunner {
private Selenium selenium;
private HttpCommandProcessor proc;
private int errorsFound;
//Генерируем уникальный текст письма что бы не путаться
DateFormat dateFormat = new SimpleDateFormat("yyyy_MM_dd_HH_ss");
Date date = new Date();
String msg_text = dateFormat.format(date);
public static void main(String[] args) {
TestRunner tr= new TestRunner();
tr.errorsFound = 0;
try{
tr.seleniumStart();
//----Тестируем почту----
tr.goMail();
tr.postSend();
tr.outBoxVerifyMessage();
}catch(Exception ex) {}
}
//Starting selenium server
public void seleniumStart() throws InterruptedException
{
proc = new HttpCommandProcessor("localhost", 4444,
"*firefox3 c:\\Program Files\\MozillaFirefox3.6\\firefox.exe", "http://q/");
selenium = new DefaultSelenium(proc);
selenium.start();
Thread.sleep(5000);
selenium.open("/site_media/");
}
//Shutting down selenium server
public void seleniumStop()
{
if (selenium != null) {
selenium.stop();
selenium = null;
}
System.out.println("\r\nTesting compleated. Errors found: "+errorsFound);
}
//Error reporter
public void printError(String test, String step)
{
errorsFound ++;
System.out.println("\r\nError in Test: '"+test+"' on step: '"+step+"'");
}
//Step runner
public void testExec(String testName, String step, String cmd, String locator,
int time) throws Exception
{
for (int t = 0;; t++) {
if (t >= time) printError(testName, "Step "+step);
try {
if (proc.getBoolean(cmd, new String[] {locator})) break;
} catch (Exception e) { }
Thread.sleep(500);
}
}
//-----------------------------------------------------------------------------------
////////////////////////////////////////ПОЧТА////////////////////////////////////////
//-----------------------------------------------------------------------------------
//Переходим на страницу почты
public void goMail() throws Exception {
String test = "goMail";
//Step 1. Ждем загрузки флешки
testExec(test, "1", "isFlexMonkey", "", 60);
//Step 2. Кликаем на кнопку "Почта"
testExec(test, "2", "isFlexMonkey",
"", 60);
//Step 9. Пишем текст письма
testExec(test, "9", "isFlexMonkey",
"", 60);
}
//-----------------------------------------------------------------------------------------------
//Проверка отправленного письма в исходящих
public void outBoxVerifyMessage() throws Exception {
String test = "outBoxVerifyMessage";
//Step 1. Переходим в исходящие
testExec(test, "1", "isFlexMonkey",
"", 10);
//Step 3. Проверяем текст
testExec(test, "3", "isFlexMonkey",
"", 10);
//Step 4. Проверяем получателя
testExec(test, "4", "isFlexMonkey",
"", 10);
}
}
In this example, sending mail by the application’s internal mail is tested, as well as checking its display among outgoing emails.
And now in more detail. The seleniumStart and seleniumStop functions start and end a session. The first function initializes an instance of the Selenium class, which sends commands to the server. Parameters such as the host and port (on which the Selenium server is running), the browser (in which testing will be carried out), and the url (under which the tested application is available) are transferred to it.
Next comes the “printError” function, which takes the test name and step number as parameters and displays them in the console. This function is called when a timeout is reached at any step in the test.
In order not to write a block of repeating code in each step, it was moved to a separate function called “testExec”. Its parameters are the name and number of the step to be performed (which are transmitted to “printError” after the timeout), the command for the server, the path (locator) to the element in the application that we are trying to “reach out to”, and the timeout. Please note that the application path in the above example has 2 types: “VerifyProperty” and “UIEvent”. The first is used to get \ compare the attribute values of an element, and the second to perform actions on the specified element, for example, click on the button. You can specify the UI element of the application not only by its name or id, but also by the combination of several of its properties, which are additionally indicated in the "arg" tags. More detailed documentation is available on the website.Gorilla Logic .
In order not to write the locators manually, I wrote down the sequence of steps through FlexMonkey, exported the tests to an xml file and inserted them into Java code. The result is this result:
Or you can install the flexmonkium.xpi plugin in addition to the Selenium IDE and record tests directly from the Firefox browser.
But in any case, these 2 methods do not guarantee 100% accurate test performance, especially when application components have a dynamically generated ID. In this case, you will have to manually add additional arguments that are explicitly inherent in the required component. Component properties can be viewed in FlexMonkey or FlexMonkium Console.
Result
At the moment, most of the regression tests are automated on the project. Their number was> 350, the runtime of which is about 3-4 hours.
I apologize for such a large piece of code in the example that inflated the length of the article.