Test Automation for Windows Applications Using .Net

Test Automation

Testing is an activity performed to evaluate and improve the quality of software. This activity, in general, is based on the detection of defects and problems in software systems.
Testing of software systems consists of dynamically verifying the behavior of programs on a finite (limited) set of tests, selected appropriately from the usually performed actions of the application area and providing verification of compliance with the expected behavior of the system.
The main approach for testing software is testing the black box. With this approach, the tester does not know the internal structure of the program. The tester interacts with the program: enters data, presses buttons, manipulates other visual components and evaluates the results.

Since usually full testing is impossible or difficult to implement, the task of the tester is to select the data and the sequence of actions on which errors of the tested program are most likely to occur.
When developing software, testers carry out a large number of identical tests during acceptance and regression testing. In this regard, testing automation can save a lot of man-hours. When conducting automated testing, the task of the tester is to develop a set of tests. With direct testing, testers can do more important and more intelligent tasks. In addition, large sets of test tasks can be run at night, which will intensify the development process and reduce project costs. In the morning, testers analyze the results, refine or recheck some tests and draw up the required reports for transmission to the project manager or developers.

.Net platform mechanisms

When developing Windows applications on the .Net platform, you can use the platform to create a lightweight and flexible tool that will run developed tests. It is not necessary to create a complete solution, sometimes it makes sense to quickly create a set of automated tests that will be set directly in the code. To quickly modify and add tests, you must create and use an assembly with various methods that will perform the actions necessary for the tester with the user interface of the program.
With automated testing of the user interface of Windows-based applications, two solutions are possible. The first of these is based on the Reflection mechanism.
The reflection mechanism allows you to get objects that describe assemblies, modules, and types. Reflection can be used to dynamically create an instance of a type, bind a type to an existing object, and also obtain a type from an existing object and call its methods or access its fields and properties.
The second approach is based on the low-level functions of the Win32 API libraries: FindWindow, FindWindowEx, SendMessage, PostMessage and the P / Invoke mechanism (invoking unmanaged code).
Unmanaged code invocation is a service that allows a managed program code to invoke unmanaged functions implemented in dynamic link libraries (DLLs). An unmanaged code call detects and calls an exported function.
The second approach works not only for .Net applications, but for any Windows applications with a user interface.

Test application

The application under test adds two integers and after clicking on the button displays the result. If the addition failed for some reason, the word “Error” is displayed instead of the result. Access to the code when testing the "black box" is not required. We can not use our code and get all the necessary information from the assembly and the running application. But when accessing the code, some procedures are simplified. The application is located at D: \ visual studio 2010 \ Projects \ WindowsFormsApplication1 \ WindowsFormsApplication1 \ bin \ Debug \ AUT.exe The name AUT is a common abbreviation for Application Under Testing.

Testing based on the reflection mechanism.

When using the reflection mechanism, it is necessary to load the assembly, get the type of the form, create an instance and run the application with this form in a new stream.
If the tester does not have access to the source code, then Red Gate's Reflector utility can be used. With its help, you can see the names of the form, controls and methods. This way you can find out that the form class is called Form1. The text fields are named textBox1 and textBox2, the button is button1, the result is output in Label with the name label1. The method of processing a button click is called button1_Click and accepts parameters of type object and EventArgs as input. To create an automated test of data, knowledge about the internal structure of the application is enough.





To increase code reuse, create a number of classes and methods.
The test application is launched using the StartApplication method.

static Form StartApplication(string path, string formName)
{
Form result = null;
Assembly a = Assembly.LoadFrom(path);
Type t = a.GetType(formName);
result = (Form)a.CreateInstance(t.FullName);
ApplicationState aps = new ApplicationState(result);
ThreadStart ts = new ThreadStart(aps.RunApp);
Thread thread = new Thread(ts);
thread.Start();
return result;
}

* This source code was highlighted with Source Code Highlighter.

The path to the assembly and the name of the form are passed to the input. The specified assembly is loaded, the type of the form is determined, an instance of the form is created, and using it the application is launched in a new thread: the delegate and the method to start in the new thread are specified.
Class ApplicationState.

class ApplicationState
{
public readonly Form formToRun;

public ApplicationState(Form f)
{
this.formToRun = f;
}

public void RunApp()
{
Application.Run(formToRun);
}
}

* This source code was highlighted with Source Code Highlighter.

For reuse, a number of methods, fields, and delegates must be defined.

static BindingFlags flags = BindingFlags.Public |
BindingFlags.NonPublic |
BindingFlags.Static |
BindingFlags.Instance;

delegate void SetControlPropertyValueHandler(Form f, string controlName, string propertyName, object newValue);

static void SetControlPropertyValue(Form f, string controlName, string propertyName, object newValue)
{
if (f.InvokeRequired)
{
f.Invoke(
new SetControlPropertyValueHandler(SetControlPropertyValue),
new object[] { f, controlName, propertyName, newValue }
);
}
else
{
Type t1 = f.GetType();
FieldInfo fi = t1.GetField(controlName, flags);
object ctrl = fi.GetValue(f);
Type t2 = ctrl.GetType();
PropertyInfo pi = t2.GetProperty(propertyName);
pi.SetValue(ctrl, newValue, null);
}
}
static AutoResetEvent are = new AutoResetEvent(false);
delegate void InvokeMethodHandler(Form f, string methodName, params object[] parms);
static void InvokeMethod(Form f, string methodName, params object[] parms)
{
if (f.InvokeRequired)
{
f.Invoke(
new InvokeMethodHandler(InvokeMethod),
new object[] { f, methodName, parms }
);
}
else
{
Type t = f.GetType();
MethodInfo mi = t.GetMethod(methodName, flags);
mi.Invoke(f, parms);
are.Set();
}
}
delegate object GetControlPropertyValueHandler(Form f, string controlName, string propertyName);
static object GetControlPropertyValue(Form f, string controlName, string propertyName)
{
if (f.InvokeRequired)
{
object iResult = f.Invoke(
new GetControlPropertyValueHandler(GetControlPropertyValue),
new object[] { f, controlName, propertyName }
);
return iResult;
}
else
{
Type t1 = f.GetType();
FieldInfo fi = t1.GetField(controlName, flags);
object ctrl = fi.GetValue(f);
Type t2 = ctrl.GetType();
PropertyInfo pi = t2.GetProperty(propertyName);
object gResult = pi.GetValue(ctrl, null);
return gResult;
}
}
delegate object GetControlHandler(Form f, string controlName);
static object GetControl(Form f, string controlName)
{
if (f.InvokeRequired)
{
object iCtrl = f.Invoke(
new GetControlHandler(GetControl),
new object[] { f, controlName }
);
return iCtrl;
}
else
{
Type t1 = f.GetType();
FieldInfo fi = t1.GetField(controlName, flags);
object gCtrl = fi.GetValue(f);
return gCtrl;
}
}
delegate object GetFormPropertyValueHandler(Form f, string propertyName);
static object GetFormPropertyValue(Form f, string propertyName)
{
if (f.InvokeRequired)
{
object iResult = f.Invoke(
new GetFormPropertyValueHandler(GetFormPropertyValue),
new object[] { f, propertyName }
);
return iResult;
}
else
{
Type t = f.GetType();
PropertyInfo pi = t.GetProperty(propertyName);
object gResult = pi.GetValue(f, null);
return gResult;
}
}
delegate void SetFormPropertyValueHandler(Form f, string propertyName, object newValue);
static void SetFormPropertyValue(Form f, string propertyName, object newValue)
{
if (f.InvokeRequired)
{
f.Invoke(
new SetFormPropertyValueHandler(SetFormPropertyValue),
new object[] { f, propertyName, newValue }
);
}
else
{
Type t = f.GetType();
PropertyInfo pi = t.GetProperty(propertyName);
pi.SetValue(f, newValue, null);
}
}

* This source code was highlighted with Source Code Highlighter.

The InvokeRequired property gets a value indicating whether the calling operator should call the invoke method during method calls from the control, since the calling operator is not in the stream in which the control was created [4].
InvokeMethod - Calls the method specified in the parameters with the specified parameters.
SetControlPropertyValue - sets the property of the specified control.
GetControlPropertyValue - Gets the value of the specified property of the control.
SetFormPropertyValue - Sets the specified property of the form.
GetFormPropertyValue - Gets the value of the form property.
GetControl - Gets the specified control.
Using these methods, you can create a testing application.

static void Main(string[] args)
{
string path = @"D:\visual studio 2010\Projects\WindowsFormsApplication1\WindowsFormsApplication1\bin\Debug\aut.exe";
string nameForm = "AUT.Form1";
Form myForm = StartApplication(path, nameForm);
string f1 = "521";
string f2 = "367";
SetControlPropertyValue(myForm, "textBox1", "Text", f1);
SetControlPropertyValue(myForm, "textBox2", "Text", f2);
object ctrl = GetControl(myForm, "button1");
InvokeMethod(myForm, "button1_Click", ctrl, EventArgs.Empty);
string res = GetControlPropertyValue(myForm, "label1", "Text").ToString();
string resTest = "FAIL";
if (res == "888") resTest = "PASS";
Console.WriteLine("{0} + {1} = {2}. Test {3}", f1, f2, res, resTest);
}

* This source code was highlighted with Source Code Highlighter.

The necessary values ​​are set in the text fields, after which the button1_Click method is called with the required parameters. After that, the value of the Text property of the control label1 is obtained and it is compared with the required one, after which information about the test is displayed on the screen.
Only a special case of using the reflection mechanism is presented; the tester can supplement the test suite.

Testing based on invoking unmanaged code.

For testing, a number of functions from the user32.dll library are used.
To import functions, use the DllImport attribute.

[DllImport("user32.dll", EntryPoint = "FindWindow", CharSet = CharSet.Auto)]
static extern IntPtr FindWindow(string lpClassName, string lpWindowName);

[DllImport("user32.dll", EntryPoint = "FindWindowEx", CharSet = CharSet.Auto)]
static extern IntPtr FindWindowEx(IntPtr hwndParent, IntPtr hwndChildAfter, string lpszClass, string lpszWindow);

[DllImport("user32.dll", EntryPoint = "SendMessage", CharSet = CharSet.Auto)]
static extern void SendMessage1(IntPtr hWnd, uint Msg, int wParam, int lParam);

[DllImport("user32.dll", EntryPoint = "PostMessage", CharSet = CharSet.Auto)]
static extern bool PostMessage1(IntPtr hWnd, uint Msg, int wParam, int lParam);

[DllImport("user32.dll", EntryPoint = "SendMessage", CharSet = CharSet.Auto)]
static extern int SendMessage3(IntPtr hWndControl, uint Msg, int wParam, byte[] lParam);

* This source code was highlighted with Source Code Highlighter.

These functions will allow you to find the main form handler, find the required controls, enter text, click on buttons and get the control value.
For reuse, we define a number of methods

static void ClickKey(IntPtr hControl, VKeys key)
{
PostMessage1(hControl, (int)WMessages.WM_KEYDOWN, (int)key, 0);
PostMessage1(hControl, (int)WMessages.WM_KEYUP, (int)key, 0);
}
static void ClickOn(IntPtr hControl)
{
PostMessage1(hControl, (int)WMessages.WM_LBUTTONDOWN, 0, 0); // button down
PostMessage1(hControl, (int)WMessages.WM_LBUTTONUP, 0, 0); // button up
}
static void SendChar(IntPtr hControl, char c)
{
uint WM_CHAR = 0x0102;
SendMessage1(hControl, WM_CHAR, c, 0);
}
static void SendChars(IntPtr hControl, string s)
{
foreach (char c in s)
{
SendChar(hControl, c);
}
}
static IntPtr FindWindowByIndex(IntPtr hwndParent, int index)
{
if (index == 0)
return hwndParent;
else
{
int ct = 0;
IntPtr result = IntPtr.Zero;
do
{
result = FindWindowEx(hwndParent, result, null, null);
if (result != IntPtr.Zero)
++ct;
} while (ct < index && result != IntPtr.Zero);
return result;
}
}
static List GetAllControls(IntPtr hwndParent)
{
IntPtr ctrl = IntPtr.Zero;
List res = new List();
ctrl = FindWindowEx(hwndParent, ctrl, null, null);
while (ctrl != IntPtr.Zero)
{
res.Add(ctrl);
ctrl = FindWindowEx(hwndParent, ctrl, null, null);
}
return res;
}
static IntPtr FindMainWindowHandle(string caption)
{
IntPtr mwh = IntPtr.Zero;
bool formFound = false;
int attempts = 0;
do
{
mwh = FindWindow(null, caption);
if (mwh == IntPtr.Zero)
{
Thread.Sleep(100);
++attempts;
}
else
{
formFound = true;
}
} while (!formFound && attempts < 25);
if (mwh != IntPtr.Zero)
return mwh;
else
throw new Exception("Could not find Main Window");
}

* This source code was highlighted with Source Code Highlighter.

The main task is to determine the order of controls. You can get a pointer to the control if its name / title / caption property is known. If this property is unknown, or it is not unique, you must use the FindWindowByIndex method. The zero control will be the window itself. The controls follow in the order of addition:

this.Controls.Add(this.label1);
this.Controls.Add(this.button1);
this.Controls.Add(this.textBox2);
this.Controls.Add(this.textBox1);

* This source code was highlighted with Source Code Highlighter.

But since it is assumed that the source code is unknown, you must use the SPY ++ utility. It shows the order of controls. The first number is Label, the second button, the third and fourth text fields. Based on this information, code is created to test the application.

static void Main(string[] args)
{
string path = @"D:\visual studio 2010\Projects\WindowsFormsApplication1\WindowsFormsApplication1\bin\Debug\aut.exe";
string nameForm = "AUT";
Process p = Process.Start(path);
//запускаем приложение
IntPtr mwh = FindMainWindowHandle(nameForm);
//получаем указатель на главное окно
IntPtr tb1 = FindWindowByIndex(mwh, 3);
IntPtr tb2 = FindWindowByIndex(mwh, 4);
//получаем указатели текстовых полей
string f1 = "521";
SendChars(tb1, f1);
string f2 ="367";
SendChars(tb2, f2);
//пишем туда
IntPtr btn = FindWindowByIndex(mwh, 2);
ClickOn(btn);
//получаем указатель на кнопку и нажимаем на нее
Thread.Sleep(150);
IntPtr lbl = FindWindowByIndex(mwh, 1);
uint WM_GETTEXT = 0x000D;
byte[] buffer = new byte[256];
string res = null;
int numFetched = SendMessage3(lbl, WM_GETTEXT, 256, buffer);
res = System.Text.Encoding.Unicode.GetString(buffer);
//получаем содержимое Label
string resTest = "FAIL";
if (res == "888") resTest = "PASS";
Console.WriteLine("{0} + {1} = {2}. Test {3}", f1, f2, res, resTest);
Thread.Sleep(3000);
p.CloseMainWindow();
p.Close();
}

* This source code was highlighted with Source Code Highlighter.

Conclusion

The development of a complete tool for automated testing was not the purpose of this article. This article only showed the mechanisms of the .Net platform that are used for automated testing. Outside of this article, questions remain related to storing test suites and saving results. The questions of choosing the test goal, data, oracle problems were not considered.

List of sources

1. swebok.sorlik.ru/4_software_testing.html IEEE Guide to Software Engineering Body of Knowledge, SWEBOK, 2004. Fundamentals of Software Engineering. Translation by Sergey Orlik.
2. Sam Kaner, Jack Folk, Young Kek Nguyen. Software testing. Fundamental business application management concepts. DiaSoft, 2001.
3. msdn.microsoft.com/en-us/library/ms173183.aspx - Reflection (C # and Visual Basic).
4. msdn.microsoft.com/en-us/library/system.windows.forms.control.invokerequired.aspx - .NET Framework class library. Property Control.InvokeRequired.
5. msdn.microsoft.com/en-us/library/26thfadc.aspx - Using unmanaged DLL functions.
6. James D. McCaffrey. Net test automation recipes: a problem-solution approach. books.google.com/books?id=3vN9zsMLvxkC
7. www.automatedtestinginstitute.com/home/index.php?option=com_content&task=view&id=1312&option=com_content&Itemid=1000 UI Automation Beneath the Presentation Layer. Bj Rollison Link to the magazine with the full version of the article. www.automatedtestinginstitute.com/home/ASTMagazine/2010/AutomatedSoftwareTestingMagazine_June2010.pdf
8. msdn.microsoft.com/en-us/magazine/cc163864.aspx Lightweight UI Test Automation with .NET. James McCaffrey.
9. habrahabr.ru/blogs/net/58582 NET in an unmanaged environment: platform invoke or what is LPTSTR.
10. PInvoke.net

Project source code