Microsoft Moles Isolation Framework, digging deeper
As you understood from the name, we will talk about a product from Microsoft Research - Microsoft Moles Isolation Framework . I met him for the first time after reading the post of haberraiser alek_sys . I liked the moth so much that I decided to share my experience of its use.
First we’ll try to determine for what purpose Microsoft.Moles is intended and what we can achieve with it:
If you have the above goals in doubt, it’s worth a little refresh in the memory of the main points of unit testing. Those who have no doubts - you can skip to the next section .
At the moment, there are a large number of frameworks (NUnit, JUnit, DUnit, xUnit, MSTest) for many programming languages designed to create and use automatic tests for various purposes (Unit Tests, Integration Tests, Functional Tests, UI Tests). As a rule, they do not allow creating complete unit tests in a pure form, since they cannot ensure complete isolation of the tested code from the surrounding logic. Developers have to resort to various tricks, create a cloud of additional mock classes, use reflection, and so on. And here Isolation Frameworks come to the rescue, which allows you to conveniently and quickly enough, organize an isolated environment for unit tests, and Microsoft Moles is one of them.
I’ll try to demonstrate the possibilities that Moles provides us with an example (I tried to make it as simple as possible but at the same time complicated enough to justify using Moles in it).
Suppose we have the FileUpdater class, with the only UpdateFileFromService (string fileId) method, whose task is to update the file in local storage with the file downloaded from the remote storage, if the local and remote file hashes do not match.
As you can see from the listing, the UpdateFileFromService method uses the FileManager and StorageService classes to access local and remote repositories. I intentionally do not cite the implementation of the methods of the FileManager and StorageService classes, so we only need to know their interface.
The following versions of the UpdateFileFromService method are obvious :
Let's try to create tests for options 2 and 3 using Microsoft.Moles, since the implementation of the test for the first case is elementary using the standard tools of any test framework.
First of all, you need to generate a Moled assembly for libraries that contain classes whose logic we will replace with custom delegates (otherwise Stub methods). This can be done using the command line using mole.exe, which comes in the installation package with Microsoft.Moles, as well as through the Visual Studio 2008/2010 interface. We will use the second method.
As you can see, after installing Microsoft.Moles, a new context menu item “Add Moles Assembly” appeared for reference assemblies (Fig. 1).
After executing this command on the assembly ClassLibrary1, we get a new group of files ClassLibrary1.moles(Fig. 2), from which the ClassLibrary1.moles file is, is a Moled assembly descriptor that contains the parameters for its generation and which can be changed if necessary. The rest of the files are automatically regenerated at each build, and there is no point in editing them.
Microsoft.Moles allows you to use two types of substitute classes - Stubs & Moles.
The generated assembly ClassLibrary1.Moles.dll contains by default pairs of classes M% ClassName% and S% ClassName% for each of the classes that are contained in the assembly ClassLibrary1.dll.
Where:
If you wish, by making changes to the ClassLibrary1.moles file , you can ensure that Moles or Stubs are generated only for certain classes (which will greatly reduce the generation time), or you can disable the generation of either Moles or Stubs if you are not using it, as well as set other generation parameters (Intellisence will tell you a list of valid parameters and their purpose). You can also use the ClassLibrary1.moles file when generating using the moles.exe command line (you can find the list of moles.exe parameters by running moles.exe without parameters).
Since the use of Moles and Stubs is somewhat similar, and the article does not pretend to be a full description of all the functionality of Microsoft.Moles - we will dwell only on examples of using Mole.
To make the examples more clear, I immediately note the naming features of the members of the generated Mole and Stub classes - first comes the name of the member of the original class, and then the type names of the passed parameters are added to it. For example, MFileManager.AllInstances.GetFileHashString is the property to override the FileManager.GetFileHash (string fileId) method . I think this style is associated with the need to ensure the uniqueness of the members generated for overloaded methods.
With Mole we get the opportunity to replace any methods and properties of classes in various ways:
Obviously, in lambda expressions, you can perform all sorts of Assert checks of input values or simulate some
activity with the return of a predetermined result.
It is also worth noting the possibility of specifying the behavior of class members to whom we clearly do not specify a substitute delegate. Each Mole generated class has an InstanceBehavior property, which can take the following values
Here's the actual test situation when the hashes of the local and remote file are different.
For the second situation - when the hashes are identical, try to implement the test yourself.
I’ll finish this for now, there may be a continuation if this information turns out to be interesting.
Microsoft Moles Reference Manual
Using MS Moles with NUnit, NAnt and .NET 3.5
MSDN - The Moles Framework
What for?
First we’ll try to determine for what purpose Microsoft.Moles is intended and what we can achieve with it:
- Complete isolation of the tested logic from the external environment.
- The ability to quickly and easily create unit tests, and testing the logic of the class becomes possible even if there is no implementation of the classes whose user is the tested class.
- It becomes easy to organize test data sets or simulate the state of related objects to create test conditions
- At times, the unit test execution time is reduced, the frequent launch of tests becomes real
- Violation of unit logic does not entail the fall of hundreds or two of tests not designed for testing it
- Convenient method testing with complex workflow
About unit testing in general
If you have the above goals in doubt, it’s worth a little refresh in the memory of the main points of unit testing. Those who have no doubts - you can skip to the next section .
What is unit testing for?
- Isolate individual program units and show that individually a unit (or a set of several units) is operational.
- Documenting the logic of units - in order to understand how a unit functions - just read the code of its tests.
- Simplification of refactoring - you can easily verify that the code works without errors after making changes.
- Reduction of bugfix time due to the correction of most errors in the development process. Minimizing the possibility of regression.
Unit Test Requirements
- A test should test the minimum amount of class code, and the number of tests for a given code should be proportional to the number of code execution paths.
- The test implementation should be as simple as possible and not require initialization of the test environment outside the test (for example, preparation of test data in the database).
- The test must be self-documenting - an outside developer must understand the logic of the unit by reading the tests implemented for this unit. Without the need to examine the contents of the database or other sources of test data
- The test should be as simple as possible - it is much easier to understand the dozen simple tests implemented for each individual test case than in one test that tests all these test cases.
- The fall of the test should not be associated with logic for which it is not intended to be tested.
- The test should not depend on external systems (database server, web server, file system, external device controllers), if it is not designed to test interaction with these systems.
- Maximum code coverage with tests with minimal execution time required
What problems arise when using unit tests
- Difficulties in preparing a test environment - creating large test data sets, creating various connections to servers, describing configuration files, the need to create emulators of external devices, etc.
- The complexity of testing a particular class, because often the class is a user of other classes, and the presence of errors in their work entails a drop in the test, which does not correspond to the unit testing paradigm - the test should fall if the tested logic is violated.
- The difficulty of providing 100% coverage of the code with tests (as a rule, this is due to the laziness of the developer and the first two problems)
- The complexity of maintaining (debugging, fixing) tests that comes from the first two reasons.
- Inability to access (except reflection) to hidden members of classes, inability to initialize readonly members with test values.
- Often the test fall is due to the fact that the logic of another unit (and not the tested one) was violated. You have to spend time looking for errors in the wrong places.
- Test execution time - the main requirement for unit tests is their frequent execution - (made a change in the unit, drove the tests), but often it is impossible to do this because the tests run too long, for large projects this time can be measured in hours. What is most of the time occupied by processes not directly related to testing - reading / writing of the database, file system, web servers, etc., initialization of the associated logic, which is not directly related to the test.
Unit Testing Toolkit.
At the moment, there are a large number of frameworks (NUnit, JUnit, DUnit, xUnit, MSTest) for many programming languages designed to create and use automatic tests for various purposes (Unit Tests, Integration Tests, Functional Tests, UI Tests). As a rule, they do not allow creating complete unit tests in a pure form, since they cannot ensure complete isolation of the tested code from the surrounding logic. Developers have to resort to various tricks, create a cloud of additional mock classes, use reflection, and so on. And here Isolation Frameworks come to the rescue, which allows you to conveniently and quickly enough, organize an isolated environment for unit tests, and Microsoft Moles is one of them.
What can Microsoft.Moles do?
I’ll try to demonstrate the possibilities that Moles provides us with an example (I tried to make it as simple as possible but at the same time complicated enough to justify using Moles in it).
Suppose we have the FileUpdater class, with the only UpdateFileFromService (string fileId) method, whose task is to update the file in local storage with the file downloaded from the remote storage, if the local and remote file hashes do not match.
As you can see from the listing, the UpdateFileFromService method uses the FileManager and StorageService classes to access local and remote repositories. I intentionally do not cite the implementation of the methods of the FileManager and StorageService classes, so we only need to know their interface.
public class FileUpdater
{
private StorageService storageService;
public StorageService Service
{
get
{
if (storageService == null)
{
storageService = new StorageService();
}
return storageService;
}
}
public void UpdateFileFromService(string fileId)
{
if (string.IsNullOrEmpty(fileId))
{
throw new Exception("fileId is empty");
}
var fileManager = new FileManager();
string localFileHash = fileManager.GetFileHash(fileId);
string remoteFileHash = Service.GetFileHash(fileId);
if (localFileHash != remoteFileHash)
{
FileStream file = Service.DownloadFile(fileId);
fileManager.SaveFile(fileId, remoteFileHash, file);
}
}
}
public class FileManager
{
public string GetFileHash(string fileId)
{
throw new NotImplementedException();
}
public void SaveFile(string fileId, string remoteFileHash, FileStream file)
{
throw new NotImplementedException();
}
}
public class StorageService
{
public string GetFileHash(string fileId)
{
throw new NotImplementedException();
}
public FileStream DownloadFile(string fileId)
{
throw new NotImplementedException();
}
}
* This source code was highlighted with Source Code Highlighter.The following versions of the UpdateFileFromService method are obvious :
- fileId - empty string
- local and remote file hashes are not identical
- local and remote file hashes are identical
Let's try to create tests for options 2 and 3 using Microsoft.Moles, since the implementation of the test for the first case is elementary using the standard tools of any test framework.
Mole and Stub Class Generation
First of all, you need to generate a Moled assembly for libraries that contain classes whose logic we will replace with custom delegates (otherwise Stub methods). This can be done using the command line using mole.exe, which comes in the installation package with Microsoft.Moles, as well as through the Visual Studio 2008/2010 interface. We will use the second method.
As you can see, after installing Microsoft.Moles, a new context menu item “Add Moles Assembly” appeared for reference assemblies (Fig. 1).
After executing this command on the assembly ClassLibrary1, we get a new group of files ClassLibrary1.moles(Fig. 2), from which the ClassLibrary1.moles file is, is a Moled assembly descriptor that contains the parameters for its generation and which can be changed if necessary. The rest of the files are automatically regenerated at each build, and there is no point in editing them.
Microsoft.Moles allows you to use two types of substitute classes - Stubs & Moles.
- Stub provides a lightweight isolation framework that fake implementations of abstract, virtual methods, and member interfaces. Suitable for spoofing virtual methods or members that are implementations of interfaces
- Mole uses a powerful call redirection framework that uses code profiler APIs to intercept calls to used classes, which will redirect calls to a fake object. Allows you to redirect a call to any member of the class
The generated assembly ClassLibrary1.Moles.dll contains by default pairs of classes M% ClassName% and S% ClassName% for each of the classes that are contained in the assembly ClassLibrary1.dll.
Where:
- M% ClassName% - Mole class
- S% ClassName% - Stub class
If you wish, by making changes to the ClassLibrary1.moles file , you can ensure that Moles or Stubs are generated only for certain classes (which will greatly reduce the generation time), or you can disable the generation of either Moles or Stubs if you are not using it, as well as set other generation parameters (Intellisence will tell you a list of valid parameters and their purpose). You can also use the ClassLibrary1.moles file when generating using the moles.exe command line (you can find the list of moles.exe parameters by running moles.exe without parameters).
Using Mole Classes
Since the use of Moles and Stubs is somewhat similar, and the article does not pretend to be a full description of all the functionality of Microsoft.Moles - we will dwell only on examples of using Mole.
To make the examples more clear, I immediately note the naming features of the members of the generated Mole and Stub classes - first comes the name of the member of the original class, and then the type names of the passed parameters are added to it. For example, MFileManager.AllInstances.GetFileHashString is the property to override the FileManager.GetFileHash (string fileId) method . I think this style is associated with the need to ensure the uniqueness of the members generated for overloaded methods.
With Mole we get the opportunity to replace any methods and properties of classes in various ways:
- Replacing all instances of a class
MFileManager.AllInstances.GetFileHashString = (fileManager, fileId) =>
{
//данное лямбда выражение будет выполняться вместо метода GetFileHash(string fileId) всех инстансов класса FileManager
return Guid.NewGuid().ToString();
};
* This source code was highlighted with Source Code Highlighter.
Replacing static classes and static properties is done similarly, only without specifying AllInstances . - Replacing a specific instance of a class
var storageServiceMole = new MStorageService();
storageServiceMole.GetFileHashString = (fileId) =>
{
//данное лямбда выражение будет выполняться вместо метода GetFileHash(string fileId) обьекта storageService класса StorageService
return testRemoteHash;
};
var storageService = storageServiceMole.Instanc
* This source code was highlighted with Source Code Highlighter.
or so, with the transfer in the constructor of the initialized class objectvar storageService = new StorageService();
var storageServiceMole = new MStorageService(storageService);
storageServiceMole.GetFileHashString = (fileId) =>
{
//данное лямбда выражение будет выполняться вместо метода GetFileHash(string fileId) обьекта storageService класса StorageService
return testRemoteHash;
};
* This source code was highlighted with Source Code Highlighter. - Replacing the class constructor in order to overlap the necessary members of the Mole class with objects for all instances of the class.
MFileUpdater.Constructor = (@this) =>
{
var mole = new MFileUpdater(@this);
mole.ServiceGet = (x) => initializedServiceInstance;
};
* This source code was highlighted with Source Code Highlighter. - Substitution with further reference to the original implementation of the class. If you need to make some checks of the input values before calling the original class method, use the following method
var fileUpdaterMole = new MFileUpdater() { InstanceBehavior = MoleBehaviors.Fallthrough };
fileUpdaterMole.UpdateFileFromServiceString = (y) =>
{
fileUpdaterMole.UpdateFileFromServiceString = null;//отменяем перенаправление вызова на кастомный делегат
fileUpdaterMole.Instance.UpdateFileFromService(y);//теперь вызов будет обработан оригинальным методом FileUpdater.UpdateFileFromService
};
fileUpdaterMole.Instance.UpdateFileFromService(Guid.NewGuid().ToString());
* This source code was highlighted with Source Code Highlighter.
Obviously, in lambda expressions, you can perform all sorts of Assert checks of input values or simulate some
activity with the return of a predetermined result.
InstanceBehavior
It is also worth noting the possibility of specifying the behavior of class members to whom we clearly do not specify a substitute delegate. Each Mole generated class has an InstanceBehavior property, which can take the following values
- MoleBehaviors.DefaultValue - unsubstituted members of the class will be replaced by an empty delegate and return the default value of the type of the returned result
var storageService = new StorageService();
var storageServiceMole = new MStorageService(storageService) { InstanceBehavior = MoleBehaviors.DefaultValue };
//результатом следующего вызова будет null;
var result = storageService.GetFileHash(Guid.NewGuid().ToString());
* This source code was highlighted with Source Code Highlighter. - MoleBehaviors.NotImplemented - when accessing an unsubstituted member, a NotImplementedException will be thrown
var storageService = new StorageService();
var storageServiceMole = new MStorageService(storageService) { InstanceBehavior = MoleBehaviors.NotImplemented };
//следующий вызов породит исключение NotImplementedException
storageService.GetFileHash("328576BA-7345-4847-84AC-170EF03FFA7A");
* This source code was highlighted with Source Code Highlighter. - MoleBehaviors.Fallthrough - calls to unsubstituted members will be processed according to their original implementation in the replaced class
var storageService = new StorageService();
var storageServiceMole = new MStorageService() {InstanceBehavior = MoleBehaviors.Fallthrough};
//следующий вызов будет обработан согласно оригинальной реализации метода StorageService.GetFileHash(string fileId)
storageService.GetFileHash("328576BA-7345-4847-84AC-170EF03FFA7A");
* This source code was highlighted with Source Code Highlighter. - MoleBehaviors.Current - The default behavior of unsubstituted methods after the Mole class overrides the class with the class. When an unsubstituted member is called, a MoleNotImplementedException will be thrown
var storageService = new StorageService();
var storageServiceMole = new MStorageService() {InstanceBehavior = MoleBehaviors.Current};
//следующий вызов породит исключение MoleNotImplementedException
storageService.GetFileHash("328576BA-7345-4847-84AC-170EF03FFA7A");
* This source code was highlighted with Source Code Highlighter. - MoleBehaviors.CurrentProxy leaves the current behavior for all non-overlapping members of the overlapping class instance
var storageService = new StorageService();
var storageServiceMole1 = new MStorageService(storageService) { InstanceBehavior = MoleBehaviors.NotImplemented };
storageServiceMole1.GetFileHashString = (fileId) =>
{
return Guid.NewGuid().ToString();
};
var storageServiceMole2 = new MStorageService(storageService) { InstanceBehavior = MoleBehaviors.CurrentProxy };
storageServiceMole2.DownloadFileString = (fileId) =>
{
return testFileStream;
};
//следующий вызов будет перенаправлен обработчику storageServiceMole1.GetFileHashString
storageService.GetFileHash("328576BA-7345-4847-84AC-170EF03FFA7A");
//следующий вызов будет перенаправлен обработчику storageServiceMole2.GetFileHashString
storageService.DownloadFile("328576BA-7345-4847-84AC-170EF03FFA7A");
* This source code was highlighted with Source Code Highlighter. - Custom implementation of behavior - possible if you create your own implementation of the IMoleBehavior interface
Test implementation example
Here's the actual test situation when the hashes of the local and remote file are different.
///
/// Тест ситуации, когда хеши локального и удаленного файлов различны
///
[TestMethod]
[HostType("Moles")]
public void TestUpdateFileNonMatchedHash()
{
var callOrder = 0; // переменная для проверки очередности вызовов внутри тестируемого метода
var testFileId = Guid.NewGuid().ToString();
var testLocalHash = Guid.NewGuid().ToString();
var testRemoteHash = Guid.NewGuid().ToString();
var testFileStream = new FileStream(@"c:\testfile.txt", FileMode.OpenOrCreate);
var storageServiceMole = new MStorageService() { InstanceBehavior = MoleBehaviors.Fallthrough };
//Замещаем метод GetFileHash класса StorageService
storageServiceMole.GetFileHashString = (fileId) =>
{
Assert.AreEqual(1, callOrder++);//метод должен быть вызван вторым по очереди
Assert.AreEqual(testFileId, fileId);
Assert.AreNotEqual(testLocalHash, testRemoteHash);
return testRemoteHash;
};
storageServiceMole.DownloadFileString = (fileId) =>
{
Assert.AreEqual(2, callOrder++);
Assert.AreEqual(testFileId, fileId);
return testFileStream;
};
//замещаем методы класса FileManager.
//MFileManager.AllInstances используется потому что инстанс класса FileManager создается в контексте метода UpdateFile
//и у нас нет возможности получить доступ к созданному обьекту, поэтому перекрываем необходимые методы
//для всех инстансов класса FileManager
MFileManager.AllInstances.GetFileHashString = (fileManager, fileId) =>
{
Assert.AreEqual(0, callOrder++);
Assert.AreEqual(testFileId, fileId);
return Guid.NewGuid().ToString();
};
MFileManager.AllInstances.SaveFileStringStringFileStream = (fileManager, fileId, fileHash, fileStream) =>
{
Assert.AreEqual(3, callOrder++);
Assert.AreEqual(testFileId, fileId);
Assert.AreEqual(testRemoteHash, fileHash);
Assert.AreSame(testFileStream, fileStream);
};
var fileUpdaterMole = new MFileUpdater
{
InstanceBehavior = MoleBehaviors.Fallthrough,
//Замещаем getter свойства FileUpdater.Service своим делегатом, который будет возвращать инициализированный ранее moled инстанс класса StorageService
ServiceGet = () => storageServiceMole.Instance
};
var fileUpdater = fileUpdaterMole.Instance;
fileUpdater.UpdateFileFromService(testFileId);
}
* This source code was highlighted with Source Code Highlighter.For the second situation - when the hashes are identical, try to implement the test yourself.
additional information
- Mole types cannot be used in multithreaded tests.
- Moles cannot be used to replace some types of mscorlib.
- Tests using Microsoft.Moles must be instrumented by Moles - for MSTest, this is an attribute of the HostType ("Moles") test method . For other test frameworks, it is necessary to run tests using moles.runner.exe (for nunit it will look like moles.runner.exe "yourTestDLLName" /runner:nunit-console.exe ) Accordingly, fans who work with the test framework through the GUI will also have to run shell through mole.exe
- Example configuration of Nant tasks for invoking tests that are instrumented by moles
* This source code was highlighted with Source Code Highlighter.
I’ll finish this for now, there may be a continuation if this information turns out to be interesting.
Sources
Microsoft Moles Reference Manual
Using MS Moles with NUnit, NAnt and .NET 3.5
MSDN - The Moles Framework