Formulation of the problem

One of the integral parts of any ECM-system is the management of business processes, or workflow.

Business processes in each individual organization have many nuances. They are constantly changing due to changes within the organization, changes in legislation, etc. Therefore, it is cheaper and more logical to involve either analysts or programmers specializing in business logic in developing business processes. So, the creation and change of business processes should be as simple and convenient as possible.

Also, when a process changes, already running processes should work correctly. It is impossible to stop the long and complicated approval of the agreement only because now the agreed document should be printed not by the initiator of the agreement, but by the secretary.

This dictates some requirements that were presented to the business process engine:

• Processes should be developed based on high-level blocks. An example of such a block can be the creation of a task to coordinate a document, the start of a subtask, the execution of an arbitrary piece of code, etc.
• When changing the process scheme, it is necessary to ensure that already running processes can be converted to a new version of the scheme.

When developing a new version of the business process engine, we decided to try the Windows Workflow Foundation (hereinafter WF).

High-Level Block Development

Simplify business process development

Each high-level block of a route can consist of a large number of Activities (For example, for a task block, you need 68 activities). This is due to the fact that each block has several events in which handlers you can write code. Also, for each part of the block (events, internal logic of the block), error handling should work. This processing does the following: if an exception was thrown, then it is analyzed, and in some cases you need not to interrupt the process, but try again after a while. Moreover, the waiting time until the next attempt gradually increases from 5 minutes to 1 hour. This is necessary for situations when the operation could not be completed due to communication problems, SQL server timeout, etc.

Blocks could be made compound activities, but WF does not allow doing activities with multiple outgoing arrows. For example, the route block “Document approval task” should look like this:



But WF allows you to do just this:



Moreover, you still have to make a variable and pass the result of the task through it.
The second problem is blocks executed in parallel. The only way to do this in WF is to use the Parallel block. But then instead of the intuitive:



We get:


All this has led us to the fact that we do not have enough WF activities per se, we need a higher-order scheme that describes the route “from above”. When developing a route, our block classes (not related to WF in any way) are used, and only then the finished scheme is converted to Activity. Process diagrams are stored in XML, Activity is generated when the route is published to the application server. In addition to blocks, circuits contain connections between blocks (arrows from one block to another).

Convert blocks to Activity

For each block, there is a paired builder class that generates activity. It looks something like this:

public override System.Activities.Activity BuildContent()
{
  var result = new Variable(this.Block.ResultVariableName);
  return new Sequence()
   {
	 Variables =
	{
	  result
	},
	 Activities =
	{
	  new Assign
	  {
		To = new OutArgument(this.result), 
		Value = new InArgument(false)
	  },
	  //...          
	  new Persist()
	}
   };
}

We do not use composite activities in order not to have problems with conversion.

The only difficulty in converting the route described by our blocks is parallel branches. Such route branches are processed separately, then the result is combined into Parallel.

Converting an already running process to a new scheme

Convert to WF

The conversion of the WF process takes place in several stages:

1. For the old version of Activity, InstanceConverter.PrepareForUpdate is called. This call caches the current schema description in it itself.
2. Activity is being modified.
3. For a modified Activity, DynamicUpdateServices.CreateUpdateMap is called. This call creates UpdateMap - a map of changes, on the basis of which running instances of the scheme are converted.
4. When loading a saved instance, a change map is indicated in WorkflowApplication.

The main problem here is the inability to create UpdateMap based on two Activity. Those. if version 1 is deployed on one server, version 2 is deployed, version 3 is on the third, then upgrading to version 5 will be problematic. It will be even more difficult to upgrade the first server from version 1 to version 4.

How we solve the conversion problem

Schemas on the server are stored in the form of XML, in which our blocks lie, not activity. Thus, you need to convert from one version of our route view to another. It goes like this:

1. For the old version, Activity is built.
2. For the constructed Activity, InstanceConverter.PrepareForUpdate is called.
3. A diff is built between the old version of the route and the new one. It consists of added, deleted, modified blocks and links. To build the correct diff, each block and each connection has its own unique ID.
4. By this diff, the prepared Activity changes.
5. A map of changes is being built.
6. Each route instance is loaded with this change map and immediately unloaded. This is done immediately for all instances, so that the change map is used exactly once.

The change in Activity in step 4 occurs as follows: the activity generated for the block is packaged in FlowStep (if several arrows with conditions exit the block, then FlowDecision is generated after FlowStep). When changing / adding / removing links, the values ​​of the FlowStep.Next properties change.

Each block is stored in a variable in the circuit in serialized form. When changing the properties of a block, the default value of this variable changes.

When a block is added, the corresponding set of activities is generated and inserted at the desired location in the circuit. Removing a block is simply cleaning up the FlowStep.Next that leads to it.

Conversion when changing generated activities

In addition to changing the business process, conversion may be required when changing the activities generated for the block. For example, if you need to add new functionality to a block, or just fix a bug. We did it this way:
Each route scheme stores a version of the Activity generation algorithm.
When changing the logic of generating activity for a block, the version increases, and the converter learns to convert the activity of this block from the old version to the new one.
When converting a route, the converter converts the activity of blocks for which the generation logic has changed (determined by the version of the scheme).
The only feature is that the conversion should also take place in the form of a change in existing activities, and not generation from scratch, otherwise UpdateMap will not pick up.

Conclusion

After reading the article, it may seem that we used the Workflow Foundation in vain - this is not so. Thanks to the use of WF, we got out of the box hosting, storage of process instances, all the logic of process execution, including parallel.

The article describes only the solution to the problem of low-level WF. Behind the scenes were issues of hosting processes, problems of converting some sets of Activity, and much more.