Transaction hell

In past articles, we have already mentioned transaction management in our platform . In this article, we’ll talk more about the implementation of transactions, their management, and more.

From the very beginning, we decided that the application server should support "transactional integrity." By this term, we mean that any call to the application server must either succeed, or all changes must be undone. Accordingly, at the start of processing a server call, a transaction is created (to be precise, it occurs upon the first change in the database) and is fixed (or canceled) when leaving the call:


At the same time, the application developer does not have direct access to the connection to the database, and all communication takes place through the corresponding layer. As a result, of course, an application developer may try to execute fixing requests (for example, execute a procedure in which truncate is executed), but such developers are never immune from a shot in the leg.
Yes, such an implementation, theoretically, increases the time and, as a consequence, the likelihood of excessive blocking. On practiceavoiding this is easy enough. Transactional integrity allows us to maintain data in a consistent state under conditions of frequent changes in a large (about 5MB) amount of source code (in the application part). In turn, its absence often leads to a violation of consistency, usually as a result of the following scenario:
Suppose we have some functionality implemented in the UpdateDocument function:

public void UpdateDocument(Document doc)
{
      try
      {
          // что-то делаем с документом и фиксируем транзакцию
          Database.Commit();
      }
      catch (Exception ex)
      {
          Log("Ошибка при обновлении документа: ", ex);
          Database.Rollback();
      }
}

Suppose we have a command on a document that calls the UpdateDocument method:

public void Command(Document doc)
{
    try
    {
       UpdateDocument(doc);
       // еще какие-то действия
       if (somethingWrong) 
       {
          throw new Exception("Случилось страшное!");
       }
       Database.Commit();
    }
    catch (Exception e)
    {
       Log("Ошибка в команде над документом: ", ex);
       Database.Rollback();
    }
}

when these 2 functions are written side by side and do not change, it is easy to notice a problem - after calling UpdateDocument in the Command function, part of the changes made before this call will be committed. If in the future something goes wrong, then only those changes will be rolled back that are made after calling UpdateDocument!
It’s less obvious that after calling UpdateDocument, a new transaction will begin. This has many consequences, for example, temporary tables will be empty, or the locks set before the changes are committed will be removed, and, therefore, in a new transaction, queries will return values ​​other than expected.
Errors of this kind are extremely difficult to diagnose, moreover, they can lead to gradual corrosion of data that no one will notice during the days, months, or even years when the problem is very serious.
That is why we have chosen an implementation option with transactional integrity. However, there are situations when it is necessary to do intermediate fixations. For example - one of the tasks in the solution deals with the "removal of reserves." In fact, it goes through the documents and changes them sequentially. By tradition, we will give how it was implemented in the first versions (all the code that was superfluous for understanding was deleted):

....
  foreach(docId in docs)
  {
     try
     {
         var doc = DocumentManager.GetDocument(docId);
         doc.state = DocStateCancelled;
         DocumentManager.Save(doc);
         Server.Commit(); 
      }
      catch(Exception e)
      {
         LogManager.Log("something is wrong");
      }
  }
...

Everything worked fine until the logic changed and the Save method did not start throwing an exception. This led to the fact that strange artifacts began to appear in the data, it was extremely difficult to determine their origin.
And this is what happened - at some iteration an exception was thrown, which was written to the log. And at the next, successful iteration, changes made before throwing an exception were recorded!
As a solution, it was proposed to abandon the possibility of gaining access to the current connection, but to provide the opportunity to request a new connection, for which it is already possible to commit changes. At the same time, all nested calls that do not explicitly request a new connection work with the connection received above the call stack.
The code looks like this:

....
  foreach(docId in docs)
  {
     using (var ts = new TransactionScope())
     {
        try
        {
            var doc = DocumentManager.GetDocument(docId);
            doc.state = DocStateCancelled;
            DocumentManager.Save(doc);
            ts.Complete();
        }
        catch(Exception e)
        {
           LogManager.Log("something is wrong");
        }
     }
  }
...

All calls (including nested) inside using use the transaction created in the CreateTransactionScope call. As you can see, the developer does not need to worry about getting a connection on his own, moreover, he does not have tools for this. The application developer in nested calls, if necessary, fixing the intermediate data can only request a new transaction. Thus, practically at the level of linguistic constructions, we are spared from intermediate fixations that lead to data corruption.

In conclusion

There are other options that lead to corrosion of data that can be combated with similar methods. We will try to talk about them in future articles.