Asynchrony: why they won’t do it right?

Asynchronous programs are damn inconvenient to write. It is so inconvenient that even in node.js , declared as “everything is right-asynchronous,” they added the same synchronous analogs of asynchronous functions. What can we say about the Python syntax that does not allow declaring a lambda with any kind of complex code inside ...

It's funny that a beautiful solution to the problem does not require anything extraordinary, but for some reason it has not yet been implemented.

The essence of the problem

Let's say we have such a synchronous code:

var f = open(args);
checkConditions(f);
var result = readAll(f);
checkResult(result);

An asynchronous analogue will look much worse:

asyncOpen(args, function(error, f){
  if(error)
    throw error;
  checkConditions(f);
  asyncReadAll(f, function(error, result){
    if(error)
      throw error;
    checkResult(result);
  });
});

The longer the call chain, the worse the code.

Maybe you're not scared enough? Then try to write an analog of the following code, replacing all calls with asynchronous:

while(true)
{
  var result = getChunk(args1);
  while(needsPreprocessing(result))
  {
    result = preprocess(result);
    if(!result)
      result = obtainFallback(args2);
  }
  processResult(result);
}

And the point is not that it will take a lot of beech trees. The main ambush is the synchronous code above, it reads great. But in that asynchronous disgrace that you can do, even you yourself in a couple of hours just can’t figure it out.

By the way, the example is not unfounded. The distant similarity had to somehow be implemented on python, and it was in an asynchronous form.

Pickaxe and scrap solutions

You may have seen in node.js such a concept as the Promise . So, she is no more . The most common callbacks turned out to be much more humane. Therefore, I will not talk about Promise.

And I tell you about the library the Do . This library is based on the concept of continuables. Here is an example demonstrating the difference in approaches:

// callback-style
asyncFunc(args, function(error, result){
  if(error)
    throw error;
  doSomething(result);
});

// continuables-style
var continuable = continuableFunc(args);
continuable(function(result){ // callback
  doSomething(result);
}, function(error){ // errback
  throw error;
});

// continuables-style short
continuableFunc(args)(doSomething, errorHandler);

Continuable is a function that returns another function that takes callback and errback as parameters and makes an asynchronous call.

In some cases, this approach can significantly simplify the code - take a look at the “continuables-style short”. Here we use doSomething directly as a callback, since the function signature is suitable for us, and as errback we use some kind of “standard” errorHandler defined somewhere else.

Do can do a lot. Parallel calls, asynchronous map, some other interesting things. You can read more about this in the article " Do combo library ." There you can read about how to convert functions sharpened by callback-style (standard for node.js) to continuables-style.

However, back to the examples I started with. How can Do help in our case? Actually, with this:

Do.chain(
  continuableOpen(args),
  function(f){
    checkConditions(f);
    return continuableReadAll(f);
  }
)(function(result){
  checkResult(result);
}, errorHandler);

This is a continuables-style analogue of the very first example. Well, maybe a little better than callback-style, or maybe not. At least the growth of the indentation with the growth of the chain length is stopped, and the error handler is concentrated at one point. But the code looks scary, especially when compared to the original four-line synchronous version. A more complex example is the one with cycles - Do not at all tough, again you have to make a terrible garden.

yield hurries to the rescue

Pickaxe and crowbar did not help, I want something sublime. I would like the asynchronous call to be no more complicated than the synchronous one. And ideally - almost did not differ from him. And it is possible.

The solution infrastructure is best described by Ivan Sagalaev in the article " ADISP ". ADISP is a Python library he wrote that brings happiness.

Something similar can be collected on JS as well , Er.js serves as an example , but they pushed a lot of magic there for the first acquaintance, so I recommend Sagalaev’s article.

The approach used in ADISP allows you to write code in the following style:

var func = process(function(){
  while(true)
  {
    var result = yield getChunk(args1);
    while(yield needsPreprocessing(result))
    {
      result = yield preprocess(result);
      if(!result)
        result = yield obtainFallback(args2);
    }
    yield processResult(result);
  }
});

Yes, this is the same scary example with loops. All calls are asynchronous. The func wrapping function is provided only to show that it will have to be decorated. process - a decorator similar to that described by Sagalaev. getChunk, needsPreprocessing, preprocess, obtainFallback, processResult - asynchronous functions decorated by the async decorator in ADISP terminology.

The approach works wherever there is yield in the Python style. That is, an excellent asynchronous node.js in span since V8 does not yet support yield.

Native solution

Is there anything else needed when using the yield trick we can achieve such decent results? I think so, because:

- Using the yield keyword in the context of asynchronous calls looks strange. Still, this word is intended for several other things.
- The need to decorate the framing function is an inconvenience and an extra reason for error
. The code of the same ADISP, although not complicated, but to understand how this thing works, you need to pretty much break your brain. I somehow had to use ADISP in a slightly modified form. I ran into strange behavior and long and painfully delved into what was the matter. The cant turned out to be in a completely different place, but the chances of going crazy during debugging were more than real.

The yield example clearly shows that the runtime has everything for implementing asynchronous calls in a convenient, beautiful way. So much so that without even touching the internal mechanisms you can achieve the required. A legitimate question - why not provide a built-in, native solution, the implementation of which should not be too complicated?

In fact, the runtime should remember the context at the place of the asynchronous call and stop execution, and at the time the callback is called, simply restore everything in the right form and, if necessary, throw an exception. And she knows how to do it, at least potentially. The question is how to tell her when we need this trick.

Non-Blocking Library Functions

As a rule, asynchronous functions are implemented either in the core of the language (for example, setTimeout) or in library functions (for example, the functions of the fs module in node.js). Thus, the problem of beautiful asynchronous calls is primarily related to library functions.

This means a wonderful thing - beautiful asynchronous calls can be made by simply adding a special convention for asynchronous library functions. No need to change the language, no need to come up with a new keyword and puzzle over backward compatibility. Just give the library author a way to indicate that the asynchronous library function needs a way to bring back to life the context from which it was called, and the current execution must be stopped. Such functions can be safely used, for example, as follows:

while(true)
{
  doSomePeriodicTask();
  nbSleep(1000);
}

Here nbSleep is a non-blocking call to sleep that will actually interrupt execution at the call point and someday start it again from the same point, using the saved context as a callback.

Suppose we even have to have pairs of functions - one regular, with a callback (after all, in some cases the option with a callback is preferable), and the second is non-blocking. This is not scary, you can make a wrapper if you wish:

var asyncUnlink = fs.unlink;
fs.unlink = function(fName, callback){
  if(callback)
    return asyncUnlink(fName, callback);
  return nbUnlink(fName);
};

At the very least, synchronous analogs that can cause trouble can be thrown out in FIG, replacing their use with the use of non-blocking analogs.

The async keyword?

If we still want to add beautiful asynchronous calls at the language level, then, apparently, we can not do without a new keyword. It is clear that this is more of a fantasy: changing the language is too much freedom, in contrast to changing the execution environment. Nevertheless, let's take a look at one eye, what could happen:

var result1 = async(callback, myAsyncFunc(args, callback)); // long form
var result2 = async myAsyncFunc(args); // short form
var result3 = async(cb, createTask(args, cb), function(task){TaskManager.register(task);});

- Long form: callback is the name of the variable into which the return context will be stored for transfer to the asynchronous function
- Short form: the return context will be added with the last argument
- The third option is “turned inside out”: the return value will be passed to the lambda (register the created asynchronous "Task" in a certain "manager" - maybe we want to cancel it?), And return to the call point in the usual way for async

Why might you need language support? I think only if we need to do something like this with our tricky callback. For example, give it to several functions (oh, how vicious the practice will be). In most cases, support at the library function level should be enough. And certainly calls to non-blocking library functions will look better than the dominance of the async keyword.

The idea with a “turned inside out” call, by the way, is also applicable to non-blocking library functions, it is enough to pass a function to them for calling immediately before the completion of the current execution.

In the end

I hope that someday non-blocking library functions will be added to V8 and node.js, making them even more asynchronous and more beautiful. I hope that they are also added in Python. I hope that this will not stop and in all new and potentially favorite languages ​​and environments, instead of synchronous functions, there will be non-blocking functions - wherever it makes sense.

* Все исходники в этой статье подсвечены с помощью Source Code Highlighter.