Rewriting Require.js using Promise. Part 1

In order to not have problems with dependencies and modules with a large number of browser javascript, usually require.js is used . Also, many people know that this is just one of many AMD standard bootloaders , and it has alternatives. But few people know how they are arranged inside. In fact, writing such a tool is not difficult, and in this article we will write our own version of the AMD bootloader step by step. At the same time, we’ll deal with Promise , which recently appeared in browsers and will help us deal with asynchronous operations.

The basis of require.js is the functionrequire(dependencies, callback). The first argument is the list of modules to load, and the second is the function that will be called at the end of loading, with the modules in the arguments. Using Promise to write it is not difficult at all:

function require(deps, factory) {
  return Promise.all(deps.map(function(dependency) {
    if(!modules[dependency]) {
      modules[dependency] = loadScript(dependency);
    }
    return modules[dependency];
  }).then(function(modules) {
    return factory.apply(null, modules);
  });
}

Of course, this is not all, but there is a basis. Therefore, we continue.

Loading modules

Our first function will be the script load function:

function loadScript(name) {
  return new Promise(function(resolve, reject) {
    var el = document.createElement("script");
    el.onload = resolve;
    el.onerror = reject;
    el.async = true;
    el.src = './' + name + '.js';
    document.getElementsByTagName('body')[0].appendChild(el);
  });
}

Downloading occurs asynchronously, so we will return a Promise object, by which we can find out about the end.
The Promise constructor accepts a function in which an asynchronous process will occur. Two arguments are passed to it - resolveand reject. These are functions for reporting the result. In case of success, we will call resolve, and in case of error - reject.
To subscribe to a result, a Promise instance has a then method. But we may need to wait for several modules to load. And for this there is a special aggregator function Promise.allthat will collect several promise into one, and its result, if successful, will be an array of the results of loading all the necessary modules. With these two simple functions, you can already get a minimally working prototype.

The github repository contains tags for the key steps in this article. At the end of each chapter there is a link to github, where you can see the full version of the code written for this step. In addition, in the test folder are tests that show that our functionality works as it should. Travis-CI is connected to the project , which performed tests for each step.

See the code for this step on Github .

Modules Declaration

In fact, we never loaded our modules. The thing is that when we add a script to a page, we lose control over it and don’t learn anything about the result of its work. Therefore, when we load module A, they may slip module B to us, but we will not notice it. To prevent this from happening, you need to give the modules the opportunity to introduce themselves. For this, the AMD standard has a function define(). For example, registering module A looks like this:

define('A', function() {
  return 'module A';
});

When the module introduced itself by name, we will not confuse it with anything and can mark it as loaded. To do this, we need to define a module registrar - a function define. At the last step, we simply waited for the script to load successfully and did not check its contents. Now we will wait for the call define. And at the time of the call, we will find our module and mark it as loaded.

To do this, we need to create a module blank at the start of loading, which can turn into a real module after loading. This can be done using deferred objects. They are close to Promise, but it hides resolve and reject inward, and from the outside it gives only the opportunity to know the result. Deferred objects have resolve and reject methods, that is, having access to deferred, you can easily change its result. Also the deferred object has a fieldpromisein which Promise is written, the result of which we set. Deferred are easily made from Promise Prescription with Stackoverflow .

When loading modules into require, we will create a deferred object for each module and store it in the cache (pendingModules).
definehe can get it out of there and call resolve to mark it as loaded and save it.

function define(name, factory) {
  var module = factory();
  if(pendingModules[name]) {
    pendingModules[name].resolve(module);
    delete pendingModules[name];
  } else {
    modules[name] = module;
  }
}

Also, sometimes it is necessary to register a module before it is asked. Then it will not be in the pendingModules list, in this case we can immediately put it in modules.

The loadScript function will now store deferred objects in the cache and return a promise for this object, by which the require function will wait for the module to load.

function loadScript(name) {
  var deferred = defer(),
    el = document.createElement("script");
  pendingModules[name] = deferred;
  el.onerror = deferred.reject;
  el.async = true;
  el.src = './' + name + '.js';
  document.getElementsByTagName('body')[0].appendChild(el);
  return deferred.promise;
}

See the code for this step on Github .

Dependencies in modules, loop detection

Hooray, now we can load modules. But sometimes a module may need other modules to work. For this, AMD has one more argument
for the function define- dependencieswhich can be between name and factory.
When a module has dependencies, we cannot just take and call factory, we need to load the dependencies first.
Fortunately, for this we already have a require function, here it will come just in place. Where previously there was just a challenge factory()now it will be require:

define(name, deps, factory) {
  ...
-  var module = factory();
+  var module = require(deps, factory);
  ...
}

It is worth noting that now in the variable there modulewill be no module, but the promise of the module. When we pass it to resolve, the promise of the source module will not be fulfilled, but will now wait for the dependencies to load. This is a rather convenient feature of Promise, when our asynchronous process is stretched into several stages, we can resolve Promise from the next stage to resolve, and the internal logic recognizes this and switches to waiting for the result from the new Promise.

When we load module dependencies, danger lies in wait for us. For example, module A depends on B, and B depends on A. Load module A, it will require module B. After loading B, it will require A, and as a result, they will wait for each other indefinitely. The situation may be worse if the chain does not have two modules, but more. It is necessary to be able to stop such cyclic dependencies. To do this, we will save the download history to show a warning when we notice that our download went in a circle. We used require()to load module dependencies, but this function has a fixed set of arguments prescribed in the standard, it must be observed. Let's create our own internal function _require(deps, factory, path), with which we can transfer information about the module loading history, and in the public API we will make its call:

function require(deps, factory) {
  return _require(deps, factory, []);
}

At first, our boot history will be empty, so we will pass an empty array as path. There _require()will now be the same loading logic, plus history tracking.

function loadScript(name, path) {
  var deferred = defer();
+  deferred.path = path.concat(name);
  ...
}
function _require(deps, factory, path) {
  return Promise.all(deps.map(function (dependency) {
+    if(path.indexOf(dependency) > -1) {
+      return Promise.reject(new Error('Circular dependency: '+path.concat(dependency).join(' -> ')));
+    }
  ...
}

A global array with a list of all modules will not work for us, the loading history of each module has its own, we will save it in a deferred object of the loaded module, so that it can then be read into defineand passed in _requireif you need to load more modules. I note that we add a new module to the story through .concat(), instead .push(), because we need an independent copy of the story so as not to spoil the story to other modules that were loaded before us. And instead of the usual, throw new Error()we return Promise.reject (). This means that the promise has not been fulfilled, and an error handler is called, just as it does with an error during script loading, only the message indicates another reason - the cycle in dependencies.

See the code for this step on Github .

Error processing

It is time to implement bug reporting for users. The require function also has a third argument - the function that is called in case of an error. Promise may inform us of an error if .then()we pass in two functions. The first one is already transmitted and called if everything is fine, the second one will be called if something goes wrong.

We will call an additional argument errback, as in the original require.js

function _require(deps, factory, errback, path) {
  ...
  })).then(function (modules) {
    return factory.apply(null, modules);
+  }, function(reason) {
+    if(typeof errback === 'function') {
+      errback(reason);
+    } else {
+      console.error(reason);
+    }
+    return Promise.reject(reason);
  });
}

In case the user does not care about errors, we will do it ourselves, display a message in the console. Also, it is no accident that in the error handler we return this value. The promise logic is designed so that if we pass the function in case of an error, then it believes that we will fix everything in it and that we can continue to work, similar to the try-catch block.
But for require.js, the loss of a module is fatal, we will not be able to continue working without all the modules, pass the error on using Promise.reject.

See the code for this step on Github .

Anonymous modules

The AMD standard provides the ability to define modules without a name. In this case, its name is determined by the script that is now loaded on the page. The property document.currentScriptis not supported by all browsers, so we will have to determine the current script in a different way. We will make the loading of modules sequential, which means that we will expect only one module at a time. Using Promise, you can easily get the implementation of a FIFO queue:

var lastTask = Promise.resolve(),
    currentContext;
function invokeLater(context, fn) {
  lastTask = lastTask.then(function() {
      currentContext = context;
      return fn();
  }).then(function() {
    currentContext = null;
  });
}

We always keep a promise from the last operation, the next operation will subscribe to its completion and leave a new promise.
We will use this queue to load scripts. Now we have not a list pendingModules, but one pendingModule, and the rest will wait.

function loadScript(name, path) {
  var deferred = defer();
  deferred.name = name;
  deferred.path = path.concat(name);
  invokeLater(deferred, function() {
    return new Promise(function(resolve, reject) {
      //прежний код загрузки скрипта
    });
  });
  return deferred.promise;
}

The function still returns the pending module, but it does not start loading immediately, but in the order of the queue. And the name of the module is added to deferred to know which module we will be waiting for. And now we can write a fairly short define:

define(function() { return 'module-content'; });

And the module will get the name by the name of its file, by which we refer to it, so you can not specify it separately.

See the code for this step on Github .

Delayed Initialization

When we meet define, this does not mean that it needs to be initialized immediately. Maybe no one has asked him yet and he may not be required. Therefore, it can be saved along with information about its dependencies and called only when it is really useful. It will also be useful if the modules can be declared in any order, and their dependencies will be analyzed at the very end, during application initialization.

We will get a separate object predefines, and we will save modules into it if no one requested them.

function define(name, deps, factory) {
  ...
  } else {
-    modules[name] = _require(deps, factory, null, []);
+    predefines[name] = [deps, factory, null]; 
  }
}

And in time requirewe will first check predefinesfor the modules that interest us

function _require(deps, factory, errback, path) {
  ...
+  var newPath = path.concat(dependency);
+  if(predefines[dependency]) {
+      modules[dependency] = _require.apply(null, predefines[dependency].concat([newPath]))
+  }
  else if (!modules[dependency]) {
      modules[dependency] = loadScript(dependency, newPath);
  }
  ...
}

Such optimization will avoid unnecessary queries when we define modules in advance.

define('A', ['B'], function() {});
define('B', function() {});
require(['A'], function() {});

Previously, in the first step, we would have started loading the 'B' module from somewhere, and would not have found it. And now we can wait until all the modules are declared themselves, and only then call require. Also now the order of their announcement does not matter.
It is enough that the entry point (call to require) comes last.

See the code for this step on Github .

Thus, we have already received a complete solution for loading modules and resolving dependencies. But require.js allows you to do a lot more with modules. Therefore, in the next part of the article we will add support for settings and plugins, and we will make this functionality fully compatible with the original require.js.