Support for HTTP / 2 Server Push technology in Node.js

In July 2017, Node.js 8 introduced an HTTP / 2 implementation. Since then, it has gone through several stages of improvements, and now the Node.js Foundation says that they are almost ready to bring HTTP / 2 support out of the category of experimental capabilities. If you want to try HTTP / 2 in Node.js environment, it is best to do it using Node.js 9 - here you can find all the latest bug fixes and improvements. The material, the translation of which we publish today, is devoted to working with HTTP / 2, and, in particular, with Server Push, in Node.js.

The basics

In order to test HTTP / 2, the easiest way is to use the compatibility layer, which is part of the new kernel module http2:

const http2 = require('http2');
const options = {
 key: getKeySomehow(),
 cert: getCertSomehow()
};
// Здесь необходим https, иначе браузер не сможет
// установить соединение с сервером
const server = http2.createSecureServer(options, (req, res) => {
 res.end('Hello World!');
});
server.listen(3000);

The compatibility layer provides the same high-level API (request listener with familiar objects requestand response) that you can use by connecting the module to the project with a httpcommand require(‘http’). This makes it easy to translate existing projects to HTTP / 2.

The compatibility layer also provides a convenient way to migrate to HTTP / 2 framework authors. So, the Restify and Fastify libraries already support HTTP / 2 using the HTTP / 2 compatibility layer Node.js.

Fastify is a new web framework that is performance-oriented and designed to make it convenient for programmers to work with. It has a rich ecosystem of plugins. Recently released its version 1.0.0.

Using HTTP / 2 with is fastifyquite simple:

const Fastify = require('fastify');
// Здесь необходим https, иначе браузер не сможет
// установить соединение с сервером
const fastify = Fastify({
 http2: true
 https: {
   key: getKeySomehow(),
   cert: getCertSomehow()
 }
});
fastify.get('/fastify', async (request, reply) => {
 return 'Hello World!';
});
server.listen(3000);

While the ability to run the same application code on top of HTTP / 1.1 and on top of HTTP / 2 is important during the implementation phase of the protocol, the compatibility layer, by itself, does not provide access to some of the most powerful HTTP / 2 features. The kernel module http2allows you to work with these additional features through the new kernel API ( Http2Stream ), which can be accessed through the stream listener:

const http2 = require('http2');
const options = {
 key: getKeySomehow(),
 cert: getCertSomehow()
};
// Здесь необходим https, иначе браузер не сможет
// установить соединение с сервером
const server = http2.createSecureServer(options);
server.on('stream', (stream, headers) => {
 // stream - это дуплексный поток
 // headers - это объект, содержащий заголовки запроса
 // команда respond отправит заголовки клиенту
 // мета-заголовки начинаются со знака двоеточия (:)
 stream.respond({ ':status': 200 });
 // тут, кроме того, доступны команды stream.respondWithFile()
 // и stream.pushStream()
 stream.end('Hello World!');
});
server.listen(3000);

Fastify can be accessed Http2Streamthrough the API request.raw.stream. It looks like this:

fastify.get('/fastify', async (request, reply) => {
 request.raw.stream.pushStream({
  ':path': '/a/resource'
 }, function (err, stream) {
  if (err) {
    request.log.warn(err);
    return
  }
  stream.respond({ ':status': 200 });
  stream.end('content');
 });
 return 'Hello World!';
});

HTTP / 2 Server Push - Features and Challenges

Compared to HTTP / 1, HTTP / 2 provides, in many cases, a huge performance improvement. Server Push technology is one of the features of HTTP / 2 that is relevant.

Here's how, simplified, a typical HTTP session looks like.


Hacker News Session

1. The browser requests an HTML document from the server
2. The server processes the request and sends the document to the browser, possibly pre-generating it.
3. The browser receives the server response and parses the HTML document.
4. The browser identifies the resources needed to output the HTML document, such as style sheets, images, JavaScript files, and so on. The browser then sends requests to receive these resources.
5. The server responds to each request, sending the browser what it requested.
6. The browser renders the page using HTML code and related resources.

All this means that during a typical browser-server communication session, to display one HTML document, the browser needs to make several independent requests and wait for answers to them. The first request loads the HTML code, the rest - additional materials, without which the document cannot be correctly displayed. It would be great if all of these additional materials could be sent to the browser along with the original HTML document, which would save the browser from having to download them separately. As a matter of fact, the HTTP / 2 Server Push technology is intended for the organization of such work scenarios.

When using HTTP / 2, the server can automatically, on its own initiative, send additional resources along with the response to the original request. These are the resources that, according to the server, the browser will certainly request later. When the browser needs these resources, instead of sending additional requests to receive them, it is enough to use the data that the server sent to it in advance.

For example, suppose a file with the /index.htmlfollowing contents is stored on a server :

  
  


  This is an awesome Unicorn! 

Having received the corresponding request, the server will respond to it by sending this file. At the same time, the server knows that for the correct output of the file /index.html, files /static/awesome.cssand are needed /static/unicorn.png. As a result, the server, using the Server Push mechanism, will send these files along with the file /index.html.

for (const asset of ['/static/awesome.css', '/static/unicorn.png']) {
  // stream - это ServerHttp2Stream.
  stream.pushStream({':path': asset}, (err, pushStream) => {
    if (err) throw err;
    pushStream.respondWithFile(asset);
  });
}

On the client side, as soon as the browser parses the file code /index.html, it will understand that the files static/awesome.cssand are needed to render this document /static/unicorn.png. In addition, it will become clear to the browser that these files have already been sent to it at the initiative of the server and stored in the browser cache. As a result, he will not have to send two additional requests to the server. Instead, it simply takes data from the cache that is already loaded there.

Until now, all this looks very good. However, if you look closely, in the above scenario, you can find potential difficulties. To begin with, it is not so easy for a server to find out what additional resources can be sent on its initiative in response to an initial browser request. The logic of this decision can be taken to the application level, blaming the developer. But even a site developer may find it difficult to make such decisions. One way to do this is as follows: the developer looks at the HTML code and compiles a list of additional resources necessary for the page to display correctly in the browser. However, as the application develops, maintaining such a list up to date is time consuming and fraught with errors.

Another possible problem lies in the fact that the internal mechanisms of the browser are engaged in caching resources that were recently downloaded. Let us return to the above example. If, for example, the browser downloaded the file /index.htmlyesterday, then it would have downloaded the file /static/unicorn.png, which usually falls into the browser cache. When the browser downloads again /index.htmland after that tries to download the file /static/unicorn.png, it knows that this file is already in the cache. Therefore, the browser does not fulfill the request to download this file, instead receiving it from the cache. In this case, sending the file to the /static/unicorn.pngbrowser at the initiative of the server will be a waste of network resources. It would be nice for the server to have some kind of mechanism that allows it to understand whether a browser has already cached a certain resource.

In fact, other non-trivial tasks are associated with Server Push technology. If you are interested, read this document .

Automate the use of HTTP / 2 Server Push

In order to simplify support for Server Push for Node.js developers, Google has published an npm package for its automation: h2-auto-push . This package is designed to solve many difficult problems, among them are those that we talked about above, and those that are mentioned in this document .

The package reveals patterns in requests coming from browsers, and finds out what additional resources are associated with those source resources that browsers apply for. Later, when requesting the same source resources, additional resources are automatically sent to the browser at the initiative of the server. In addition, the package evaluates the possibility that the browser already has some resources in its cache, and if it turns out that it is, it does not send these resources to it.

This package is designed for use in the middleware layer of various web frameworks. In particular, we are talking about tools for serving static files. As a result, using this package facilitates the development of auxiliary tools for automating the sending of materials to browsers at the initiative of servers. For example, take a look at the fastify-auto-push package . This is a plugin for fastify , designed to automate the sending of materials to browsers at the initiative of servers and using the h2-auto-push package .

This middleware is also pretty easy to use from applications:

const fastify = require('fastify');
const fastifyAutoPush = require('fastify-auto-push');
const fs = require('fs');
const path = require('path');
const {promisify} = require('util');
const fsReadFile = promisify(fs.readFile);
const STATIC_DIR = path.join(__dirname, 'static');
const CERTS_DIR = path.join(__dirname, 'certs');
const PORT = 8080;
async function createServerOptions() {
  const readCertFile = (filename) => {
    return fsReadFile(path.join(CERTS_DIR, filename));
  };
  const [key, cert] = await Promise.all(
      [readCertFile('server.key'), readCertFile('server.crt')]);
  return {key, cert};
}
async function main() {
  const {key, cert} = await createServerOptions();
  // Браузеры поддерживают для HTTP/2 только https.
  const app = fastify({https: {key, cert}, http2: true});
  // Создаём и регистрируем плагин AutoPush. Он должен быть зарегистрирован первым в
  // цепочке промежуточного ПО.
  app.register(fastifyAutoPush.staticServe, {root: STATIC_DIR});
  await app.listen(PORT);
  console.log(`Listening on port ${PORT}`);
}
main().catch((err) => {
  console.error(err);
});

Summary

According to tests conducted by the Node.js Foundation, it was found that use h2-auto-pushimproves performance by about 12% compared to using HTTP / 2 without using Server Push technology, and gives a performance increase of approximately 135% compared to HTTP / 1.

Dear readers! How do you feel about Server Push technology?