Multithreading in Node.js: module worker_threads
- Transfer
On January 18, the release of the Node.js platform version 11.7.0 was announced . Among the notable changes in this version we can point out the conclusion from the discharge of the experimental module worker_threads, which appeared in Node.js 10.5.0 . Now to use it, the --experimental-worker flag is not needed. This module, since its inception, has remained fairly stable, so the decision was made , reflected in Node.js 11.7.0.
The author of the material, the translation of which we publish, proposes to discuss the possibilities of the worker_threads module, in particular, he wants to talk about why this module is needed, and about how, for historical reasons, multithreading is implemented in Node.js. Here we will talk about what problems are associated with writing multi-threaded JS applications, about the existing ways to solve them, and about the future of parallel data processing using so-called “worker threads” (sometimes called “worker threads”). or simply "workers".
JavaScript was designed as a single-threaded programming language that runs in a browser. “Single-threading” means that in the same process (in modern browsers we are talking about separate browser tabs) only one set of instructions can be executed at a time.
This simplifies the development of applications, facilitates the work of programmers. Initially, JavaScript was a language suitable only for adding some interactive features to web pages, for example, something like form validation. Among the tasks for which JS was designed, there was not something particularly complex that needed multithreading.
Ryan dahl, the creator of Node.js, saw an interesting opportunity in this language restriction. He wanted to implement a server platform based on an asynchronous I / O subsystem. This meant that the programmer did not need to work with threads, which greatly simplifies development for a similar platform. When developing programs designed for parallel code execution, problems may arise that are very difficult to solve. For example, if several threads try to access the same memory area, this can lead to the so-called “process race condition” disrupting the operation of the program. Such errors are difficult to reproduce and correct.
Are Node.js applications single-threaded. Yes, in a way it is. In fact, Node.js allows you to perform certain actions in parallel, but to do this, the programmer does not need to create threads or synchronize them. The Node.js platform and the operating system perform parallel I / O operations with their own means, and when it comes time to process the data with our JavaScript code, it works in single-threaded mode.
In other words, everything except our JS code works in parallel. In synchronous blocks of JavaScript code, commands are always executed one by one, in the order in which they are presented in the source code:
All this is wonderful - in the event that all that our code is doing is performing asynchronous I / O operations. The program consists of small blocks of synchronous code that quickly operate on data, for example, sent to files and streams. The code of program fragments works so fast that it does not block the execution of the code of its other fragments. Much more time than the execution of the code takes to wait for the results of asynchronous I / O operations. Consider a small example:
It is possible that the database query shown here will be executed for about a minute, but the message
What happens if we need to perform heavy calculations using JavaScript? For example - to process a large set of data stored in memory? This can lead to the fact that the program will contain a fragment of synchronous code, the execution of which takes a long time and blocks the execution of other code. Imagine that these calculations take 10 seconds. If we are talking about a web server that processes a request, it will mean that it will not be able to process other requests for at least 10 seconds. This is a big problem. In fact, calculations that are longer than 100 milliseconds can already cause a similar problem.
JavaScript and the Node.js platform were not originally designed for solving tasks that use processor resources intensively. In the case of JS running in a browser, performing such tasks means “brakes” the user interface. In Node.js, this can limit the ability to request the platform to perform new asynchronous I / O tasks and the ability to respond to events associated with their completion.
Let's return to our previous example. Imagine that, in response to a request to the database, several thousand of some encrypted entries came in, which, in a synchronous JS code, need to be decrypted:
The results, after they are received, appear in the callback function. After that, until the end of their processing, no other JS code can be executed. Usually, as already mentioned, the load on the system created by such a code is minimal, it rather quickly performs the tasks assigned to it. But in this case, the results of the query, which have a considerable amount, came to the program, and we still need to process them. Something like this can take a few seconds. If we are talking about a server that many users are working with, this will mean that they can continue working only after the completion of a resource-intensive operation.
Given the above, it may seem that to solve heavy computational problems in Node.js you need to add a new module that allows you to create and manage threads. How can you do without something like that? Sadly enough, those who use a mature server platform, such as Node.js, do not have the means to beautifully solve tasks related to processing large amounts of data.
All this is true, but if you add the ability to work with streams in JavaScript, this will lead to a change in the very nature of this language. In JS, you cannot simply add the ability to work with threads, for example, in the form of a new set of classes or functions. To do this, you need to change the language itself. In languages that support multi-threading, such a thing as “synchronization” is widely used. For example, in java evensome numeric types are not atomic. This means that if you do not use synchronization mechanisms to work with them from different threads, all this may result in, for example, after a pair of threads tries to change the value of the same variable at the same time, several bytes of this variable will be set to one a stream, and a little - another. As a result, such a variable will contain something incompatible with the normal operation of the program.
Node.js will not execute the next block of code in the event queue until the previous block completes. This means that to solve our problem, we can break it into parts represented by synchronous code fragments, and then use the view construction
Imagine that we have a large array that needs to be processed, while complex processing is required during the processing of each element of such an array:
As already mentioned, if we decide to process the entire array in one go, it will take too much time and will not allow another application code to be executed. Therefore, we divide this big task into parts and use the construction
Now we, in one go, do the processing of ten elements of the array, after which, with the help
As you can see, this code looks much more complicated than its original version. And often the algorithm can be much more complicated than ours, which means that, when implemented, it will not be easy to break up the calculations into parts and understand where, in order to achieve the right balance, you need to set up a call
Perhaps the above approach is
The reality is that background processes can be used in Node.js. The point is that you can create a fork of the process and implement the above described scheme of work using the mechanism for exchanging messages between the child and parent processes. The main process can interact with the child process, sending events to it and receiving them from it. Shared memory with this approach is not used. All data exchanged between processes is “cloned,” that is, when changes are made to a copy of this data by one process, these changes to another process are not visible. This is similar to an HTTP request — when a client sends it to a server, the server only receives a copy of it. If processes do not use shared memory, this means that when they work simultaneously, the “race condition” cannot occur, and that we do not need to burden ourselves with working with threads. Looks like our problem is solved.
True, in fact it is not. Yes - we have one of the solutions to the problem of performing intensive calculations, but it is, again, imperfect. Creating a fork of a process is a resource-intensive operation. It takes time to complete. In fact, we are talking about creating a new virtual machine from scratch and about increasing the amount of memory consumed by the program, which is due to the fact that the processes do not use shared memory. Given the above, it is appropriate to ask whether it is possible, after performing a certain task, to reuse the fork of the process. We can give a positive answer to this question, but here we must remember that the fork of the process is going to transfer various resource-intensive tasks that will be performed synchronously in it. Here you can see two problems:
In order to solve these problems, we will need several forks, not just one, but we will have to limit their number, since each of them takes away system resources and it takes time to create each of them. As a result, modeled on systems serving database connections, we need something like a pool of processes ready for use. The process pool management system, upon receipt of new tasks, will use free processes to perform them, and when a process copes with the task, it will be able to assign a new one to it. There is a feeling that such a scheme of work is not easy to implement, and, in fact, the way it is. We will use the worker-farm package to implement this scheme :
So, is our problem solved? Yes, it can be said that it has been solved, but with this approach a lot more memory is required than it would be necessary if we had a multi-threaded solution at our disposal. Threads consume far less resources than processes. That is why in Node.js and appeared module
Worker threads run in an isolated context. They exchange information with the main process using messages. This saves us from the problem of the “race condition” that affects multi-threaded environments. In this case, the threads of the workers exist in the same process in which the main program is located, that is, with this approach, much less memory is used compared to using forks of processes.
In addition, working with workers, you can use shared memory. So, specially for this purpose objects of type are created
If you are using Node.js platform to version 11.7.0, then, to include the ability to work with the module
In addition, it is worth remembering that creating a worker (as well as creating a stream in any language), although it requires much less resources than creating a fork of the process, also creates a certain load on the system. Perhaps in your case, even this load may be too large. In such cases, the documentation recommends creating a pool of workers. If you need it, then, of course, you can create your own implementation of such a mechanism, but perhaps you should look for something suitable in the NPM registry.
Consider an example of working with worker threads. We will have the main file,
As you can see, using the workflow threading mechanism is quite simple. Namely, when creating a worker, you need to pass the
Above, by creating an object of type
In the code of the worker we are interested in two things. First, we need the data transmitted by the main application. In our case, they are represented by a variable
Here is a very simple example, but using the same mechanisms one can build much more complex structures. For example, from a worker's thread, you can send multiple messages to the main thread that carry information about the state of data processing if our application needs a similar mechanism. More from the worker data processing results can be returned in parts. For example, something like this can be useful in a situation where the worker is busy, for example, processing thousands of images, and you, not waiting for them to process them, want to notify the main application about the completion of each of them.
Details about the module
You may have heard of web workers. They are designed for use in the client environment, this technology has existed for quite some time and enjoys quite good support for modern browsers. The API for working with web workers is different from what the Node.js module gives us
The module
Dear readers! How do you organize hard computing in Node.js applications?
The author of the material, the translation of which we publish, proposes to discuss the possibilities of the worker_threads module, in particular, he wants to talk about why this module is needed, and about how, for historical reasons, multithreading is implemented in Node.js. Here we will talk about what problems are associated with writing multi-threaded JS applications, about the existing ways to solve them, and about the future of parallel data processing using so-called “worker threads” (sometimes called “worker threads”). or simply "workers".
Life in a single threaded world
JavaScript was designed as a single-threaded programming language that runs in a browser. “Single-threading” means that in the same process (in modern browsers we are talking about separate browser tabs) only one set of instructions can be executed at a time.
This simplifies the development of applications, facilitates the work of programmers. Initially, JavaScript was a language suitable only for adding some interactive features to web pages, for example, something like form validation. Among the tasks for which JS was designed, there was not something particularly complex that needed multithreading.
Ryan dahl, the creator of Node.js, saw an interesting opportunity in this language restriction. He wanted to implement a server platform based on an asynchronous I / O subsystem. This meant that the programmer did not need to work with threads, which greatly simplifies development for a similar platform. When developing programs designed for parallel code execution, problems may arise that are very difficult to solve. For example, if several threads try to access the same memory area, this can lead to the so-called “process race condition” disrupting the operation of the program. Such errors are difficult to reproduce and correct.
Is the Node.js platform single-threaded?
Are Node.js applications single-threaded. Yes, in a way it is. In fact, Node.js allows you to perform certain actions in parallel, but to do this, the programmer does not need to create threads or synchronize them. The Node.js platform and the operating system perform parallel I / O operations with their own means, and when it comes time to process the data with our JavaScript code, it works in single-threaded mode.
In other words, everything except our JS code works in parallel. In synchronous blocks of JavaScript code, commands are always executed one by one, in the order in which they are presented in the source code:
let flag = falsefunctiondoSomething() {
flag = true
// Тут идёт ещё какой-то код (он не меняет состояние переменной flag)...
// Мы можем быть уверены в том, что здесь в переменную flag записано значение true.
// Какой-то другой код не может поменять эту переменную,
// так как код здесь выполняется синхронно.
}All this is wonderful - in the event that all that our code is doing is performing asynchronous I / O operations. The program consists of small blocks of synchronous code that quickly operate on data, for example, sent to files and streams. The code of program fragments works so fast that it does not block the execution of the code of its other fragments. Much more time than the execution of the code takes to wait for the results of asynchronous I / O operations. Consider a small example:
db.findOne('SELECT ... LIMIT 1', function(err, result) {
if (err) returnconsole.error(err)
console.log(result)
})
console.log('Running query')
setTimeout(function() {
console.log('Hey there')
}, 1000)It is possible that the database query shown here will be executed for about a minute, but the message
Running querywill go to the console immediately after this request is initiated. In this case, the message Hey therewill be displayed in a second after the execution of the request, regardless of whether its execution has completed or not. Our Node.js application simply calls the function that triggers the request, and the execution of its other code is not blocked. After the request is completed, the application will be notified of this using the callback function, and immediately it will receive an answer to this request.CPU intensive tasks
What happens if we need to perform heavy calculations using JavaScript? For example - to process a large set of data stored in memory? This can lead to the fact that the program will contain a fragment of synchronous code, the execution of which takes a long time and blocks the execution of other code. Imagine that these calculations take 10 seconds. If we are talking about a web server that processes a request, it will mean that it will not be able to process other requests for at least 10 seconds. This is a big problem. In fact, calculations that are longer than 100 milliseconds can already cause a similar problem.
JavaScript and the Node.js platform were not originally designed for solving tasks that use processor resources intensively. In the case of JS running in a browser, performing such tasks means “brakes” the user interface. In Node.js, this can limit the ability to request the platform to perform new asynchronous I / O tasks and the ability to respond to events associated with their completion.
Let's return to our previous example. Imagine that, in response to a request to the database, several thousand of some encrypted entries came in, which, in a synchronous JS code, need to be decrypted:
db.findAll('SELECT ...', function(err, results) {
if (err) returnconsole.error(err)
// Большой объём результатов и их обработка, требовательная к ресурсам процессора.
for (const encrypted of results) {
const plainText = decrypt(encrypted)
console.log(plainText)
}
})The results, after they are received, appear in the callback function. After that, until the end of their processing, no other JS code can be executed. Usually, as already mentioned, the load on the system created by such a code is minimal, it rather quickly performs the tasks assigned to it. But in this case, the results of the query, which have a considerable amount, came to the program, and we still need to process them. Something like this can take a few seconds. If we are talking about a server that many users are working with, this will mean that they can continue working only after the completion of a resource-intensive operation.
Why in JavaScript never will be flows?
Given the above, it may seem that to solve heavy computational problems in Node.js you need to add a new module that allows you to create and manage threads. How can you do without something like that? Sadly enough, those who use a mature server platform, such as Node.js, do not have the means to beautifully solve tasks related to processing large amounts of data.
All this is true, but if you add the ability to work with streams in JavaScript, this will lead to a change in the very nature of this language. In JS, you cannot simply add the ability to work with threads, for example, in the form of a new set of classes or functions. To do this, you need to change the language itself. In languages that support multi-threading, such a thing as “synchronization” is widely used. For example, in java evensome numeric types are not atomic. This means that if you do not use synchronization mechanisms to work with them from different threads, all this may result in, for example, after a pair of threads tries to change the value of the same variable at the same time, several bytes of this variable will be set to one a stream, and a little - another. As a result, such a variable will contain something incompatible with the normal operation of the program.
Primitive solution of the problem: iterations of the event loop
Node.js will not execute the next block of code in the event queue until the previous block completes. This means that to solve our problem, we can break it into parts represented by synchronous code fragments, and then use the view construction
setImmediate(callback)to plan the execution of these fragments. The code defined by the function callbackin this construct will be executed after the tasks of the current iteration (tick) of the event loop are completed. After that, the same design is used to queue the next batch of calculations. This allows not to block the cycle of events and, at the same time, to solve voluminous tasks.Imagine that we have a large array that needs to be processed, while complex processing is required during the processing of each element of such an array:
const arr = [/*large array*/]
for (const item of arr) {
// для обработки каждого элемента массива нужны сложные вычисления
}
// код, который будет здесь, выполнится только после обработки всего массива.As already mentioned, if we decide to process the entire array in one go, it will take too much time and will not allow another application code to be executed. Therefore, we divide this big task into parts and use the construction
setImmediate(callback):const crypto = require('crypto')
const arr = newArray(200).fill('something')
functionprocessChunk() {
if (arr.length === 0) {
// код, выполняющийся после обработки всего массива
} else {
console.log('processing chunk');
// выберем 10 элементов и удалим их из массива
const subarr = arr.splice(0, 10)
for (const item of subarr) {
// произведём сложную обработку каждого из элементов
doHeavyStuff(item)
}
// поставим функцию в очередь
setImmediate(processChunk)
}
}
processChunk()
functiondoHeavyStuff(item) {
crypto.createHmac('sha256', 'secret').update(newArray(10000).fill(item).join('.')).digest('hex')
}
// Этот фрагмент нужен лишь для подтверждения того, что, обрабатывая большой массив,// мы даём возможность выполняться и другому коду.let interval = setInterval(() => {
console.log('tick!')
if (arr.length === 0) clearInterval(interval)
}, 0)Now we, in one go, do the processing of ten elements of the array, after which, with the help
setImmediate(), we plan to perform the next batch of calculations. And this means that if in the program you need to execute some other code, it can be executed between operations on processing fragments of an array. For this, here, at the end of the example, is the code in which it is used setInterval(). As you can see, this code looks much more complicated than its original version. And often the algorithm can be much more complicated than ours, which means that, when implemented, it will not be easy to break up the calculations into parts and understand where, in order to achieve the right balance, you need to set up a call
setImmediate()planning the next part of the calculation. In addition, the code is now asynchronous, and if our project depends on third-party libraries, then we may not be able to break the process of solving a heavy task into parts.Background processes
Perhaps the above approach is
setImmediate()normally suitable for simple cases, but is far from ideal. In addition, streams are not used here (for obvious reasons) and we also do not intend to change the language for the sake of it. Is it possible to perform parallel data processing without using streams? Yes, it is possible, and for this we need some kind of mechanism for background data processing. The point is to run a task, transfer data to it, and so that this task, without interfering with the main code, would use all that it needs, spend on work as much time as it needs, and then return the results to main code. We need something like the following code snippet:// Запускаем script.js в новом окружении, без использования разделения памяти.const service = createService('script.js')
// Отправляем сервису входные данные и создаём механизм получения результатов
service.compute(data, function(err, result) {
// тут будут результаты обработки данных
})The reality is that background processes can be used in Node.js. The point is that you can create a fork of the process and implement the above described scheme of work using the mechanism for exchanging messages between the child and parent processes. The main process can interact with the child process, sending events to it and receiving them from it. Shared memory with this approach is not used. All data exchanged between processes is “cloned,” that is, when changes are made to a copy of this data by one process, these changes to another process are not visible. This is similar to an HTTP request — when a client sends it to a server, the server only receives a copy of it. If processes do not use shared memory, this means that when they work simultaneously, the “race condition” cannot occur, and that we do not need to burden ourselves with working with threads. Looks like our problem is solved.
True, in fact it is not. Yes - we have one of the solutions to the problem of performing intensive calculations, but it is, again, imperfect. Creating a fork of a process is a resource-intensive operation. It takes time to complete. In fact, we are talking about creating a new virtual machine from scratch and about increasing the amount of memory consumed by the program, which is due to the fact that the processes do not use shared memory. Given the above, it is appropriate to ask whether it is possible, after performing a certain task, to reuse the fork of the process. We can give a positive answer to this question, but here we must remember that the fork of the process is going to transfer various resource-intensive tasks that will be performed synchronously in it. Here you can see two problems:
- Although with this approach, the main process is not blocked, the descendant process is capable of performing the tasks transferred to it only sequentially. If we have two tasks, the execution of one of which takes 10 seconds, and the second takes 1 second, and we are going to execute them in that order, then we will hardly like the need to wait for the first one to complete before starting the second one. Since we are engaged in creating forks of processes, we would like to take advantage of the capabilities of the operating system for scheduling tasks and use the computing resources of all the cores of our processor. We need something that resembles working at the computer of a person who listens to music and travels through web pages. To do this, you can create two process fork and organize with their help parallel execution of tasks.
- In addition, if one of the tasks leads to the completion of the process with an error, all tasks sent to such a process will be unprocessed.
In order to solve these problems, we will need several forks, not just one, but we will have to limit their number, since each of them takes away system resources and it takes time to create each of them. As a result, modeled on systems serving database connections, we need something like a pool of processes ready for use. The process pool management system, upon receipt of new tasks, will use free processes to perform them, and when a process copes with the task, it will be able to assign a new one to it. There is a feeling that such a scheme of work is not easy to implement, and, in fact, the way it is. We will use the worker-farm package to implement this scheme :
// главное приложениеconst workerFarm = require('worker-farm')
const service = workerFarm(require.resolve('./script'))
service('hello', function (err, output) {
console.log(output)
})
// script.js// Этот код будет выполняться в процессах-форкахmodule.exports = (input, callback) => {
callback(null, input + ' ' + world)
}Module worker_threads
So, is our problem solved? Yes, it can be said that it has been solved, but with this approach a lot more memory is required than it would be necessary if we had a multi-threaded solution at our disposal. Threads consume far less resources than processes. That is why in Node.js and appeared module
worker_threads. Worker threads run in an isolated context. They exchange information with the main process using messages. This saves us from the problem of the “race condition” that affects multi-threaded environments. In this case, the threads of the workers exist in the same process in which the main program is located, that is, with this approach, much less memory is used compared to using forks of processes.
In addition, working with workers, you can use shared memory. So, specially for this purpose objects of type are created
SharedArrayBuffer. They should be used only in cases when the program needs to perform complex processing of large amounts of data. They allow saving resources required for serialization and de-serialization of data when organizing data exchange between workers and the main program using messages.Work with worker threads
If you are using Node.js platform to version 11.7.0, then, to include the ability to work with the module
worker_threadsyou need when running Node.js, use the flag --experimental-worker. In addition, it is worth remembering that creating a worker (as well as creating a stream in any language), although it requires much less resources than creating a fork of the process, also creates a certain load on the system. Perhaps in your case, even this load may be too large. In such cases, the documentation recommends creating a pool of workers. If you need it, then, of course, you can create your own implementation of such a mechanism, but perhaps you should look for something suitable in the NPM registry.
Consider an example of working with worker threads. We will have the main file,
index.jsin which we create a worker thread and pass it some data for processing. The corresponding API is based on events, but I'm going to use a promise here that is allowed when the first message comes from the worker:// index.js// Если вы пользуетесь Node.js старше версии 11.7.0, воспользуйтесь // для запуска этого кода командой node --experimental-worker index.jsconst { Worker } = require('worker_threads')
functionrunService(workerData) {
returnnewPromise((resolve, reject) => {
const worker = new Worker('./service.js', { workerData });
worker.on('message', resolve);
worker.on('error', reject);
worker.on('exit', (code) => {
if (code !== 0)
reject(newError(`Worker stopped with exit code ${code}`));
})
})
}
asyncfunctionrun() {
const result = await runService('world')
console.log(result);
}
run().catch(err =>console.error(err))As you can see, using the workflow threading mechanism is quite simple. Namely, when creating a worker, you need to pass the
Workerpath to the file with the code of the worker and the data to the designer . Remember that this data is cloned, not stored in shared memory. After starting the worker, we expect messages from him listening to the event message. Above, by creating an object of type
Worker, we passed the constructor the name of the file with the worker code - service.js. Here is the code for this file:const { workerData, parentPort } = require('worker_threads')
// Тут, асинхронно, не блокируя главный поток,// можно выполнять тяжёлые вычисления.
parentPort.postMessage({ hello: workerData })In the code of the worker we are interested in two things. First, we need the data transmitted by the main application. In our case, they are represented by a variable
workerData. Secondly, we need a mechanism to transfer information to the main application. This mechanism is represented by an object parentPortthat has a method postMessage(), using which we transfer the results of data processing to the main application. That's how it all works.Here is a very simple example, but using the same mechanisms one can build much more complex structures. For example, from a worker's thread, you can send multiple messages to the main thread that carry information about the state of data processing if our application needs a similar mechanism. More from the worker data processing results can be returned in parts. For example, something like this can be useful in a situation where the worker is busy, for example, processing thousands of images, and you, not waiting for them to process them, want to notify the main application about the completion of each of them.
Details about the module
worker_threadscan be found here .Web workers
You may have heard of web workers. They are designed for use in the client environment, this technology has existed for quite some time and enjoys quite good support for modern browsers. The API for working with web workers is different from what the Node.js module gives us
worker_threads, it's all about the differences between the environments in which they work. However, these technologies are able to solve similar problems. For example, web workers can be used in client applications to perform data encryption and decryption, compression and decompression. With their help, you can process images, implement computer vision systems (for example, we are talking about face recognition) and solve other similar tasks in the browser.Results
The module
worker_threadsis a promising addition to the Node.js capabilities. Using this module, you can perform resource-intensive calculations without blocking server applications. Threads of workers are very similar to traditional threads, but since they do not use shared memory, they are free from the problems associated with the traditional multi-threaded programming like the “race condition”. What to choose for those who need such opportunities right now? Perhaps, since the module has worker_threadsonly recently worn experimental status, while in order to perform background data processing in Node.js, it is worth looking at something like worker-farm , having planned to switch to worker_threadsafter the Node.js community has gained more experience with this module . Dear readers! How do you organize hard computing in Node.js applications?