NODE.JS + Windows: take a look inside
An article for node.js programmers who understand the principles of asynchronous-event programming, but do not know how it works from the inside. If the standard picture with the “looped” circles is not enough for you and you want to at least take a look at what the cycle of events has under the hood, then you are under cat.
For simplicity of description, I decided to build my article around a simple file open operation, i.e. open functions from fs module . Naturally, because of this, VERY much will remain outside the scope of the article, but you will have to compromise between ease of perception and technical details.
To understand how Node.js works under Windows, you need to understand IOCP technology. Input / output completion port is a technology designed to perform asynchronous I / O operations used in Windows. The main object of this technology is the IOCP port created using the CreateIoCompletionPort () function. We are primarily interested in the fact that the IOCP port encapsulates an event queue created in the operating system. The PostQueuedCompletionStatus () function queues the event, and the GetQueuedCompletionStatus () function retrieves it. Moreover, if the queue is empty, then the thread that called GetQueuedCompletionStatus is suspended until the first event occurs.
Now, before proceeding directly to our example, we will consider some points in initializing node.js. At startup, a special structure is created that describes the cycle of events , let's call it loop. Among other fields, the structure contains a link to the IOCP port, and an asynchronous request counter. For simplicity, we denote it as the integer variable req_count. When the loop initializes, the IOCP port is created:
Next, the event loop starts, which we will talk about later.
And finally, the algorithm of the open function itself.
First, a structure describing the asynchronous request is created and initialized , let's call it fs_open_req. It stores a link to a callback onOpen, a path and file access modifiers, and other information describing the request. In addition, the structure contains a field for storing the query result or a link to it.
Secondly, the counter of asynchronous requests in the loop structure is increased.
Thirdly, a separate stream is created in which the file will be opened. In this case, the main thread node.js returns from the open function and then leaves for the next iteration of the event loop. The file 1.txt is opened in the spawned stream by the operating system. Its descriptor is written to the fs_open_req structure.
Fourth, after opening the file and completing all necessary operations, the generated thread calls PostQueuedCompletionStatus (), thereby placing the file open event in the IOCP queue. Moreover, through one of the PostQueuedCompletionStatus parameters, a link to the fs_open_req structure is attached to the generated event.
At the entrance to the event loop, the counter of asynchronous requests is checked. If there are no registered requests, the program ends. If there is, the GetQueuedCompletionStatus () function is called, which either returns the next event, or, if there are no events, suspends the thread until they occur.
At one iteration, the GetQueuedCompletionStatus function will return an event about opening a file and with it a link to the fs_open_req structure. Next, node.js will decrease the counter of asynchronous requests and start callback onOpen, passing the result of opening the file to it as a parameter.
That, in fact, is all. I wanted to show only the basic principles, so a lot remained to be described. For example, network I / O operations are organized in a slightly different way and make fuller use of IOCP capabilities. But let's leave it next time.
Limitation
For simplicity of description, I decided to build my article around a simple file open operation, i.e. open functions from fs module . Naturally, because of this, VERY much will remain outside the scope of the article, but you will have to compromise between ease of perception and technical details.
require('fs').open("c:\\1.txt", 'r', function onOpen(err, result){
console.log("Result: ", result);
});
IOCP
To understand how Node.js works under Windows, you need to understand IOCP technology. Input / output completion port is a technology designed to perform asynchronous I / O operations used in Windows. The main object of this technology is the IOCP port created using the CreateIoCompletionPort () function. We are primarily interested in the fact that the IOCP port encapsulates an event queue created in the operating system. The PostQueuedCompletionStatus () function queues the event, and the GetQueuedCompletionStatus () function retrieves it. Moreover, if the queue is empty, then the thread that called GetQueuedCompletionStatus is suspended until the first event occurs.
Initialization
Now, before proceeding directly to our example, we will consider some points in initializing node.js. At startup, a special structure is created that describes the cycle of events , let's call it loop. Among other fields, the structure contains a link to the IOCP port, and an asynchronous request counter. For simplicity, we denote it as the integer variable req_count. When the loop initializes, the IOCP port is created:
iocp _handle= CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 1);
Next, the event loop starts, which we will talk about later.
Open function .
And finally, the algorithm of the open function itself.
First, a structure describing the asynchronous request is created and initialized , let's call it fs_open_req. It stores a link to a callback onOpen, a path and file access modifiers, and other information describing the request. In addition, the structure contains a field for storing the query result or a link to it.
Secondly, the counter of asynchronous requests in the loop structure is increased.
Thirdly, a separate stream is created in which the file will be opened. In this case, the main thread node.js returns from the open function and then leaves for the next iteration of the event loop. The file 1.txt is opened in the spawned stream by the operating system. Its descriptor is written to the fs_open_req structure.
Fourth, after opening the file and completing all necessary operations, the generated thread calls PostQueuedCompletionStatus (), thereby placing the file open event in the IOCP queue. Moreover, through one of the PostQueuedCompletionStatus parameters, a link to the fs_open_req structure is attached to the generated event.
Cycle of events.
At the entrance to the event loop, the counter of asynchronous requests is checked. If there are no registered requests, the program ends. If there is, the GetQueuedCompletionStatus () function is called, which either returns the next event, or, if there are no events, suspends the thread until they occur.
At one iteration, the GetQueuedCompletionStatus function will return an event about opening a file and with it a link to the fs_open_req structure. Next, node.js will decrease the counter of asynchronous requests and start callback onOpen, passing the result of opening the file to it as a parameter.
Conclusion
That, in fact, is all. I wanted to show only the basic principles, so a lot remained to be described. For example, network I / O operations are organized in a slightly different way and make fuller use of IOCP capabilities. But let's leave it next time.