You must be joking, Mr. Dahl, or why Node.js is the crown of the evolution of web servers

Node.js is a thing around which there is a lot of noise, rave reviews and angry shouts. At the same time, according to my observations, the following idea about what Node.js is fixed in the minds of people is: "this is a thing that allows you to write JavaScript on the server side and uses the JavaScript engine from Google Chrome." Fans of the tongue sighed enthusiastically: “Ah! It came true! ”, The opponents gritted their teeth:“ Well, we still didn’t have enough of this nonsense with prototypes and dynamic typing on the servers! ” And together they ran to break spears into blogs and forums.

At the same time, many representatives of both camps are of the opinion that Node.js is an esoteric toy, a fun idea for porting the language of browser scripts to “new wheels”. In order to be completely honest, I confess that I also adhered to this point of view. At one fine moment, I took my heart and decided to "dig a little deeper." It turned out that the creator of Node.js, Ryan Dahl, was far from a fanatic, but a man trying to solve a real problem. And his creation is not a toy, but a practical solution.


So what is Node.js? On the official websiteflaunts the inscription: "Evented I / O for V8 JavaScript." Not very meaningful, right? Well, let's try to “invent a bicycle” and “think up” this notorious “Evented I / O for V8 JavaScript”. Of course, we will not write any code (poor Ryan still had to do this), but just try to build a chain of conclusions that will lead us to the idea of ​​creating Node.js and how it should be arranged.

Keep it simple, stupid

So, you know that web applications use a client-server programming model . The client is the user's browser, and the server is, for example, one of the machines in the data center of a hoster (select any one to your taste). The browser requests some resource from the server, which, in turn, gives it to the client. Such a method of customer communication and the server is called " client pull ", ie to the client literally pulls (.. English. Pull) server - "Let me out the page, well, let ...". The server "ponders" the answer to the annoying client's question and gives it to him in a digestible form.



So, we have a model of the simplest web server - a program that receives requests from browser clients, processes them and returns a response.

Parallel universe

Great, but such a simple server can only communicate with one user at a time. If at the moment of processing the request one more client addresses him, then he will have to wait until the server answers first. So we need to parallelize the processing of requests from users. The obvious solution: handle user requests in separate threads or processes of the operating system. Let us call such a process or thread - worker'om ( Engl. Worker).



In one form or another, this approach is followed by many of the most popular web servers today (for example, Apache and IIS) This model is relatively simple to implement and at the same time can satisfy the needs of most small and medium-sized web resources today.

But this model is completely incapable if you need to process thousands of requests at the same time. There are several reasons for this. First, the creation of both processes and threads is a damn consignment for any operating system. But we can go to the trick and create threads or processes in advance and use them as needed. OK, we just came up with mechanisms called " thread pool " for threads and " prefork»For processes. This will help us not to waste resources on creating processes and threads, since this overhead operation can be performed, for example, when the server starts. Secondly, what if all created workers are busy? Create new ones? But we already fully loaded all the processor cores of our server, if we add a few more threads or processes, they will compete for processor time with already running threads and processes. This means that both will work even slower. Yes, and as noted earlier, creating and servicing threads and processes is a costly thing in terms of RAM consumption and if we create a thread for each of a thousand users, we may soon find ourselves in a situation where there simply will not be any memory left on the server, and worker ''

To infinity and beyond!

It would seem that we were in an unsolvable situation with the available computing resources. The only solution is to scale the hardware resources , which is expensive in every way. Let's try to look at the problem from the other side: what are the majority of our workers busy with? They accept a request from a client, create a response, and send it to the client. So where is the weak link? There are two of them here - receiving a request from a client and sending a response. To understand that this is so, just remember the average speed of an Internet connection today. But I / O subsystem can work in asynchronous mode, and therefore may not block workers. Hmm, then it turns out actually the only thing our workers will do is generate a response for the client and manage tasks for the I / O subsystem. Previously, each worker could serve only one client at a time, since he took on the responsibility of fulfilling the entire request processing cycle. Now, when we have delegated network I / O to the I / O subsystem, one worker can serve several requests at the same time, for example, generating a response for one client, while the answer for another is given by the I / O subsystem. It turns out that now we do not need to allocate a thread for each of the users, but we can create one worker for the server processor, thus providing it with maximum hardware resources.

In practice, such delegation is implemented using an event-oriented programming paradigm . Programs developed according to this paradigm can be implemented as a finite state machine . Certain events translate a given machine from one state to another. In our case, the server will be implemented in the form of an endless cycle that will generate responses for clients, poll the descriptors of the I / O subsystem for their readiness to perform a particular operation, and, if successful, pass them a new task. The process of polling descriptors for the I / O subsystem is called “ polling ”". The fact is that effective polling implementations are currently available only in * nix systems, because the latter provide very fast system calls with linear runtime for these purposes (for example, epoll on Linux and kqueue on BSD systems). This is a very efficient server model, because it allows you to use hardware resources to the maximum. In fact, none of the server subsystems is idle idle, as you can easily see by looking at the figure.



A similar concept is used by servers such as nginx and lightppd , which have proven themselves in highly loaded systems.

Let's come together

But (there is always a “one but”), before that we started from the idea that generating a response takes an order of magnitude less time than communicating with a client. And this is partly true. Nevertheless, sometimes generating a response can be a complex and complex task, which may include reading and writing to disk, working with a database (which can also be located on a remote server). Well, it turns out we have actually returned to the original problem. In practice, it is resolved as follows: the system is divided into two parts - front-end and back-end. Front-end is a server with which the client communicates directly. As a rule, this is a server with an asynchronous event model, which can quickly establish communication with clients and give them the results of the request (for example, nginx). Back-end is a server with a blocking I / O model (for example, Apache), to which the front-end delegates the creation of a response for the client, just like it does with the I / O subsystem. Such a front-end is also called a “ reverse proxy, ” because it is essentially a normal proxy server , but installed in the same server environment as the server to which it redirects requests.

If we draw analogies with real life, then the front-end is a manager with brilliant white teeth and an expensive suit, the back-end is a group of workers at the factory, and the input / output subsystem is the transport department of the company for which the manager works and which owns plant. Clients turn to the manager by sending him letters through the transport department. The manager makes a deal with the client for the supply of a batch of products and sends an order to the workers to make the batch. The manager himself, in turn, does not expect until the workers finish the execution of the order, but continues to deal with their immediate responsibilities - makes deals with customers and makes sure that the whole process goes smoothly and smoothly. Periodically, the manager contacts the workers to inquire about the degree of readiness of the order, and if the batch is ready, This instructs the transport department to send the order to the customer. Well, of course, periodically monitors that the goods reach the client. This is how the idea of ​​the division of labor, invented thousands of years ago, found unexpected application in high technology.

And the reaper, and the Shvets, and the igret on the pipe (it would seem, what does JavaScript have to do with it?)

Well, all this works fine, but somehow our system has become extremely complicated, don’t you? Yes, although we delegate the generation of the response to another server, this is still not the fastest process, because during it blockages can occur due to file I / O and working with the database, which inevitably leads to processor downtime. So how do we restore integrity to the system while eliminating bottlenecks in the response generation process? Elementaryly, Watson - we will do all the I / O and work with the database non-blocking, built on events (yes, the most evented I / O )!

“But this changes the whole paradigm of creating web applications, and most of the existing frameworks are no longer applicable or applicable, but the solutions using them are not elegant!” - you say, and you will be right. Yes, and the human factor cannot be ruled out - by applying Murphy’s law it can be argued that “ if it is possible to use the functions of blocking I / O, then someone will do it sooner or later ”, thus breaking the entire initial idea. It is only a matter of time, project scope and qualifications of programmers. “Be careful making abstractions. You might have to use them. ”( Engl. “ Be careful when creating abstractions, because you may have to use them. ” ) Ryan says in a speech on Google Tech Talk. So let's stick to minimalism and create only the foundation that will allow us to develop web applications and at the same time will be so well tuned for an asynchronous programming model that we will not have the opportunity, and most importantly, the desire, to deviate from it. So what is the minimum we need?

Obviously, for starters, we need a runtime environment , the main requirements of which are fast execution of the response generation code and asynchronous input / output. What modern programming language is tailored to the event model, is known to all web developers and at the same time has fast and rapidly developing implementations? The answer is obvious - this is JavaScript. Moreover, we have at our disposal JavaScript engine V8 from Google, distributed under a very liberal BSD license. V8 is beautiful in many aspects: firstly, it uses JIT compilation and many other optimization techniques , and secondly, it is a sample of a well-made, thought-out and actively developing software product (usually I give V8 as an example of really high-quality C ++ code for colleagues at work). Add to all this the libev library , which will allow us to easily organize the event loop and provide a higher-level wrapper for polling mechanisms (so we don’t have to worry about the features of its implementation for various operating systems). We also need the libeio libraryfor fast asynchronous file I / O. Well, on this our runtime can be considered ready.

And, of course, we need a standard library that will contain JavaScript wrappers for all basic I / O operations and functions without which you will not go far in web development (for example, parsing HTTP headers and URLs, counting hashes, DNS- resolving, etc.).



It’s probably worth congratulating us - we just came up with the concept of a very fast server - Node.js.

I'm just sayin '

Summarizing, I want to say that Node.js is a very young project, which, if used correctly, can revolutionize the world of web development. Today, the project has a number of unresolved problems that complicate its use in real highly loaded systems (although precedents already exist ). For example, Node.js is essentially just one worker. If you have, say, a dual-core processor, then the only way to fully utilize its hardware resources with Node.js is to run two server instances (one for each core) and use reverse proxy (for example, the same nginx) to balance the load between them.
But, all such problems are solvable and they are actively working on, and a huge community is already being built around Node.js and many large companies are paying considerable attention to this development. It remains only to wish Mr. Dahl to bring his case to the end (which, incidentally, can help him ), and you, dear reader, should spend a lot of pleasant time developing for Node.js.