Own http server in less than 40 lines of code in libevent and C ++ 11

Looking through Habr from time to time, I periodically meet posts on the topic of creating my own web server in C ++ or in another language. Since C ++ from programming languages ​​is of more interest to me, I read this blog the most. If you look through it, you can easily find how to write your web server “on sockets”, using boost.asio or something else. Some time ago, I also published my post on the creation of such an http-server as an example of solving a test task. But not limited to the obtained one and for the sake of interest, I made comparisons with libevent and boost.asio developments. And as such, he refused to perform the test task as such.

For myself, at work, I considered libevent and libev. Each has its own advantages. If there is a desire or need for the speedy development of a small http-server, then libevent is of great interest to me, and with some C ++ 11 innovations, the code becomes much more compact and allows you to create a basic http-server in less than 40 lines.

The material of the post will probably be useful to those who are not yet familiar with libevent and there is a need to soon create their own http-server, as well as the material may be of interest to people who do not yet have such a need, and even if they already had experience in creating one, it’s interesting to know their opinion and experience. And since the post does not contain anything fundamentally new, it can be used as material for starting work in this direction, and therefore I will try to put a note “training material”.

The good thing about libevent, unlike, for example, libev and boost.asio, is that it has its own built-in http-server, and some abstraction for working with buffers. It also has a considerable set of auxiliary functions. You can also parse the HTTP protocol yourself by writing a simple state machine or some other method. When working with libevent, this is all there is. This is such a nice little thing, or you can go down to a lower level and write your own parser for HTTP, while doing work with sockets on libevent. I liked the level of detail at the library in that if you want to do something quickly, you can find a higher-level interface in it, which is usually less flexible. With the emergence of large needs, you can gradually go down level after level lower and lower. The library allows you to do many things: asynchronous input-output, work with the network, work with timers, rpc, etc. You can use it to create both server and client software.

What for?

Creating your own small http-server can be determined for each by its own needs, desire or unwillingness to use fully functional ready-made servers for one reason or another. Suppose you have some server software that works according to some protocol and solves some problems, and you have a need to issue some API for this software via the HTTP protocol. Only a few small functions are possible to configure the server and obtain its current status via HTTP. For example, having organized the processing of GET requests with parameters and given a small xml with a response or in some other format. In this case, you can create your own http-server with little effort, which will be the interface for your main server software. In addition, if you need to create your own small specific service for distributing a certain set of files or even create your own web application, then you can also use such a self-made small server. In general, you can use it to build self-contained server software, as well as to create auxiliary services within larger systems.

Simple http server in less than 40 lines

To create a simple single-threaded http server using libevent, you need to follow these few simple steps:

• Initialize the global library object using the event_init function. This function can only be used for single-threaded processing. For multi-threaded work, an object must be created for each thread (more on this below).
• The creation of the http server itself is carried out by the evhttp_start function in the case of a single-threaded server with a global event processing object. An object created with evhttp_start at the end should be deleted with evhttp_free.
• To respond to incoming requests, you need to set the callback function using evhttp_set_gencb.
• Then you can start the event loop with the event_dispatch function. This function is also designed to work in a single thread with a global object.
• When processing a request, you can get a buffer for response with the evhttp_request_get_output_buffer function. Add some content to this buffer. For example, you can use the evbuffer_add_printf function to send a string, and evbuffer_add_file to send a file. After that, the response to the request should be sent, and this can be done using evhttp_send_reply.

It turned out less than 40 lines that are able to process http requests, returning the string “Hello World”, and if you replace the evbuffer_add_printf function with evbuffer_add_file, you can send files. You can call this server the basic configuration. For the most part, any auto dealer or realtor wants their cars and apartments to never leave in the basic configuration under any circumstances, but only with additional options. But whether such options are needed by the consumer and to what extent ...

What such a basic configuration can provide in terms of speed can be checked using the ab utility for * nix systems with a small variation of parameters.


The test was conducted on a not-so-new laptop (2 cores, 4GB of RAM) running the 32-bit Ubuntu 12.10 operating system.

Multithreaded http server

Is multithreading necessary? The question is rhetorical ... It is possible to organize all IOs in one thread, and put requests in a queue and rake it into several threads. In this case, the above server can simply be supplemented with a queue and a pool of threads for processing and nothing else should be fenced. If there is a desire or need to build a multi-threaded server, then it will be slightly longer than the previous one, but not by much. C ++ 11 with its smart pointers allows you to implement RAII well , as was shown with std :: unique_ptr in the example above, and the presence of lambda functions slightly reduces the code.

An example of a multithreaded server is similar in its ideology to a single-threaded one, and some features associated with multithreading increase it by about 2 times the amount of code. Eighty with a few lines of code for a multi-threaded http server in C ++ is not so much.

One solution that can be made:

• Create multiple threads, for example, equal to twice the number of processor cores. C ++ 11 has support for working with threads and now you no longer need to write your own wrappers.
• For each thread, create your own event handling object using the event_base_new function. The created object at the end should be deleted by the event_base_free function, and std :: unique_ptr and RAII allow this to be done more compactly.
• For each stream, taking into account the above object, create your own http-server object using the evhttp_new function. This object must also be deleted at the end, and this can be done using evhttp_free.
• As in the previous example, install the request handler using evhttp_set_gencb.
• This step may be the strangest. It is necessary to create and bind a socket to a network interface for several handlers, each of which is located in its own thread. Here you can use the API for working with sockets (create a socket, configure it, bind it to a specific interface), and then pass the socket for operation to the server with the evhttp_accept_socket function. This is a long time. Libevent provides several functions to solve this problem. As mentioned above, libevent makes it possible, if necessary, to go down a level lower and lower depending on the need and choose the best one for yourself. In this case, for the first thread, all the work of creating a socket, its configuration and binding is performed by the evhttp_bind_socket_with_handle function, and the socket for other flows is extracted from the configured object using evhttp_bound_socket_get_fd. All other threads already use the received socket, setting it for processing by the evhttp_accept_socket function. A bit strange, but much easier than using the API for working with sockets, and even easier when considering cross-platform. It would seem that the API for Berkeley sockets is the same, but if you wrote cross-platform software using it, for example, for Windows and Linux, then code written for one operating system is definitely not equivalent to code for another.
• Запустить цикл обработки событий. В отличии от однопоточного сервера это надо сделать иным способом, так как объекты у всех разные. Для этого есть специальная функция в libevent (event_base_dispatch). Для себя я в ней вижу один минус — ее трудно править корректным способом (например, надо иметь ситуацию, в которой можно вызвать event_base_loopexit). Для этого надо немного извернуться. А так можно воспользоваться функцией event_base_loop. Эта функция не блокирующая даже если нет событий к обработке, она возвращает управление, что дает упрощенную возможность завершения цикла обработки событий и возможность что-то делать между вызовами. Есть и минус — чтобы напрасно не загружать процессор на холостом ходу надо поставить хоть небольшую задержку (в C++11 — 'это легко сделать примерно так: std::this_thread::sleep_for(std::chrono::milliseconds(10)) ).
• Request processing is similar to the first example.
• During creation and configuration of the next thread, something may be wrong with its function: for example, some libevent function reported an error. In this case, you can throw an exception and catch it, and then send it outside the stream using the same C ++ 11 tools (std :: exception_ptr, std :: current_exception and std :: rethrow_exception)

In the code, you can notice that each thread is created after some waiting time. This is a small hack that will already be fixed in the final version of the server. For now, we can only say that if this is not done, then the threads will need to be somehow synchronized so that they work out the “strange step” for creating and binding the socket. For simplicity, let this hack remain for now. Also in the above code, the lambda function may seem like a controversial solution. Lambdas can be a good solution when used, for example, as some kind of predicate when working with standard algorithms. At the same time, you can think about their use when writing larger pieces of code. In the example above, you could put everything into a regular function, pass all the necessary parameters and get the code in C ++ 03 style. At the same time, the use of lambda has reduced the amount of code. In my opinion, when the code is small, then lambdas can quite well fit into it even with not its shortest content and not adversely affect the quality of the code, of course you should not go to extremes and recall student everyday life with writing laboratory work in 700 lines in the only main functions.

The multithreaded server was tested with the same parameters as the previous example.

The final version of the server

Basic equipment is provided, equipment with a small set of options is also there. Now the turn came up and to create something more useful and functional, as well as with a little tuning.

Minimum http server:

#include "http_server.h"
#include "http_headers.h"
#include "http_content_type.h"
#include 
int main()
{
  try
  {
    using namespace Network;
    HttpServer Srv("127.0.0.1", 5555, 4,
      [&] (IHttpRequestPtr req)
      {
        req->SetResponseAttr(Http::Response::Header::Server::Value, "MyTestServer");
        req->SetResponseAttr(Http::Response::Header::ContentType::Value,
                             Http::Content::Type::html::Value);
        req->SetResponseString("
Hello Wotld!");
      });
    std::cout << "Press Enter for quit." << std::endl;
    std::cin.get();
  }
  catch (std::exception const &e)
  {
    std::cout << e.what() << std::endl;
  }
  return 0;
}

A very minimal amount of code for a C ++ http server. There is a fee for everything. And in this case, such simplicity of the client code for creating the server is paid for by a longer implementation hidden in the proposed wrapper over libevent. In fact, sales have slightly increased. Below, fragments of it will be described.

Server Creation:

• You must create an object of type HttpServer. As parameters, at least pass the address and port on which the server will work, the number of threads and the function for processing requests (in this case, since request processing is minimal, you can do a little lambda without creating a separate function or even a whole handler class) . After creating the object, the server will work as long as its object exists.
• The handler accepts a smart pointer to the IHttpRequest interface, the implementation of which hides all work with the libevent buffer and sending a response, and its methods make it possible to receive data from an incoming request and form a response.

This interface allows you to receive from the incoming request its type, some attributes (headers), the size of the request body and the request body itself, if available, as well as generate a response with the ability to set attributes (headers), the request completion code and the response body (in this implementations, there are methods for passing a string, some buffer or file in response). Each method in its implementation can throw an exception of type HttpRequestException.

If you look at the server code again, you can notice the following lines in the request processing code:

req->SetResponseAttr(Http::Response::Header::Server::Value, "MyTestServer");
req->SetResponseAttr(Http::Response::Header::ContentType::Value,
                     Http::Content::Type::html::Value);

This is the formation of the response header, and in this example, header fields such as “Content-Type” and “Server” are set. Despite the fact that libevent has a fairly wide functionality that goes far beyond the needs of HTTP, there is no list of header field constants in it; there is only an incomplete list of return codes (most commonly used). In order not to mess with the strings defining the header fields (for example, to avoid typos in the user code), all constants are already defined in the proposed wrapper over libevent.

String constants can be defined as simple macros in the old style of pure C in the header files, and to spread their declarations and definitions between .h and .cpp files while making them typified already in the C ++ style. However, you can do without file spacing, and make all typed definitions in C ++ style only in the header file. To do this, you can use some approach with templates and write such a macro (macros, of course, an evil recognized by C ++, as well as balm in small doses; heterogeneous solutions have greater viability).

In almost the same way, constants are defined for setting the type of content; have a slight modification. There was a desire to implement a search for content type by file extension for convenience when sending files in response to a request.

If you want to get something from an incoming request, for example, from which host and from which page the transition to the requested resource was made and, for example, does the user have cookies, you can get all this from the header of the incoming request this way:

std::string Host = req->GetHeaderAttr(Http::Request::Header::Host::Value);
std::string Referer = req->GetHeaderAttr(Http::Request::Header::Referer::Value);
std::string Cookie = req->GetHeaderAttr(Http::Request::Header::Cookie::Value);

Similarly, in the response, you can, for example, set the user some Cookies, which later work with his session and track if he wants to wander around your resource (an example of working with response headers is given in the server code).

If you want to organize some of your API via HTTP, then it is just as easy to do. Suppose you need to create methods: opening a session, obtaining statistical information about the server, and closing the session. Let for this query string to your server look something like this:

http://myserver.com/service/login/OpenSession?user=nym&pwd=kakoyto
http://myserver.com/service/login/CliseSession?sessionId=nym1234567890
http://myserver.com/service/stat/GetInfo?sessionId=nym1234567890

By responding to these query strings, the user server may generate some kind of response, for example, in xml format. This is the business of the server developer. But how to work with such requests, get parameters from them is given below:

auto Path = req->GetPath();
auto Params = req->GetParams();

One of the ways for the examples above would be / service / login / OpenSession, and the parameters are a map of the passed key / value pairs. Parameter Card Type:

typedef std::unordered_map RequestParams;

After analyzing everything that can be implemented using the proposed final version of the wrapper over libevent, you can look under the hood of this wrapper itself.

The request processing function can be viewed in the full version by downloading the source files of the examples, it has become a little more than in the previous examples, and has ceased to claim lambda without loss of readability of the code. I also did not cite the implementation of the IHttpRequest interface, since it is of little interest in its routine work with the libevent buffer. As for the rest, if you look at the code of the final version, it has not changed much. A small modification and a little "tuning" was added.

The user server is not required to process all types of http requests. You can specify a list of request types that the server should process, and for this, libevent has the evhttp_set_allowed_methods function (by default, the wrapper defines only the type of GET requests). When specifying a list of processed requests for all the rest, libevent itself will report the impossibility of executing such a request, thereby saving the user from additional checks.

Inquisitiveness of the mind, it is different: aimed at creation and destruction. From the destructive inquisitiveness of the mind with the desire to “fill up” the server, sending it some kind of prohibitively large http-packet header or creating a large request body can also be proactively protected by the evhttp_set_max_headers_size and evhttp_set_max_body_size functions. Of course, sending large requests can be caused not only by bad thoughts, but also by other reasons. The above methods will help to slightly reduce unwanted crashes of your server. It is possible to provide for something else, but otherwise you can already react reactively, which usually happens ...

In the end, I will give the final version, which processes GET requests (gives files from the specified directory) and displays from which host the request was made and from which page the transition to the resource processed by the server was made.

Conclusion

In addition to the functionality discussed, libevent still contains many useful features. In general: there is still something to try to write using this library and what to write about. This post showed only its small part, designed for the development of http-servers. All source files for all the examples are available on github . Based on the above examples, the http server was built from another of my home projects .
Test Result:

Two and a half thousand processed requests for an archive with source files of examples of a post in ten seconds on an already very modest configuration of iron ...

Thank you all for your attention!

Materials

• libevent
• WebServer as a test task
• HTTP
• List of HTTP header fields