Asynchronous HTTP requests in C ++: incoming through RESTinio, outgoing through libcurl. Part 1

    Preamble


    Our team is developing a small, easy-to-use, embeddable, asynchronous HTTP server for modern C ++ called RESTinio . They started to do it because it needed exactly asynchronous processing of incoming HTTP requests, but there was nothing ready for us to like. As life shows, asynchronous processing of HTTP requests in C ++ applications is necessary not only for us. Recently, developers from the same company got in touch with the question of whether it is possible to somehow make friends with asynchronous processing of incoming requests in RESTinio with issuing asynchronous outgoing requests via libcurl .

    As we clarified the situation, we found that this company was faced with conditions that we ourselves had to deal with, and because of which we started developing RESTinio. The bottom line is that a C ++ application accepts an incoming HTTP request. In the process of processing the request, the application needs to access a third-party server. This server can respond for quite some time. Say 10 seconds (although 10 seconds is still good). If you make a synchronous request to a third-party server, then the working thread is blocked on which the HTTP request is made. And this begins to limit the number of concurrent requests that the application can serve.

    The solution is that the application can asynchronously process all requests: both incoming and outgoing. Then, on a limited pool of working threads (or even on a single working thread), it will be possible to process tens of thousands of requests at the same time, even if the processing time of one request is tens of seconds.

    The trick was that the outgoing HTTP request application already used libcurl. But in the form of curl_easy , i.e. all requests were executed synchronously. They asked us, is it possible to combine RESTinio and curl_multi? The question for ourselves was interesting, because before libcurl in the form of curl_multi it was not necessary to apply. Therefore, it was interesting to immerse ourselves in this topic.

    Plunged. Got a lot of impressions. We decided to share with readers. Maybe someone will be interested in how you can live with curl_multi. For, as practice has shown, it is possible to live. But carefully ...;) What we will talk about in a small series of articles based on the experience of implementing a simple simulation of the situation described above with a slowly responding third-party service.

    Necessary disclaimers


    In order to prevent a useless and unconstructive flame in the comments (like what happened with the previous article ), I want to make a few warnings:

    • firstly, further we will talk about C ++. If you do not like C ++, if you think that C ++ is not a place in the modern world in general and in similar tasks in particular, then this article is not for you. And we have no purpose to convince someone that C ++ is good and should be used in such tasks. We are just talking about how you can solve a similar problem in C ++ if you suddenly had to do it in C ++. Also, we will not argue about why this may be required and why in real life you can’t just take and rewrite existing C ++ code to something else;
    • secondly, in C ++ there is no universally accepted code convention, therefore, any claims from the adherents of camelCase, PascalCase, Camel_With_Underscores_Case or even UPPER_CASE will not be accepted. We tried to bring the code in a more or less similar to the K & R style, so that it looked familiar to the largest number of readers. Because our "corporate" style of C ++ code design is definitely not accepted by everyone. However, if the appearance of the code violates your aesthetic feelings and you are ready to express your weighty “phi” in the comments about this, then please think about this: there is always someone who does not like the style you use. Is always. Regardless of which style you use;
    • thirdly, the code shown by us in no way claims to be the model of quality and reliability. This is not a production code. What you will see is a quick-and-dirty prototype that was sculpted on the knee literally in a day and another day was spent trying to comb the resulting code a little bit and provide it with explanatory comments. So claims like “yes, who writes it this way” or “you need to beat hands on such a shit” are not accepted, because we ourselves express them;)

    In general, if you do not like any of the above conditions, then we apologize for the time taken. Further reading makes no sense. Well, if these warnings do not scare you, then get comfortable. We hope you find it interesting.

    What is the essence of the developed simulation?


    For demonstration purposes, we have made several applications using RESTinio and libcurl. The simplest of them is a simulator of a third-party, slowly responding server called delay_server. To start the simulation, you need to run delay_server with the necessary set of parameters (address, port, desired delay times for responses).

    Also in the simulation includes several "fronts", called bridge_server_ *. It is bridge_server that accepts requests from the user and forwards requests to delay_server. It is assumed that the user first starts delay_server, then one of the bridge_server s, after which he begins to "shell" the bridge_server in a convenient way. For example, through curl / wget or utilities like ab / wrk.

    The simulation includes three bridge_server implementations:

    • bridge_server_1. A very simple option, which uses only two working threads. On one, RESTinio processes incoming HTTP requests, and on the second , outgoing HTTP requests are executed using curl_multi_perform . This implementation will be discussed in the second part of the series;
    • bridge_server_1_pipe. A more complicated version of bridge_server_1. There are also two working threads, but an additional pipe is used to transfer notifications from the RESTinio thread to the libcurl thread. Initially, we did not plan to describe this implementation, but if someone has interest, then it will be possible to consider bridge_server_1_pipe in detail in an additional article;
    • bridge_server_2. A more complex version that uses a pool of work threads. Moreover, this pool serves both RESTinio and libcurl ( curl_multi_socket_action is used ). This implementation will be considered in the final part of the series.

    Let's start this series with a description of the implementation of delay_server. Fortunately, this is the simplest and, possibly, the most understandable part. Bridge_server implementations will be much harder.

    delay_server


    What does delay_server do?


    delay_server accepts HTTP GET requests for URLs of the form / YYYY / MM / DD, where YYYY, MM and DD are numerical values. Delay_server responds to all other requests with a code of 404.

    If an HTTP GET request arrives at a URL of the form / YYYY / MM / DD, then delay_server pauses and then responds with a small text that says “Hello, World” and the length of the pause. For example, if you run delay_server with parameters:

    delay_server -a localhost -p 4040 -m 1500 -M 4000

    those. he will listen on localhost: 4040 and pause for responses between 1.5s and 4.0s. If then execute:

    curl -4 http: // localhost: 4040/2018/02/22

    then we get:

    Hello world!
    Pause: 2347ms.


    Well, or you can enable tracing of what is happening. For a server, this is:

    delayed_server -a localhost -p 4040 -m 1500 -M 4000 -t

    For curl, this is:

    curl -4 -v http: // localhost: 4040/2018/02/22

    For delay_server, we will see something like:

    [2018-02-22 16: 47: 54.441] TRACE: starting server on 127.0.0.1:4040
    [2018-02-22 16: 47: 54.441] INFO: init accept # 0
    [2018-02-22 16: 47: 54.441] INFO: server started on 127.0.0.1:4040
    [2018-02-22 16: 47: 57.040] TRACE: accept connection from 127.0.0.1haps8468 on socket # 0
    [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] start connection with 127.0.0.1{8468
    [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] start waiting for request
    [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] continue reading request
    [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] received 88 bytes
    [2018-02-22 16: 47: 57.041] TRACE: [connection: 1] request received (# 0): GET / 2018/02/22
    [2018-02-22 16: 47: 59.401] TRACE: [connection: 1] append response (# 0), flags: {final_parts, connection_keepalive}, bufs count: 2
    [2018-02-22 16: 47: 59.401] TRACE: [connection: 1] sending resp data, buf count: 2
    [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] outgoing data was sent: 206 bytes
    [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] should keep alive
    [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] start waiting for request
    [2018-02-22 16: 47: 59.402] TRACE: [connection: 1] continue reading request
    [2018-02-22 16: 47: 59.403] TRACE: [connection: 1] EOF and no request, close connection
    [2018-02-22 16: 47: 59.403] TRACE: [connection: 1] close
    [2018-02-22 16: 47: 59.403] TRACE: [connection: 1] destructor called

    and for curl:

    * Trying 127.0.0.1 ...
    * TCP_NODELAY set
    * Connected to localhost (127.0.0.1) port 4040 (# 0)
    > GET / 2018/02/22 HTTP / 1.1
    > Host: localhost: 4040
    > User-Agent: curl / 7.58.0
    > Accept: * / *
    >
    <HTTP / 1.1 200 OK
    <Connection: keep-alive
    <Content-Length: 28
    <Server: RESTinio hello world server
    <Date: Thu, 22 Feb 2018 13:47:59 GMT
    <Content-Type: text / plain; charset = utf-8
    <
    Hello world!
    Pause: 2360ms.
    * Connection # 0 to host localhost left intact

    How does delay_server do this?


    delay_server is a simple single-threaded C ++ application. A built-in HTTP server is launched on the main thread, which pulls the user-assigned callback when it receives a request to a suitable URL. This callback creates an Asio-shny timer and raises the created timer to a randomly selected pause (the pause is selected so as to fall within the limits set at start of delay_server). After that, callback returns control to the HTTP server, which allows the server to accept and process the next request. When a cocked timer is triggered by a callback, a response to a previously received HTTP request is generated and sent.

    Parsing the implementation of delay_server


    Main () function


    We begin the analysis of the delay_server implementation immediately with the main () function, gradually explaining what is happening inside and outside main () - a.

    So, the main () code looks like this:

    int main(int argc, char ** argv) {
      try {
        const auto cfg = parse_cmd_line_args(argc, argv);
        if(cfg.help_requested_)
          return 1;
        // Нам нужен собственный io_context для того, чтобы мы могли с ним
        // работать напрямую в обработчике запросов.
        restinio::asio_ns::io_context ioctx;
        // Так же нам потребуется генератор случайных задержек в выдаче ответов.
        pauses_generator_t generator{cfg.config_.min_pause_, cfg.config_.max_pause_};
        // Нам нужен обработчик запросов, который будет использоваться
        // вне зависимости от того, какой именно сервер мы будем запускать
        // (с трассировкой происходящего или нет).
        auto actual_handler = [&ioctx, &generator](auto req, auto /*params*/) {
            return handler(ioctx, generator, std::move(req));
          };
        // Если должна использоваться трассировка запросов, то должен
        // запускаться один тип сервера.
        if(cfg.config_.tracing_) {
          run_server(
              ioctx, cfg.config_, std::move(actual_handler));
        }
        else {
          // Трассировка не нужна, запускается другой тип сервера.
          run_server(
              ioctx, cfg.config_, std::move(actual_handler));
        }
        // Все, теперь ждем завершения работы сервера.
      }
      catch( const std::exception & ex ) {
        std::cerr << "Error: " << ex.what() << std::endl;
        return 2;
      }
      return 0;
    }

    What's going on here?

    First, we parse the command line arguments and get the configuration object for delay_server.

    Secondly, we create several objects that we need:

    • an instance of asio :: io_context, which will be used both for processing IO-operations of the HTTP server and for timers that will be cocked in the handler of incoming HTTP requests;
    • random delay generator, which is needed just to make the HTTP server respond slowly to requests;
    • the lambda function stored in the actual_handler variable, which will be the very same callback called by the HTTP server for incoming HTTP requests. This callback must have a specific format. But the handler () function, which performs the actual processing of requests and which will be discussed below, has a different format and requires additional arguments. Here is the lambda function and captures the necessary handler () - the arguments, exposing the signature that RESTinio requires.

    Thirdly, we are launching an HTTP server. But the launch is done taking into account whether the user wants to see the server operation trace or not. This is where a small template magic comes into play, which we actively use in RESTinio and which we have talked a bit about earlier .

    That, in fact, is the whole delay_server :)

    But the devil, as usual, is in the details. Therefore, let's go further, consider what is hiding behind these simple actions.

    Command line configuration and parsing


    Delay_server uses a very simple structure to describe the server configuration:

    // Конфигурация, которая потребуется серверу.
    struct config_t {
      // Адрес, на котором нужно слушать новые входящие запросы.
      std::string address_{"localhost"};
      // Порт, на котором нужно слушать.
      std::uint16_t port_{8090};
      // Минимальная величина задержки перед выдачей ответа.
      milliseconds min_pause_{4000};
      // Максимальная величина задержки перед выдачей ответа.
      milliseconds max_pause_{6000};
      // Нужно ли включать трассировку?
      bool tracing_{false};
    };

    Parsing the command line is quite voluminous, so we won’t dive into it much. But those who wish can look under the spoiler to make an impression of what is happening.

    Details of parsing command line arguments
    // Разбор аргументов командной строки.
    // В случае неудачи порождается исключение.
    auto parse_cmd_line_args(int argc, char ** argv) {
      struct result_t {
        bool help_requested_{false};
        config_t config_;
      };
      result_t result;
      long min_pause{result.config_.min_pause_.count()};
      long max_pause{result.config_.max_pause_.count()};
      // Подготавливаем парсер аргументов командной строки.
      using namespace clara;
      auto cli = Opt(result.config_.address_, "address")["-a"]["--address"]
            ("address to listen (default: localhost)")
        | Opt(result.config_.port_, "port")["-p"]["--port"]
            ("port to listen (default: 8090)")
        | Opt(min_pause, "minimal pause")["-m"]["--min-pause"]
            ("minimal pause before response, milliseconds")
        | Opt(max_pause, "maximum pause")["-M"]["--max-pause"]
            ("maximal pause before response, milliseconds")
        | Opt(result.config_.tracing_)["-t"]["--tracing"]
            ("turn server tracing ON (default: OFF)")
        | Help(result.help_requested_);
      // Выполняем парсинг...
      auto parse_result = cli.parse(Args(argc, argv));
      // ...и бросаем исключение если столкнулись с ошибкой.
      if(!parse_result)
        throw std::runtime_error("Invalid command line: "
            + parse_result.errorMessage());
      if(result.help_requested_)
        std::cout << cli << std::endl;
      else {
        // Некоторые аргументы нуждаются в дополнительной проверке.
        if(min_pause <= 0)
          throw std::runtime_error("minimal pause can't be less or equal to 0");
        if(max_pause <= 0)
          throw std::runtime_error("maximal pause can't be less or equal to 0");
        if(max_pause < min_pause)
          throw std::runtime_error("minimal pause can't be less than "
              "maximum pause");
        result.config_.min_pause_ = milliseconds{min_pause};
        result.config_.max_pause_ = milliseconds{max_pause};
      }
      return result;
    }

    For analysis, we tried to use the new Clara library from the author of the widely known in narrow circles library for unit tests in C ++ called Catch2 (just Catch in girlhood).

    In general, there is nothing complicated except for one trick: the parse_cmd_line_args function returns an instance of a locally defined structure. In a good way, something like this should be returned here:

    struct help_requested_t {};
    using cmd_line_args_parsing_result_t = variant;

    But in C ++ 14 there is no std :: variant, but I did not want to drag any implementation of variant / either from a third-party library or rely on the presence of std :: experimental :: variant. Therefore, they did it like this. The code, of course, smacks, but for simulated on the knee will do.

    Random delay generator


    Everything is simple here, in principle, there is nothing to discuss. Therefore, just a code. For the sake of being.

    Pauses_generator_t implementation
    // Вспомогательный тип для генерации случайных задержек.
    class pauses_generator_t {
      std::mt19937 generator_{std::random_device{}()};
      std::uniform_int_distribution distrib_;
      const milliseconds minimal_;
    public:
      pauses_generator_t(milliseconds min, milliseconds max)
        : distrib_{0, (max - min).count()}
        , minimal_{min}
        {}
      auto next() {
        return minimal_ + milliseconds{distrib_(generator_)};
      }
    };

    It is only necessary to pull the next () method when necessary and a random value in the range [min, max] will be returned.

    Handler () function


    One of the key elements of the delay_server implementation is the small handler () function, inside which the processing of incoming HTTP requests takes place. Here is the full code for this function:

    // Реализация обработчика запросов.
    restinio::request_handling_status_t handler(
        restinio::asio_ns::io_context & ioctx,
        pauses_generator_t & generator,
        restinio::request_handle_t req) {
      // Выполняем задержку на случайную величину (но в заданных пределах).
      const auto pause = generator.next();
      // Для отсчета задержки используем Asio-таймеры.
      auto timer = std::make_shared(ioctx);
      timer->expires_after(pause);
      timer->async_wait([timer, req, pause](const auto & ec) {
          if(!ec) {
            // Таймер успешно сработал, можно генерировать ответ.
            req->create_response()
              .append_header(restinio::http_field::server, "RESTinio hello world server")
              .append_header_date_field()
              .append_header(restinio::http_field::content_type, "text/plain; charset=utf-8")
              .set_body(
                fmt::format("Hello world!\nPause: {}ms.\n", pause.count()))
              .done();
          }
        } );
      // Подтверждаем, что мы приняли запрос к обработке и что когда-то
      // мы ответ сгенерируем.
      return restinio::request_accepted();
    }

    This function (through the lambda created in main () - e) is called every time the HTTP server receives an incoming GET request to the desired URL. The incoming HTTP request itself is passed in the req parameter of type restinio :: request_handle_t.

    This very restinio :: request_handle_t is a smart pointer to an object with the contents of an HTTP request. That allows you to save the req value and use it later. This is exactly one of the cornerstones of RESTinio asynchrony: RESTinio pulls a callback provided by the user and passes an instance of request_handle_t to this callback. The user can either immediately generate an HTTP response inside the callback (and then it will be trivial synchronous processing), or he can save req to himself or pass req to some other thread. Then return control to RESTinio. And to formulate an answer later, when the right time comes for this.

    In this case, an asio :: steady_timer instance is created and req is stored in the lambda function passed to async_wait for the timer. Accordingly, the HTTP request object is stored until the timer goes off.

    A very important point in handler () - e is the value it returns. By the return value, RESTinio understands whether the user took responsibility for generating a response to the request or not. In this case, the request_accepted value is returned, which means that the user promised RESTinio to generate a response to the incoming HTTP request later.

    But if handler () returned, say, request_rejected (), then RESTinio would finish processing the request and respond to the user with code 501.

    So, handler () is called when an incoming HTTP request arrives at the desired URL (why this is described below). The handler calculates the amount of delay for the response. Then a timer is created and cocked. When the timer goes off, a response will be generated to the request. Well, handler () promises RESTinio to generate a response to the request by returning request_accepted.

    That, in fact, is all. A small trifle: fmtlib is used to form the response body . In principle, one could do without it here. But, firstly, we really like fmtlib and we use fmtlib whenever we can. And secondly, we still needed fmtlib in bridge_server, so there was no reason to refuse it in delay_server.

    Function run_server ()


    The run_server () function is responsible for configuring and starting the HTTP server. It determines what requests the HTTP server will process and how the HTTP server will respond to all other requests.

    Also, run_server () determines where the HTTP server will work. For the case of delay_server, this will be the main thread of the application.

    Let's look at the run_server () code first, and then look at a few important points that we haven't talked about yet.

    So here is the code:

    template
    void run_server(
        restinio::asio_ns::io_context & ioctx,
        const config_t & config,
        Handler && handler) {
      // Сперва создадим и настроим объект express-роутера.
      auto router = std::make_unique();
      // Вот этот URL мы готовы обрабатывать.
      router->http_get(
          R"(/:year(\d{4})/:month(\d{2})/:day(\d{2}))",
          std::forward(handler));
      // На все остальное будем отвечать 404.
      router->non_matched_request_handler([](auto req) {
          return req->create_response(404, "Not found")
              .append_header_date_field()
              .connection_close()
              .done();
        });
      restinio::run(ioctx,
          restinio::on_this_thread()
            .address(config.address_)
            .port(config.port_)
            .handle_request_timeout(config.max_pause_)
            .request_handler(std::move(router)));
    }

    What is happening in it and why is it happening this way?

    Firstly, a delay similar to expressjs request routing system will be used for delay_server . In RESTinio, this is called an Express router .

    You need to create an instance of an object that is responsible for routing queries based on regular expressions. After that, you need to put a list of routes in this object and set each route to its own handler. What we do. Create a handler:

    auto router = std::make_unique();

    And indicate the route we are interested in:

    router->http_get(
          R"(/:year(\d{4})/:month(\d{2})/:day(\d{2}))",
          std::forward(handler));

    Then we also set the handler for all other requests. Which will simply respond with 404 code:

    router->non_matched_request_handler([](auto req) {
          return req->create_response(404, "Not found")
              .append_header_date_field()
              .connection_close()
              .done();
        });

    This completes the preparation of the Express router we need.

    Secondly, when calling run (), we indicate that the HTTP server must use the specified io_context and must work on the very thread on which the run () call was made. In addition, the parameters from the configuration are set for the server (since the IP address and port, the maximum allowable time for processing requests and the processor itself):

    restinio::run(ioctx,
        restinio::on_this_thread()
          .address(config.address_)
          .port(config.port_)
          .handle_request_timeout(config.max_pause_)
          .request_handler(std::move(router)));

    Here, using on_this_thread just makes RESTinio start the HTTP server in the context of the same thread.

    Why is run_server () a template?


    The run_server () function is a template function that depends on two parameters:

    template
    void run_server(
        restinio::asio_ns::io_context & ioctx,
        const config_t & config,
        Handler && handler);

    In order to explain why this is so, we start with the second template parameter - Handle.

    Inside main (), we create an actual request handler in the form of a lambda function. Only the compiler knows the real type of this lambda. Therefore, in order to pass the lambda handler to run_server (), we need the template parameter Handle. With it, the compiler will infer the desired type of handler argument in run_server ().

    But with the Server_Traits parameter the situation is a bit more complicated. The fact is that the HTTP server in RESTinio needs to set a set of properties that will determine various aspects of the server’s behavior and implementation. For example, whether the server will be adapted to work in multi-threaded mode. Will the server log the operations it performs, etc. All this is set by the Traits template parameter for the restinio :: http_server_t class. In this example, this class is not visible, because an instance of http_server_t is created inside run (). But still Traits must be set. Just the template parameter Server_Traits of the run_server () function and sets Traits for http_server_t.

    We in delay_server needed to define two different types of Traits:

    // Мы будем использовать express-router. Для простоты определяем псевдоним
    // для нужного типа.
    using express_router_t = restinio::router::express_router_t<>;
    // Так же нам потребуются два вспомогательных типа свойств для http-сервера.
    // Первый тип для случая, когда трассировка сервера не нужна.
    struct non_traceable_server_traits_t : public restinio::default_single_thread_traits_t {
      using request_handler_t = express_router_t;
    };
    // Второй тип для случая, когда трассировка сервера нужна.
    struct traceable_server_traits_t : public restinio::default_single_thread_traits_t {
      using request_handler_t = express_router_t;
      using logger_t = restinio::single_threaded_ostream_logger_t;
    };

    The first type, non_traceable_server_traits_t, is used when the server does not need to log its actions. The second type, traceable_server_traits_t, is used when logging should be.

    Accordingly, inside the main () function, depending on the presence or absence of the "-t" key, the run_server () function is called either with non_traceable_server_traits_t or with traceable_server_traits_t:

    // Если должна использоваться трассировка запросов, то должен
    // запускаться один тип сервера.
    if(cfg.config_.tracing_) {
      run_server(
          ioctx, cfg.config_, std::move(actual_handler));
    }
    else {
      // Трассировка не нужна, запускается другой тип сервера.
      run_server(
          ioctx, cfg.config_, std::move(actual_handler));
    }

    So assigning the required properties to the HTTP server is another reason why run_server () is a template function.

    The Traits topic for restinio :: http_server_t is discussed in more detail in our previous article on RESTinio .

    Conclusion of the first part


    That, in fact, is all that could be said about the implementation of delay_server based on RESTinio. We hope that the material described is clear. If not, we will be happy to answer questions in the comments.

    In subsequent articles, we will talk about integration examples of RESTinio and curl_multi, parsing the implementations of bridge_server_1 and bridge_server_2. There, parts that relate specifically to RESTinio will not be more voluminous or more complicated than what we showed in this article. And the bulk of the code and the main complexity will result from curl_multi. But this is a completely different story ...

    To be continued .

    Also popular now: