Shrimp: Scale and Share HTTP Images in Modern C ++ with ImageMagic ++, SObjectizer, and RESTinio



    Foreword


    Our small team is engaged in the development of two OpenSource tools for C ++ developers - Actor framework SObjectizer and embedded HTTP-server RESTinio . However, we regularly come across a couple of non-trivial questions:

    • which features to add to the library, and which to leave “overboard”?
    • how to clearly demonstrate the "ideologically correct" ways to use the library?

    It’s good when the answers to such questions appear during the use of our developments in real projects, when developers come to us with their complaints or Wishlist. Due to the satisfaction of users' wishes, we fill our tools with functionality that is dictated by life itself, and not “sucked out of the finger”.

    But information reaches us far from all the problems and difficulties that users face. And we can’t always use the information received, and especially the code examples, in our public materials.

    Therefore, sometimes we think up small problems for ourselves, solving which we are forced to turn from developers of tools into users. This allows us to look at our own tools with different eyes and understand for ourselves what is good, what is not good, what is missing, and what is too much.

    Today we want to tell just about one such "small" task, in which SObjectizer and RESTinio naturally came together.

    Scaling and distribution of pictures. Why exactly this?


    As a small demo task for ourselves, we chose an HTTP server that distributes scaled images upon request. You put the images in some directory, start the HTTP server, make a request to it of the form:

    curl "http://localhost:8080/my_picture.jpg?op=resize&max=1920"

    and in return you get a picture scaled to 1920 pixels on the long side.

    The choice fell on this task because it perfectly demonstrates the scenarios for which we at one time started developing RESTinio: there is a long-running and debugged code in C or C ++ to which you need to attach an HTTP input and start responding to incoming requests. At the same time, which is important, application processing of a request can take considerable time and therefore it is unprofitable to pull application code directly on the IO context. The HTTP server should be asynchronous: accept and parse the HTTP request, give the parsed request somewhere for further application processing, proceed to service the next HTTP request, return to returning the response to the HTTP request when this response is prepared by someone.

    This is exactly what happens when processing requests for scaling images. An HTTP server is able to do its direct work (i.e. reading data, parsing an HTTP request) in a fraction of a millisecond. But scaling a picture can take tens, hundreds, or even thousands of milliseconds.

    And since it can take a lot of time to scale one picture, you need to make sure that the HTTP server can continue to work while the picture is scaled. To do this, we need to spread the work of the HTTP server and scaling images to different working contexts. In the simple case, these will be different work threads. Well, since we live in multi-core processors, we will have several working threads. Some of them will serve HTTP requests, some will work with images.

    It turns out that for distributing scalable images via HTTP, we need to reuse the long-written, working C / C ++ code (in this case ImageMagic ++), and serve HTTP requests asynchronously, and perform application processing of requests in several workflows. An excellent task for RESTinio and SObjectizer, as it seemed to us.

    And we decided to name our demo project shrimp.

    Shrimp as it is


    What does Shrimp do?


    Shrimp runs as a console application, opens and listens on the specified port, receives and processes HTTP GET requests of the form:

    /.
    /.?op=resize&=

    Where:

    • image is the name of the image file to scale. For example, my_picture or DSCF0069;
    • ext is one of the extensions supported by shrimp (jpg, jpeg, png or gif);
    • side is an indication of the side for which the size is set. It can either have a value of width, in this case the picture is scaled so that the resulting width is equal to the specified value, the height of the picture is automatically selected while maintaining the aspect ratio. Or the value of height, in this case, scaling occurs in height. Either max, in this case the long side is limited, and shrimp itself determines whether the long side is height or width;
    • value is the size at which scaling occurs.

    If only the file name is specified in the URL, without the resize operation, then shrimp simply returns the original image in the response. If the resize operation is specified, then shrimp changes the size of the requested image and gives the scaled version.

    At the same time, shrimp keeps in memory a cache of scaled images. If a picture is repeatedly requested with the same resize parameters, which is already in the cache, then the value from the cache is returned. If there is no picture in the cache, then the picture is read from the disk, scaled, stored in the cache and returned in response.

    The cache is periodically cleared. Pictures that have lived in the cache for more than an hour since the last access to them are pushed out of it. Also, the oldest pictures are thrown out of the cache if the cache exceeds its maximum size (in a demo project it is 100Mb).

    We have prepared a page by going to which anyone can experiment with shrimp:



    On this page you can set the image size and click "Resize". Two requests will be made to the shrimp server with the same parameters. Most likely, the first request will be unique (i.e. there will not be a cache with such resize parameters in the cache yet), so the first request will take some time to actually scale the image. And the second request, most likely, will find the already scaled picture in the cache and give it right away.

    It is possible to judge whether a picture is given from the cache or whether it was really scaled by the text under the picture. For example, the text “Transformed (114.0ms)” indicates that the picture was scaled and the zoom operation took 114 milliseconds.

    How does Shrimp do it?


    Shrimp is a multi-threaded application that runs three groups of work threads:

    1. The pool of work threads running the HTTP server. On this pool, new connections are served, incoming requests are received and parsed, responses are generated and sent. The HTTP server is implemented through the RESTinio library.
    2. A separate working thread on which the transform_manager SObjectizer agent runs. This agent processes requests received from the HTTP server and maintains a cache of scaled images.
    3. The thread pool on which SObjectizer agents are working transformers. They perform the actual scaling of images using ImageMagic ++.

    It turns out the following working scheme: The



    HTTP server accepts an incoming request, parses it, and checks for correctness. If this request does not require a resize operation, then the HTTP server itself processes the request through the sendfile operation . If the request requires a resize operation, the request is sent asynchronously to the transform_manager agent.

    The transform_manager agent receives requests from the HTTP server, checks for the presence of already scaled pictures in the cache. If there is a picture in the cache, then transform_manager immediately generates a response for the HTTP server. If there is no picture, then transform_manager sends a request to scale the picture to one of the transformer agents. When the scaling result comes from the transformer, the result is stored in the cache and an answer is generated for the HTTP server.

    The transformer agent receives requests from transform_manager, processes them, and returns the result of the transformation back to the transform_manager agent.

    What does Shrimp have under the hood?


    The source code for the most minimal version of shrimp described in this article can be found in this repository: shrimp-demo on BitBucket or on GitHub .

    There is a lot of code, although, for the most part, in this version of shrimp, the code is quite trivial. However, it makes sense to focus on some aspects of the implementation.

    Using C ++ 17 and the most recent compiler versions


    In the implementation of shrimp, we decided to use C ++ 17 and the latest versions of compilers, in particular GCC 7.3 and 8.1. The project is heavily research. Therefore, the practical acquaintance of C ++ 17 in the framework of such a project is natural and permissible. Whereas in more mundane developments focused on practical industrial applications here and now, we are forced to look back at rather old compilers and use perhaps C ++ 14, or even just a subset of C ++ 11.

    I must say that C ++ 17 makes a good impression. It seems that we didn’t use so many innovations from the seventeenth standard in the shrimp code, but they had a positive effect: the attribute [[nodiscard]], std :: optional / std :: variant / std :: filesystem directly “ out of the box ”, and not from external dependencies, structured binding, if constexpr, the ability to assemble visitor for lambdas for std :: visit ... Individually, these are all trifles, but together they produce a powerful cumulative effect.

    So the first useful result that we got while developing shrimp: C ++ 17 is worth it to switch to it.

    HTTP server using RESTinio tools


    Perhaps the easiest part of shrimp turned out to be the HTTP server and the HTTP GET request handler ( http_server.hpp and http_server.cpp ).

    Receive and dispatch incoming requests


    Essentially, all the basic logic of the shrimp HTTP server is concentrated in this function:

    void
    add_transform_op_handler(
       const app_params_t & app_params,
       http_req_router_t & router,
       so_5::mbox_t req_handler_mbox )
    {
       router.http_get(
          R"(/:path(.*)\.:ext(.{3,4}))",
             restinio::path2regex::options_t{}.strict( true ),
             [req_handler_mbox, &app_params]( auto req, auto params )
             {
                if( has_illegal_path_components( req->header().path() ) )
                {
                   return do_400_response( std::move( req ) );
                }
                const auto opt_image_format = image_format_from_extension(
                      params[ "ext" ] );
                if( !opt_image_format )
                {
                   return do_400_response( std::move( req ) );
                }
                if( req->header().query().empty() )
                {
                   return serve_as_regular_file(
                         app_params.m_storage.m_root_dir,
                         std::move( req ),
                         *opt_image_format );
                }
                const auto qp = restinio::parse_query( req->header().query() );
                if( "resize" != restinio::value_or( qp, "op"sv, ""sv ) )
                {
                   return do_400_response( std::move( req ) );
                }
                handle_resize_op_request(
                      req_handler_mbox,
                      *opt_image_format,
                      qp,
                      std::move( req ) );
                return restinio::request_accepted();
       } );
    }

    This function prepares the HTTP GET request handler using the RESTinio ExpressJS router . When the HTTP server receives a GET request, the URL of which falls under the given regular expression, the specified lambda function is called.

    This lambda function makes a few simple checks on the correctness of the request, but in the main, its work comes down to a simple choice: if resize is not set, the requested picture will be returned in its original form using an effective system sendfile. If resize mode is set, then a message is generated and sent to the transform_manager agent:

    void
    handle_resize_op_request(
       const so_5::mbox_t & req_handler_mbox,
       image_format_t image_format,
       const restinio::query_string_params_t & qp,
       restinio::request_handle_t req )
    {
       try_to_handle_request(
          [&]{
             auto op_params = transform::resize_params_t::make(
                restinio::opt_value< std::uint32_t >( qp, "width" ),
                restinio::opt_value< std::uint32_t >( qp, "height" ),
                restinio::opt_value< std::uint32_t >( qp, "max" ) );
             transform::resize_params_constraints_t{}.check( op_params );
             std::string image_path{ req->header().path() };
             so_5::send<
                      so_5::mutable_msg>(
                   req_handler_mbox,
                   std::move(req),
                   std::move(image_path),
                   image_format,
                   op_params );
          },
          req );
    }

    It turns out that the HTTP server, having accepted the resize-request, gives it to the transform_manager agent via an asynchronous message, and continues to serve other requests.

    File sharing with sendfile


    If the HTTP server detects a request for the original picture, without the resize operation, the server immediately sends this picture through the sendfile operation. The main code associated with this is as follows (the full code for this function can be found in the repository ):

    [[nodiscard]]
    restinio::request_handling_status_t
    serve_as_regular_file(
       const std::string & root_dir,
       restinio::request_handle_t req,
       image_format_t image_format )
    {
       const auto full_path =
          make_full_path( root_dir, req->header().path() );
       try
       {
          auto sf = restinio::sendfile( full_path );
          ...
          return set_common_header_fields_for_image_resp(
                   file_stat.st_mtim.tv_sec,
                   resp )
                .append_header(
                   restinio::http_field::content_type,
                   image_content_type_from_img_format( image_format ) )
                .append_header(
                   http_header::shrimp_image_src,
                   image_src_to_str( http_header::image_src_t::sendfile ) )
                .set_body( std::move( sf ) )
                .done();
       }
       catch(...) {}
       return do_404_response( std::move( req ) );
    }

    The key point here is calling restinio :: sendfile () , and then passing the value returned by this function to set_body ().

    The restinio :: sendfile () function creates a file upload operation using the system API. When this operation is passed to set_body (), RESTinio understands that the contents of the file specified in restinio :: sendfile () will be used for the body of the HTTP response. Then it uses the system API to write the contents of this file to the TCP socket.

    Implementing the image cache


    The transform_manager agent stores the cache of converted images, where the images are placed after scaling. This cache is a simple self-made container that provides access to its contents in two ways:

    1. By searching for an element by key (similar to how this happens in the standard containers std :: map and std :: unordered_map).
    2. By accessing the oldest cache item.

    The first access method is used when we need to check the availability of the image in the cache. The second is when we delete the oldest pictures from the cache.

    We did not begin to search for something ready for these purposes on the Internet. Probably Boost.MultiIndex would be quite suitable here. But I did not want to drag Boost just for the sake of MultiIndex, so we made our trivial implementation literally on my knees. It seems to work;)

    Queue of pending requests in transform_manager


    The transform_manager agent, despite its rather decent size (an hpp file of about 250 lines and a cpp file of about 270 lines), in our simplest implementation of shrimp, turned out to be rather trivial, in our opinion.

    One of the points that makes a significant contribution to the complexity and volume of the agent code is the presence in transform_manager of not only the cache of transformed images, but also the queue of pending requests.

    We have a limited number of transformer agents (in principle, their number should approximately correspond to the number of available processing cores). If more requests come simultaneously than there are free transformers, then we can either immediately respond negatively to the request or queue the request. And then take it from the queue when a free transformer appears.

    In shrimp, we use a queue of waiting requests, which is defined as follows:

    struct pending_request_t
    {
       transform::resize_request_key_t m_key;
       sobj_shptr_t m_cmd;
       std::chrono::steady_clock::time_point m_stored_at;
       pending_request_t(
          transform::resize_request_key_t key,
          sobj_shptr_t cmd,
          std::chrono::steady_clock::time_point stored_at )
          : m_key{ std::move(key) }
          , m_cmd{ std::move(cmd) }
          , m_stored_at{ stored_at }
       {}
    };
    using pending_request_queue_t = std::queue;
    pending_request_queue_t m_pending_requests;
    static constexpr std::size_t max_pending_requests{ 64u };

    Upon receipt of the request, we put it in the queue with fixing the time of receipt of the request. Then we periodically check to see if the timeout for this request has expired. Indeed, in principle, it may happen that a bundle of “heavy” requests arrived earlier, the processing of which took too long. It is wrong to wait endlessly for a free transformer to appear, it is better to send a negative response to the client after some time, which means that the service is now overloaded.

    There is also a size limit for the queue of pending requests. If the queue has already reached its maximum size, then we immediately refuse to process the request and tell the client that we are overloaded.

    There are one important point related to the queue of pending requests, which we will focus on in the conclusion to the article.

    Type sobj_shptr_t and reusing message instances


    In determining the type of the queue of waiting requests, as well as in the signatures of some methods of transform_manager, you can see the use of the type sobj_shptr_t. It makes sense to dwell in more detail on what type it is and why it is used.

    The bottom line is that transform_manager receives a request from the HTTP server as a resize_request_t message:

    struct resize_request_t final : public so_5::message_t
    {
       restinio::request_handle_t m_http_req;
       std::string m_image;
       image_format_t m_image_format;
       transform::resize_params_t m_params;
       resize_request_t(
          restinio::request_handle_t http_req,
          std::string image,
          image_format_t image_format,
          transform::resize_params_t params )
          : m_http_req{ std::move(http_req) }
          , m_image{ std::move(image) }
          , m_image_format{ image_format }
          , m_params{ params }
       {}
    };

    and we have to do something to store this information in the queue of waiting requests. For example, you can create a new instance of resize_request_t and move the values ​​from the received message into it.

    And you can recall that the message itself in SObjectizer is a dynamically created object. And not a simple object, but with a link counter inside. And that in SObjectizer there is a special type of smart pointer for such objects - intrusive_ptr_t.

    Those. we can not make a copy of resize_request_t for the queue of waiting requests, but we can simply put in this queue a smart pointer to an existing instance of resize_request_t. What we do. And in order not to write everywhere the rather exotic name so_5 :: intrusive_ptr_t, we enter our alias:

    template
    using sobj_shptr_t = so_5::intrusive_ptr_t;
    

    Asynchronous responses to clients


    We said that HTTP requests are processed asynchronously. And we showed above how the HTTP server sends an inquiry to the transform_manager agent with an asynchronous message. But what happens to responses to HTTP requests?

    Responses are also served asynchronously. For example, in the transform_manager code you can see the following:

    void
    a_transform_manager_t::on_failed_resize(
       failed_resize_t & /*result*/,
       sobj_shptr_t cmd )
    {
       do_404_response( std::move(cmd->m_http_req) );
    }

    This code generates a negative response to the HTTP request in the case when the image could not be scaled for some reason. The response is generated in the do_404_response helper function, the code of which can be represented as follows:

    auto do_404_response( restinio::request_handle_t req )
    {
       auto resp = req->create_response( 404, "Not Found" );
       resp.append_header( restinio::http_field_t::server, "Shrimp draft server" );
       resp.append_header_date_field();
       if( req->header().should_keep_alive() )
          resp.connection_keep_alive();
       else
          resp.connection_close();
       return resp.done();
    }

    The first key point with do_404_response () is that this function is called on the working context of the transform_manager agent, and not on the working context of the HTTP server.

    The second key point is the call to the done () method on the fully formed resp object. All asynchronous magic with an HTTP response happens here. The done () method takes all the information prepared in resp and asynchronously sends it to the HTTP server. Those. a return from do_404_response () will occur immediately after the contents of the resp object are queued by the HTTP server.

    The HTTP server in its working context will detect the presence of a new HTTP response and will begin to perform the necessary actions to send the response to the appropriate client.

    Type datasizable_blob_t


    Another small point that makes sense to clarify, because it is probably incomprehensible without understanding the intricacies of RESTinio. We are talking about the presence of, at first glance, a strange type of datasizeable_blob_t, defined as follows:

    struct datasizable_blob_t
       : public std::enable_shared_from_this< datasizable_blob_t >
    {
       const void * data() const noexcept
       {
          return m_blob.data();
       }
       std::size_t size() const noexcept
       {
          return m_blob.length();
       }
       Magick::Blob m_blob;
       //! Value for `Last-Modified` http header field.
       const std::time_t m_last_modified_at{ std::time( nullptr ) };
    };

    In order to explain why this type is needed, you need to show how an HTTP response is formed with a transformed picture:

    void
    serve_transformed_image(
       restinio::request_handle_t req,
       datasizable_blob_shared_ptr_t blob,
       image_format_t img_format,
       http_header::image_src_t image_src,
       header_fields_list_t header_fields )
    {
       auto resp = req->create_response();
       set_common_header_fields_for_image_resp(
             blob->m_last_modified_at,
             resp )
          .append_header(
             restinio::http_field::content_type,
             image_content_type_from_img_format( img_format )  )
          .append_header(
             http_header::shrimp_image_src,
             image_src_to_str( image_src ) )
          .set_body( std::move( blob ) );
       for( auto & hf : header_fields )
       {
          resp.append_header( std::move( hf.m_name ), std::move( hf.m_value ) );
       }
       resp.done();
    }

    We pay attention to the call to set_body (): a smart pointer to the datasizable_blob_t instance is sent directly there. What for?

    The fact is that RESTinio supports several options for forming the body of an HTTP response . The simplest is to pass an instance of type std :: string to set_body () and RESTinio will save the value of this string inside the resp object.

    But there are times when the value for set_body () should be reused in several answers at once. For example, in shrimp this happens when shrimp receives several identical requests for transformation of the same image. In this case, it is unprofitable to copy the same value into each answer. Therefore, in RESTinio there is a set_body () variant of the form:
    template auto set_body(std::shared_ptr body);

    But in this case, an important limitation is imposed on type T: it must contain the public data () and size () methods, which are necessary so that RESTinio can access the contents of the response.

    The scaled image in shrimp is stored as a Magick :: Blob object. There is a data method in the Magic :: Blob type, but there is no size () method, but there is a length () method. Therefore, we needed the wrapper class datasizable_blob_t, which provides RESTinio with the necessary interface for accessing the value of Magick :: Blob.

    Periodic messages in transform_manager


    The transform_manager agent needs to do several things from time to time:

    • Pull pictures that have been in the cache for too long from the cache.
    • control the time spent by requests in the waiting queue of free transformers.

    The transform_manager agent performs these actions through periodic messages. It looks as follows.

    First, the types of signals that will be used as periodic messages are determined:

    struct clear_cache_t final : public so_5::signal_t {};
    struct check_pending_requests_t final : public so_5::signal_t {};

    Then the agent is subscribed, including these signals:

    void
    a_transform_manager_t::so_define_agent()
    {
       so_subscribe_self()
             .event( &a_transform_manager_t::on_resize_request )
             .event( &a_transform_manager_t::on_resize_result )
             .event( &a_transform_manager_t::on_clear_cache )
             .event( &a_transform_manager_t::on_check_pending_requests );
    }
    void
    a_transform_manager_t::on_clear_cache(
       mhood_t ) {...}
    void
    a_transform_manager_t::on_check_pending_requests(
       mhood_t ) {...}

    Thanks to the subscription, SObjectizer will call the desired handler when the agent receives the corresponding signal.

    And it remains only to run periodic messages when the agent starts:

    void
    a_transform_manager_t::so_evt_start()
    {
       m_clear_cache_timer = so_5::send_periodic(
             *this,
             clear_cache_period,
             clear_cache_period );
       m_check_pending_timer = so_5::send_periodic(
             *this,
             check_pending_period,
             check_pending_period );
    }

    The key point here is to save timer_id, which are returned by send_periodic () functions. After all, a periodic signal will only come as long as its timer_id is alive. Therefore, if the return value of send_periodic () is not saved, sending a periodic message will be immediately canceled. Therefore, the a_transform_manager_t class has the following attributes:

    so_5::timer_id_t m_clear_cache_timer;
    so_5::timer_id_t m_check_pending_timer;

    End of the first part


    Today we introduced the reader to the simplest and most minimalistic implementation of shrimp. This implementation is enough to show how RESTinio and SObjectizer can be used together for something more or less like a real task, rather than a simple HelloWorld. But it has a number of serious flaws.

    For example, in the transform_manager agent there is a certain check of uniqueness of request. But it only works if the transformed image is already in the cache. If there is no image in the cache yet and at the same time two identical requests come for the same picture, then both of these requests will be sent for processing. What is not good. It would be correct to process only one of them, and postpone the second until the processing of the first is completed.

    Such more advanced control over the uniqueness of requests would lead to a much more complex and voluminous transform_manager code. Therefore, we did not begin to implement it immediately, but decided to go the evolutionary path - from simple to complex.

    Also, the simplest version of shrimp is a “black box” that does not show any signs of its work. Which is not very convenient both during testing and during operation. Therefore, in a good way, shrimp should also add logging.

    We will try to eliminate these and some other shortcomings of the very first version of shrimp in future versions and describe them in future articles. So stay tuned.

    If someone has questions about the logic of shrimp, RESTinio or SObjectizer, we will be happy to answer in the comments. In addition, shrimp itself is a demo project, but if someone is interested in its functionality and would like to see in shrimp something else besides the resize operation, let us know, we will be happy to listen to any constructive ideas .

    To be continued ...

    Also popular now: