Microservices - MIF in C ++
About three years ago, I had the idea of creating a small framework for developing small services that could somehow interact with each other, provide an API outside, work with databases and something else. During the solution of some work tasks, the idea of his project, which was close to solving work tasks, was finally formed. About a year ago, all this formed into a MIF (MetaInfo Framework) project . It was assumed that with its help it would be possible to solve such problems as:- Development of lightweight HTTP services
- Microservice communication via interfaces transferred between processes
- Serialization and deserialization based on the reflection of data structures in different formats
- Work with databases
- Some helper components for creating service wireframes
All this is focused on the development of backend services for the web, but can also be used on other systems.
Introduction
The release of the new C ++ standard, preparation of the following. Years pass, but there is no reflection in C ++. The possibility of the appearance of reflection in C ++ was discussed, and it seemed that it would be included in the next standard. No ... Maybe reflection is not needed? Maybe ... But there are tasks where it could be useful: interprocess communication (IPC, RPC, REST API, microservices), working with databases (ORM), various (de) serializers, etc. - In general, there is room for application.
The compiler has all the type information, and why not share it with the developers? I’ll try to assume that there is just no document on which compiler developers should release this information to honest developers of other software. Such a document is the C ++ standard, in which the “merchandising” regulation does not appear in any way.
Many languages have reflection. C ++ is considered a language for developing programs where productivity is important and, it would seem, one of its areas of application is web development. Development of backend services. In this industry, there is a REST API, and ORM, and all kinds of serialization into anything (often in json). To solve these problems, you have to use ready-made libraries or write your own, in which C ++ data is connected manually with other entities, for example, mapping structures in C ++ to json format. Sometimes with the help of macros meta-information about the type is added, which is used later in the construction of the REST API, ORM, etc. There are even solutions with plugins for the compiler, for example, odb .
I would like to have something more natural, without code generation by external utilities, piling up macros and templates or manual “mapping”. This is not yet available, and to solve these problems, you need to choose one of the above approaches.
The proposed solution is based on the addition of meta-information to C ++ types with its subsequent use in solving various problems. We can say that “the type needs to be wrapped in meta-information about him” after which he will be able to boldly step across the boundaries of processes and data centers and be able to present himself in different ways (binary, json, xml, text, etc) with minimal developer intervention.
Often, such tasks are solved using code generation, for example, thrift , gSOAP , protobufetc. I wanted to get my own solution, which excludes external code generation, and everything necessary will be added to the type at the time of compilation, and at the same time I want to preserve the natural syntax of the C ++ language as much as possible without creating a new language in the existing language.
MIF in examples
I would like to show some features of the MIF project. And on the feedback to the post, I’ll probably prepare a post about implementation with all its tediousness, subtleties and explanations of why this or that solution was chosen.
The tasks listed at the beginning of the post suggest that there will be some opportunity to display C ++ data types for serialization, object-oriented interprocess communication, REST API implementation, etc. With this I suggest and start ...
Reflection
Reflection is the basis of the whole project, which allows me to solve my applied problems.
An example of how you can get the name of the C ++ data structure, the number of its fields and refer to one of the fields by its number.
struct Data
{
int field1 = 0;
std::string field2;
};
The solution to the problem could look like this:
int main()
{
Data data;
data.field1 = 100500;
data.field2 = "Text";
using Meta = Mif::Reflection::Reflect;
std::cout << "Struct name: " << Meta::Name::GetString() << std::endl;
std::cout << "Field count: " << Meta::Fields::Count << std::endl;
std::cout << "Field1 value: " << data.field1 << std::endl;
std::cout << "Field2 value: " << data.field2 << std::endl;
std::cout << "Modify fields." << std::endl;
data.*Meta::Fields::Field<0>::Access() = 500100;
data.*Meta::Fields::Field<1>::Access() = "New Text.";
std::cout << "Field1 value: " << data.field1 << std::endl;
std::cout << "Field2 value: " << data.field2 << std::endl;
return 0;
}
It would have been like this, it’s written a little differently, but the standard does not make it possible to display C ++ data types so simply. And for the above code to work, you need to add meta-information to the Data structure
MIF_REFLECT_BEGIN(Data)
MIF_REFLECT_FIELD(field1)
MIF_REFLECT_FIELD(field2)
MIF_REFLECT_END()
MIF_REGISTER_REFLECTED_TYPE(Data)
Type meta-information could be added to the type itself, expanding it. I would like to refuse such a decision in order to be able to display types whose intervention is not possible (types of third-party libraries). Several macros allow you to add all the information you need for further work. There is also the possibility of inheritance, but more on that later ...
The complete sample code
// STD
#include
#include
// MIF
#include
#include
struct Data
{
int field1 = 0;
std::string field2;
};
MIF_REFLECT_BEGIN(Data)
MIF_REFLECT_FIELD(field1)
MIF_REFLECT_FIELD(field2)
MIF_REFLECT_END()
MIF_REGISTER_REFLECTED_TYPE(Data)
int main()
{
Data data;
data.field1 = 100500;
data.field2 = "Text";
using Meta = Mif::Reflection::Reflect;
std::cout << "Struct name: " << Meta::Name::GetString() << std::endl;
std::cout << "Field count: " << Meta::Fields::Count << std::endl;
std::cout << "Field1 value: " << data.field1 << std::endl;
std::cout << "Field2 value: " << data.field2 << std::endl;
std::cout << "Modify fields." << std::endl;
data.*Meta::Fields::Field<0>::Access() = 500100;
data.*Meta::Fields::Field<1>::Access() = "New Text.";
std::cout << "Field1 value: " << data.field1 << std::endl;
std::cout << "Field2 value: " << data.field2 << std::endl;
return 0;
}
A slightly complicated example: try to write a generalized output code to the console of all the fields of the structure and its basic structures.
Structure walk example
// STD
#include
#include Result
Struct name: Base1
field1 = 1
field2 = true
Struct name: Base2
field3 = Text
Struct name: Data
field4 = 100
field5 = String
field6 = key1
Struct name: Nested
field = 100
key2
Struct name: Nested
field = 200
An example is a prototype for a full-fledged serializer, because based on the added meta-information, serializes the structure to a stream (in the example, to a standard output stream). To determine whether a type is a container, a function from the Serialization namespace is used. This namespace contains ready-made serializers in json and boost.archives (xml, text, binary). They are built on a principle close to that given in the example. If there is no need to extend the framework with your serializer, then writing such code is not necessary.
Instead of using the Printer class, you can use a ready-made serializer, for example, in json, and the amount of code will be greatly reduced.
#include
#include
// Data and meta
int main()
{
Data data;
// Fill data
auto const buffer = Mif::Serialization::Json::Serialize(data); // Сериализация в json
std::cout << buffer.data() << std::endl;
return 0;
}
Work result
{
"Base1" :
{
"field1" : 1,
"field2" : true
},
"Base2" :
{
"field3" : "Text"
},
"field4" : 100,
"field5" : "String",
"field6" :
[
{
"id" : "key1",
"val" :
{
"field" : 100
}
},
{
"id" : "key2",
"val" :
{
"field" : 200
}
}
]
}
For a change, you can try using the serialization of boost.archives in xml format.
Serialization in xml with boost.archives
// BOOST
#include
// MIF
#include
#include
// Data and meta
int main()
{
Data data;
// Fill data
boost::archive::xml_oarchive archive{std::cout};
archive << boost::serialization::make_nvp("data", data);
return 0;
}
Work result
1 1 Text 100 String 2 0 key1 100 key2 200 As you can see from the example, apart from calling a specific serializer, nothing changes. The same meta information is used. It will also be used in other places when implementing interprocess communication.
Serializations and deserializers of several formats are implemented in the framework. And if necessary, you can add support for the new format by using the example of the Printer class or by taking a (de) serializer of the json format using the proposed API to bypass C ++ data structures. There are some restrictions on types (where without them), but more on that later.
Next, I propose moving on to more interesting things - implementing interprocess communication based on interfaces transferred between processes (C ++ data structures with purely virtual methods).
Interprocess communication
What the MIF project started with is the implementation of interprocess communication. It was a priority need. At one of the first implementations of this mechanism, one of the services was developed, which at the time of writing these lines of the post had worked stably for more than six months without crashes and reboots.
There was a need to make communication of services located on several machines using interfaces. I wanted to work with services as if they were all in one process.
An example shows how close it was to get closer to the desired result.
Objective: to develop a server and client for exchanging information about the employees of a fictitious company.
The solution of such problems is reduced to several similar steps
- Define component interface (s)
- Define custom data structures (if necessary) for method parameters or as return values
- Add meta information
- Implement a server application
- Implement a client application
Complex_type example
a common part
Data structures
// data.h
namespace Service
{
namespace Data
{
using ID = std::string;
struct Human
{
std::string name;
std::string lastName;
std::uint32_t age = 0;
};
enum class Position
{
Unknown,
Developer,
Manager
};
struct Employee
: public Human
{
Position position = Position::Unknown;
};
using Employees = std::map;
} // namespace Data
} // namespace Service
Meta information
// meta/data.h
namespace Service
{
namespace Data
{
namespace Meta
{
using namespace ::Service::Data;
MIF_REFLECT_BEGIN(Human)
MIF_REFLECT_FIELD(name)
MIF_REFLECT_FIELD(lastName)
MIF_REFLECT_FIELD(age)
MIF_REFLECT_END()
MIF_REFLECT_BEGIN(Position)
MIF_REFLECT_FIELD(Unknown)
MIF_REFLECT_FIELD(Developer)
MIF_REFLECT_FIELD(Manager)
MIF_REFLECT_END()
MIF_REFLECT_BEGIN(Employee, Human)
MIF_REFLECT_FIELD(position)
MIF_REFLECT_END()
} // namespace Meta
} // namespace Data
} // namespace Service
MIF_REGISTER_REFLECTED_TYPE(::Service::Data::Meta::Human)
MIF_REGISTER_REFLECTED_TYPE(::Service::Data::Meta::Position)
MIF_REGISTER_REFLECTED_TYPE(::Service::Data::Meta::Employee)
Interface
// imy_company.h
namespace Service
{
struct IMyCompany
: public Mif::Service::Inherit
{
virtual Data::ID AddEmployee(Data::Employee const &employee) = 0;
virtual void RemoveAccount(Data::ID const &id) = 0;
virtual Data::Employees GetEmployees() const = 0;
};
} // namespace Service
Meta information
// ps/imy_company.h
namespace Service
{
namespace Meta
{
using namespace ::Service;
MIF_REMOTE_PS_BEGIN(IMyCompany)
MIF_REMOTE_METHOD(AddEmployee)
MIF_REMOTE_METHOD(RemoveAccount)
MIF_REMOTE_METHOD(GetEmployees)
MIF_REMOTE_PS_END()
} // namespace Meta
} // namespace Service
MIF_REMOTE_REGISTER_PS(Service::Meta::IMyCompany)
The definition of the data structure and the addition of meta-information to it is the same as in the examples with reflection, except that everything is separated by namespaces.
An interface definition is a C ++ definition of a data structure that contains only purely virtual methods.
For interfaces, there was another wish - the ability to request from one interface others contained in the implementation, and possibly not connected into a single hierarchy. Therefore, the defined interface should always inherit Mif :: Service :: IService or any other that inherits from Mif :: Service :: IService. There is multiple inheritance. Inheritance is done through an intermediate entity Mif :: Service :: Inherit. This is a template with a variable number of parameters. Its parameters are inheritance interfaces or implementations ( inheritance) This is necessary to implement a mechanism for requesting interfaces that is the same as dynamic_cast, but also working outside the process.
Adding meta information to an interface is no different than adding meta information to data structures. This is a different, but similar, set of macros. Perhaps later, however, everything will be reduced to a single set of macros for defining data structures and interfaces. While they are different. This happened during the development of the project.
It is not necessary to indicate its basic interfaces when adding meta-information to an interface. The entire hierarchy will be found at compile time. Here, a small auxiliary entity Mif :: Service :: Inherit plays its main role in the search for heirs and related meta-information.
When adding meta-information to interfaces, only the interface and its methods are indicated without specifying parameters, return values, and cv-qualifiers. There was a desire to add meta-information to interfaces in the spirit of minimalism. The lack of overload became the price of minimalism. I consider this a low price for the opportunity not to list all the parameters and return value types for each method and not to edit them with small changes in the interface.
Having defined common entities, it remains to implement server and client applications.
Each interface can have many implementations. They need to be distinguished somehow. When creating an object, you must explicitly specify the desired implementation. To do this, you need the identifiers of the interface implementations and their connection with the implementations.
For convenience and rejection of “magic values in the code”, implementation identifiers are better placed in one or more header files. In an MIF project, numbers are used as an identifier. In order to somehow make them unique while not counting some counters or putting everything into one single enum and to be able to logically separate identifiers into different files and namespaces, it is suggested to use crc32 from the string as identifiers, with the idea of uniqueness which the developer should have less.
To implement the IMyCompany interface, you need an identifier
// id/service.h
namespace Service
{
namespace Id
{
enum
{
MyCompany = Mif::Common::Crc32("MyCompany")
};
} // namespace Id
} // namespace Service
Server application
IMyCompany Implementation
// service.cpp
// MIF
#include
#include
#include
// COMMON
#include "common/id/service.h"
#include "common/interface/imy_company.h"
#include "common/meta/data.h"
namespace Service
{
namespace Detail
{
namespace
{
class MyCompany
: public Mif::Service::Inherit
{
public:
// …
private:
// …
// IMyCompany
virtual Data::ID AddEmployee(Data::Employee const &employee) override final
{
// ...
}
virtual void RemoveAccount(Data::ID const &id) override final
{
// ... }
}
virtual Data::Employees GetEmployees() const override final
{
// ...
}
};
} // namespace
} // namespace Detail
} // namespace Service
MIF_SERVICE_CREATOR
(
::Service::Id::MyCompany,
::Service::Detail::MyCompany
)
There are several points that I would like to draw attention to:
- Inheritance in the implementation is also through Mif :: Service :: Inherit. This is not necessary, but it can be considered a good form and will be useful when implementing several interfaces with the inheritance of some previously implemented interfaces.
- The entire implementation can and should preferably be done in one cpp file, without splitting into h and cpp files. This allows you to strengthen encapsulation and in large projects to reduce compilation time due to the fact that all the necessary include implementation files are in the implementation file. When they are modified, fewer dependent cpp files are recompiled.
- Each implementation has an entry point - MIF_SERVICE_CREATOR, which is an implementation factory. The parameters are the implementation class, identifier, and, if necessary, a variable number of parameters passed to the implementation constructor.
To complete the server application, it remains to add an entry point - the main function.
// MIF
#include
// COMMON
#include "common/id/service.h"
#include "common/ps/imy_company.h"
class Application
: public Mif::Application::TcpService
{
public:
using TcpService::TcpService;
private:
// Mif.Application.Application
virtual void Init(Mif::Service::FactoryPtr factory) override final
{
factory->AddClass<::Service::Id::MyCompany>();
}
};
int main(int argc, char const **argv)
{
return Mif::Application::Run(argc, argv);
}
When creating an entry point, you need to implement your application class — an inheritor from the base application class or from one of the predefined application templates. In the redefined Init method, you need to add to the factory all existing implementations of the interfaces that the service will export (factory-> AddClass). You can pass implementation constructor parameters to the AddClass method.
The service uses the predefined tcp transport, serialization based on boost.archive in binary format with gzip compression for exchanging information about interfaces, methods, parameters, returned results, exceptions and object instances.
You can use another type of transport (for example, http, which is also available in MIF or implement your own), serialize and collect your own unique data processing chain (determining packet boundaries, compression, encryption, multi-threaded processing, etc.). To do this, you need to use not the application template, but the base class of applications (Mif :: Application :: Application), to determine independently the necessary parts of the data processing chain or transport.
There were no predefined application templates in the first version of the MIF project. The examples didn’t look so short, but they showed the whole path that needs to be done for complete control over the flow of data processing. The whole chain is shown in the examples of the first version of the project ( MIF 1.0 ).
Client application
On the client side, everything that was defined in the common part is used.
A client is the same application framework (in the example, a predefined application template is used), in which a remote factory of classes / services is requested, through which the desired object is created and its methods are called.
// MIF
#include
#include
// COMMON
#include "common/id/service.h"
#include "common/ps/imy_company.h"
class Application
: public Mif::Application::TcpServiceClient
{
public:
using TcpServiceClient::TcpServiceClient;
private:
void ShowEmployees(Service::Data::Employees const &employees) const
{
// ...
}
// Mif.Application.TcpServiceClient
virtual void Init(Mif::Service::IFactoryPtr factory) override final
{
auto service = factory->Create(Service::Id::MyCompany);
{
Service::Data::Employee e;
e.name = "Ivan";
e.lastName = "Ivanov";
e.age = 25;
e.position = Service::Data::Position::Manager;
auto const eId = service->AddEmployee(e);
MIF_LOG(Info) << "Employee Id: " << eId;
}
{
Service::Data::Employee e;
e.name = "Petr";
e.lastName = "Petrov";
e.age = 30;
e.position = Service::Data::Position::Developer;
auto const eId = service->AddEmployee(e);
MIF_LOG(Info) << "Employee Id: " << eId;
}
auto const &employees = service->GetEmployees();
ShowEmployees(employees);
if (!employees.empty())
{
auto id = std::begin(employees)->first;
service->RemoveAccount(id);
MIF_LOG(Info) << "Removed account " << id;
auto const &employees = service->GetEmployees();
ShowEmployees(employees);
try
{
MIF_LOG(Info) << "Removed again account " << id;
service->RemoveAccount(id);
}
catch (std::exception const &e)
{
MIF_LOG(Warning) << "Error: " << e.what();
}
}
}
};
int main(int argc, char const **argv)
{
return Mif::Application::Run(argc, argv);
}
Result
The result of the server application
2017-08-09T14:01:23.404663 [INFO]: Starting network application on 0.0.0.0:55555
2017-08-09T14:01:23.404713 [INFO]: Starting server on 0.0.0.0:55555
2017-08-09T14:01:23.405442 [INFO]: Server is successfully started.
2017-08-09T14:01:23.405463 [INFO]: Network application is successfully started.
Press 'Enter' for quit.
2017-08-09T14:01:29.032171 [INFO]: MyCompany
2017-08-09T14:01:29.041704 [INFO]: AddEmployee. Name: Ivan LastName: Ivanov Age: 25 Position: Manager
2017-08-09T14:01:29.042948 [INFO]: AddEmployee. Name: Petr LastName: Petrov Age: 30 Position: Developer
2017-08-09T14:01:29.043616 [INFO]: GetEmployees.
2017-08-09T14:01:29.043640 [INFO]: Id: 0 Name: Ivan LastName: Ivanov Age: 25 Position: Manager
2017-08-09T14:01:29.043656 [INFO]: Id: 1 Name: Petr LastName: Petrov Age: 30 Position: Developer
2017-08-09T14:01:29.044481 [INFO]: Removed employee account for Id: 0 Name: Ivan LastName: Ivanov Age: 25 Position: Manager
2017-08-09T14:01:29.045121 [INFO]: GetEmployees.
2017-08-09T14:01:29.045147 [INFO]: Id: 1 Name: Petr LastName: Petrov Age: 30 Position: Developer
2017-08-09T14:01:29.045845 [WARNING]: RemoveAccount. Employee with id 0 not found.
2017-08-09T14:01:29.046652 [INFO]: ~MyCompany
2017-08-09T14:02:05.766072 [INFO]: Stopping network application ...
2017-08-09T14:02:05.766169 [INFO]: Stopping server ...
2017-08-09T14:02:05.767180 [INFO]: Server is successfully stopped.
2017-08-09T14:02:05.767238 [INFO]: Network application is successfully stopped.
The result of the client application
2017-08-09T14:01:29.028821 [INFO]: Starting network application on 0.0.0.0:55555
2017-08-09T14:01:29.028885 [INFO]: Starting client on 0.0.0.0:55555
2017-08-09T14:01:29.042510 [INFO]: Employee Id: 0
2017-08-09T14:01:29.043296 [INFO]: Employee Id: 1
2017-08-09T14:01:29.044082 [INFO]: Employee. Id: 0 Name: Ivan LastName: Ivanov Age: 25 Position: Manager
2017-08-09T14:01:29.044111 [INFO]: Employee. Id: 1 Name: Petr LastName: Petrov Age: 30 Position: Developer
2017-08-09T14:01:29.044818 [INFO]: Removed account 0
2017-08-09T14:01:29.045517 [INFO]: Employee. Id: 1 Name: Petr LastName: Petrov Age: 30 Position: Developer
2017-08-09T14:01:29.045544 [INFO]: Removed again account 0
2017-08-09T14:01:29.046357 [WARNING]: Error: [Mif::Remote::Proxy::RemoteCall] Failed to call remote method "IMyCompany::RemoveAccount" for instance with id "411bdde0-f186-402e-a170-4f899311a33d". Error: RemoveAccount. Employee with id 0 not found.
2017-08-09T14:01:29.046949 [INFO]: Client is successfully started.
2017-08-09T14:01:29.047311 [INFO]: Network application is successfully started.
Press 'Enter' for quit.
2017-08-09T14:02:02.901773 [INFO]: Stopping network application ...
2017-08-09T14:02:02.901864 [INFO]: Stopping client ...
2017-08-09T14:02:02.901913 [INFO]: Client is successfully stopped.
2017-08-09T14:02:02.901959 [INFO]: Network application is successfully stopped.
Yes, exceptions also transcend process boundaries ...
[WARNING]: Error: [Mif::Remote::Proxy::RemoteCall] Failed to call remote method "IMyCompany::RemoveAccount" for instance with id "411bdde0-f186-402e-a170-4f899311a33d". Error: RemoveAccount. Employee with id 0 not found.
It can be seen from the message that the method of deleting information about the employee with identifier 0 was called again. There is no such record on the server side, the server reported an exception with the text “Employee with id 0 not found” / The
example demonstrated the interprocess communication of the client with the server, as much as possible hiding all the details associated with the transport and the format of the transmitted data.
This example completes the display of the base underlying the MIF project. Additional features include
- The ability to request interfaces from an implementation not integrated into a single hierarchy (not counting the inheritance from Mif :: Service :: IService).
- Pass pointers and smart pointers to interfaces between services. What can be useful for different implementations of services with callbacks. For example, a service based on publish / subscribe. An example is the implementation of the visitor pattern , parts of which are in different processes and interact via tcp.
HTTP
In the example of interprocess communication, you can easily replace an existing TCP transport with an HTTP transport. You will be able to access the services using familiar means, for example, curl or from a browser.
HTTP support makes it possible to build services that can simultaneously be a web server with some json REST API, and at the same time support previously demonstrated interaction via C ++ interfaces.
Comfort is often associated with restrictions. This limitation when using HTTP transport is the inability to call back methods passed to interfaces. The solution based on HTTP transport is not focused on building publish / subscribe applications. This is because working over HTTP involves a request-response approach. The client sends requests to the server and waits for a response.
The TCP transport proposed in MIF has no such restriction. Both the client and the server can call methods. In this case, the distinction between client and server is blurred. A channel appears through which objects can communicate with each other, calling methods from either side. This allows two-way interaction to build publish / subscribe architecture, which was demonstrated in the example by interprocessa visitor .
MIF HTTP pays special attention, as Initially, the focus was on developing backend services for the web. It was necessary to create small HTTP web services, receive data from providers via HTTP, and only after that the ability to use HTTP as a transport when marshaling interfaces between processes was added. Therefore, I would like to show at the beginning examples of creating simple web services, clients, and at the end give an example of a web server with support for transferring interfaces between processes.
Simple HTTP web server
// MIF
#include
#include
#include
class Application
: public Mif::Application::HttpServer
{
public:
using HttpServer::HttpServer;
private:
// Mif.Application.HttpServer
virtual void Init(Mif::Net::Http::ServerHandlers &handlers) override final
{
handlers["/"] = [] (Mif::Net::Http::IInputPack const &request,
Mif::Net::Http::IOutputPack &response)
{
auto data = request.GetData();
MIF_LOG(Info) << "Process request \"" << request.GetPath()
<< request.GetQuery() << "\"\t Data: "
<< (data.empty() ? std::string{"null"} :
std::string{std::begin(data), std::end(data)});
response.SetCode(Mif::Net::Http::Code::Ok);
response.SetHeader(
Mif::Net::Http::Constants::Header::Connection::GetString(),
Mif::Net::Http::Constants::Value::Connection::Close::GetString());
response.SetData(std::move(data));
};
}
};
int main(int argc, char const **argv)
{
return Mif::Application::Run(argc, argv);
}
To check the operation, you can use the command
curl -iv -X POST "http://localhost:55555/" -d 'Test data'
Only about three dozen lines of code and a multithreaded HTTP echo server based on the MIF application framework are ready. As backend libevent is used. Simple testing with the ab utility produces an average result of up to 160K queries per second. Of course, everything depends on the hardware on which testing is carried out, the operating system, the network, etc. But for comparison, something similar was done in Python and Go. The Python server worked 2 times slower, and on Go the result was better on average by 10% of the server result from the example. So, if you are a C ++ developer and Python and Go are alien to you or there are other reasons that may be officially fixed by internal instructions within the framework of the project, you can only love C ++, then you can use the proposed solution and get good results in terms of speed and development time ...
HTTP client
The client part is represented by the class Mif :: Net :: Http :: Connection . This is the foundation of HTTP transport marshaling C ++ interfaces in MIF. But so far not about marshaling ... The class can be used separately from the entire infrastructure of the interprocess interaction of MIF microservices, for example, for downloading suppliers' data, for example, media content, weather forecasts, stock quotes, etc.
By running the above echo service, you can access it with this client:
// STD
#include
#include
#include
#include
// MIF
#include
#include
int main()
{
try
{
std::string const host = "localhost";
std::string const port = "55555";
std::string const resource = "/";
std::promise promise;
auto future = promise.get_future();
Mif::Net::Http::Connection connection{host, port,
[&promise] (Mif::Net::Http::IInputPack const &pack)
{
if (pack.GetCode() == Mif::Net::Http::Code::Ok)
{
auto const data = pack.GetData();
promise.set_value({std::begin(data), std::end(data)});
}
else
{
promise.set_exception(std::make_exception_ptr(
std::runtime_error{
"Failed to get response from server. Error: "
+ pack.GetReason()
}));
}
}
};
auto request = connection.CreateRequest();
request->SetHeader(Mif::Net::Http::Constants::Header::Connection::GetString(),
Mif::Net::Http::Constants::Value::Connection::Close::GetString());
std::string data = "Test data!";
request->SetData({std::begin(data), std::end(data)});
connection.MakeRequest(Mif::Net::Http::Method::Type::Post,
resource, std::move(request));
std::cout << "Response from server: " << future.get() << std::endl;
}
catch (std::exception const &e)
{
std::cerr << "Error: " << e.what() << std::endl;
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
This is a basic example without focusing on error handling. It demonstrates the ability to develop a simple client. Of course, it is better to use something else, for example, based on the curl library, but sometimes in simple cases the above code may be enough.
Dual-interface HTTP web server
The example summarizes the HTTP support in the MIF project. The example shows an HTTP web server that can be accessed from a browser or using curl, as well as a client working through C ++ interfaces and not wanting to know anything about transport and data format. It uses the same Mif :: Application :: HttpServer application framework as before. The following code snippets are required for demonstration. The whole example is available on github in the http example .
a common part
Interface
namespace Service
{
struct IAdmin
: public Mif::Service::Inherit
{
virtual void SetTitle(std::string const &title) = 0;
virtual void SetBody(std::string const &body) = 0;
virtual std::string GetPage() const = 0;
};
} // namespace Service
Adding meta-information to the interface, the identifier of the service-implementation are all similar to the examples of interprocess communication given earlier.
Server side
Application wireframe
class Application
: public Mif::Application::HttpServer
{
//...
private:
// Mif.Application.HttpService
virtual void Init(Mif::Net::Http::ServerHandlers &handlers) override final
{
std::string const adminLocation = "/admin";
std::string const viewLocation = "/view";
auto service = Mif::Service::Create(viewLocation);
auto webService = Mif::Service::Cast(service);
auto factory = Mif::Service::Make();
factory->AddInstance(Service::Id::Service, service);
std::chrono::microseconds const timeout{10000000};
auto clientFactory = Service::Ipc::MakeClientFactory(timeout, factory);
handlers.emplace(adminLocation, Mif::Net::Http::MakeServlet(clientFactory));
handlers.emplace(viewLocation, Mif::Net::Http::MakeWebService(webService));
}
};
The resource handler is no longer set directly, but through an additional wrapper, which allows you to:
- Add different handlers to the resource
- Автоматически делать разбор параметров запроса и данных из тела запроса
- Автоматически сериализовать ответ в выбранный формат
- Объединять два подхода: работу по HTTP через, возможно, REST API и работу с клиентом с поддержкой маршалинга C++ интерфейсов
All these features are implemented in the base class Mif :: Net :: Http :: WebService. A custom class must inherit it. Because Since this is a service that contains purely virtual methods, the implementation of which is hidden in the service code of the framework, then instances of the derived classes should be created just like all services - by the factory method. In order for the service to be used as a web server HTTP handler, you need to create a wrapper with the Mif :: Net :: Http :: MakeServlet function.
MIF services and web services are different services. MIF services should be understood only as implementation classes of interfaces that are not required to deal only with processing HTTP requests or other network requests. Web services are already a more familiar concept. But if necessary, in MIF, all this is easily combined into a single whole and the given example of this confirmation.
Service handler
namespace Service
{
namespace Detail
{
namespace
{
class WebService
: public Mif::Service::Inherit
<
IAdmin,
Mif::Net::Http::WebService
>
{
public:
WebService(std::string const &pathPrefix)
{
AddHandler(pathPrefix + "/stat", this, &WebService::Stat);
AddHandler(pathPrefix + "/main-page", this, &WebService::MainPage);
}
private:
// …
// IAdmin
virtual void SetTitle(std::string const &title) override final
{
// ...
}
// …
// Web hadlers
Result Stat()
{
// ...
std::map resp;
// Fill resp
return resp;
}
Result
MainPage(Prm const &format)
{
// ...
}
};
} // namespace
} // namespace Detail
} // namespace Service
MIF_SERVICE_CREATOR
(
::Service::Id::Service,
::Service::Detail::WebService,
std::string
)
I would like to draw attention to several points:
- The need to use Mif :: Service :: Inherit when writing implementations has already been mentioned above. Here the role of Mif :: Service :: Inherit is fully justified. On the one hand, the class inherits the IAdmin interface and implements all its methods, and on the other, it inherits the implementation of the IWebService interface as the base class Mif :: Net :: Http :: WebService, which makes working with HTTP easier.
- Resource handlers are added to the constructor. The full path is built from the path of the resource with which the Mif :: Net :: Http :: WebService descendant class is associated with the path that is specified when AddHandler is called. In the example, the path to get statistics on completed requests will look like / view / stat
- Возвращаемый тип обработчика может быть любым, который можно вывести в поток. Кроме того в качестве возвращаемого типа можно использовать обертку-сериализатор. Она сериализует переданный встроенный или пользовательский тип в свой формат. В этом случае для сериализации пользовательских типов используется добавленная к ним метаинформация.
- Методы-обработчики могут принимать параметры. Для описания параметров используется сущность Prm. Класс Prm принимает тип параметра, его имя. При необходимости пользовательская реализация разбора параметров может быть передана в Prm. Поддерживается конвертация параметров запроса к интегральным типам, типам с плавающей точкой, множества параметров с разделителем в виде точки с запятой (некоторые stl контейнеры, например, list, vector, set). Так же дата, время и timestamp целиком, которые можно привести к типам boost::posix_time. Десериализация данных тела запроса производится классом Content. Класс использует метаинформацию и преданный десериализатор. При необходимости получить доступ к заголовкам запроса или ко всей карте параметров, в качестве параметра обработчика можно указать Headers и / или Params. Одновременно можно использовать все указанные классы. Подробная работа с параметрами показана в примере http_crud
- When defining a factory method for creating an implementation class, an additional parameter is passed to the MIF_SERVICE_CREATOR macro, which must be passed to the implementation constructor. The use of the macro MIF_SERVICE_CREATOR was mentioned above.
Client The
client does not have any special differences from the previously given examples. An exception is the lack of a predefined query processing chain. It is formed by the developer independently. There are no predefined request processing chains for working with HTTP in MIF. Implementation features ... About the reasons, perhaps something will be written in subsequent posts. The full client code is an example http .
Results
For testing, the assembled server must be started and try to access it, for example, using the curl command or from the browser
curl "http://localhost:55555/view/main-page?format=text"
curl "http://localhost:55555/view/main-page?format=html"
curl "http://localhost:55555/view/main-page?format=json"
Then launch a client application that accesses through the admin interface and changes data and try to run the curl commands again. You may notice that the changes made by the client have been successfully applied and the result returned via HTTP has changed.
Work with databases
Backedn without a DB? How without them? Not necessarily, but not rarely ...
Currently, working with databases in MIF is implemented without reliance on meta-information. So far, almost a classic of working with the database. This part was needed to develop services that had to store something in the database. But the development of this part was carried out with the aim that it will become the basis for ORM, where all meta-information added to types will already be fully used. With ORM somehow has not worked out yet. There have been several attempts to implement. It turned out either very cumbersome or not flexible. I think that a compromise between conciseness and flexibility will be reached soon and will appear in the next versions of ORM, because I really want to be able to transfer some checks to the compiler, and not catch errors at the time of program execution that are associated with simple errors and typos in raw SQL query strings.
While there is no ORM, a bit of classics ...
Working with the database contains several steps:
- Creating a database connection object
- Fulfillment of requests through received connection
- Results Processing
- Transactions
Whoever worked with any wrappers such as JDBC (in Java) or something similar for C ++ will not discover anything new for themselves here.
The db_client example shows working with two DBMSs: PostgreSQL and SQLite.
HTTP CRUD server
The http_crud example is the logical conclusion to the demonstration of working with HTTP and the database - a classic of simple microservices for the web that do not interact directly with each other.
The web server class contains handlers for all basic CRUD operations. Work with the database is in handlers. Such code can often be found in various successfully working services. But in spite of its initial scope, it has a drawback - dependence on the DBMS. And if you want to abandon the relational database and go to the NoSQL database, the code of all the handlers will be almost completely rewritten. In many real projects, I try to avoid similar work with the database. Move high-level logic functions into separate facades. Facades are usually C ++ interfaces with specific implementations. And replacing the database comes down to the next implementation of the interface and a request from the factory of implementation with a new identifier. Playing the captain of evidence, we can say that this approach has proven itself more than once.
Conclusion
Everything that was discussed above: reflection, serialization, interprocess communication, interface marshaling, working with the database, HTTP support - everything was reflected in the final example of building a small system on microservices . The example demonstrates a small service consisting of two microservices interacting with each other. One service is the facade to the database, and the second provides the Json API for external clients. The example is a modified version of the considered example http_crud and is the logical conclusion (union) of everything that was discussed in the framework of the post.
Some parts of MIF have not been considered, for example, db_client, but working with the database is shown in other examples. Work with services (interface implementations) is also partially affected in the examples. Some parts of MIF, such as serialization, database handling, and HTTP support, have already been tested several times. Interprocess communication has so far been given less time for verification and debugging, despite the fact that some parts in the development have been given a lot of time. For example, these include support for passing interfaces as parameters to another process and calling their methods back. What is essentially “ego-features”. I wanted to try to implement a similar mechanism, shown in the visitor example , but in the future there is a desire to bring this part to a complete, proven and debugged solution.
In the future, perhaps, if there is interest in the material presented, the post will continue with what was not included in this, for example, that it was “under the hood” of MIF, why certain decisions were made, what tasks had to be solved, what thrift did not fit or other similar decisions, what shortcomings I see and what I would like to change and improve, etc. The project appeared as a “home project” and, to my surprise, is a project to which I have been paying attention for almost a year now, there should not be any problems with something else to write about.
To summarize, I’ll duplicate the link to the MIF on C ++ project .
Thanks for attention.