October 29, 2013 at 13:48RMI means C ++ and boost.preprocessor
This is my first publication on this resource, therefore, I ask you to treat my mistakes with understanding.
RMI is a very trivial task for introspection- enabled PLs . But, C ++, unfortunately, does not apply to them.
In this publication, I want to demonstrate the possibility of implementing a very usable RMI using the C ++ preprocessor.
1. Provide the simplest syntax so that it is impossible to make a mistake.
2. The identification (linking) of procedures should be hidden from the user so that it is impossible to make a mistake.
3. The syntax should not impose restrictions on the used C ++ types.
4. There should be the possibility of versioned procedures, but so that compatibility with existing clients does not break.
Regarding the fourth point:
For example, we already have two add / div procedures, one version of each. We want add to add a new version. If you just add one more version, the procedure IDs that the client programs collected before the change was made will float.
Because The final result is supposed to be used in conjunction with C ++ code, I see three options:
I will speak about each of the options, respectively:
Regarding the first, second and third paragraphs of the requirements - the preprocessor version is suitable.
So, the choice is made - we use the preprocessor. And yes, of course boost.preprocessor .
C ++ preprocessor data types:
Types, as can be seen, are more than enough.
After a little thought, reading about the possibilities and limitations of each of them, and also taking into account the desired simplicity of the syntax and the inability to make a mistake - the choice was made in favor of sequences and tuples.
A few illustrative examples.
And, finally, such an example:
Apparently, everything is really unrealistically simple.
After some thought, the following syntax emerges:
Well ... very preemptive, however.
Because one of the requirements is the versioning of the procedures, and even such that the compatibility with existing clients does not break - we will need two IDs to identify the procedures. The first is the procedure ID, the second is the version ID.
I will explain with an example.
Let's say this is a description of the API of our service. Suppose we already have client programs using this API.
Now, for
Thus, we have a new ID version of the procedure, while the former remained unchanged.
It was: (0: 0), it became: (0: 0) (0: 1)
i.e. this is exactly what we tried to achieve. Previous clients both used (0: 0) and will continue to use these identifiers without worrying that new versions of these procedures have appeared.
We also agree that all new procedures must be added to the end.
Next, we need to make sure that IDs are automatically affixed on both sides of the service. Easy! - just use the same described sequence twice to generate client and server sides!
It's time to imagine how we want to see all this in the long run:
So that there is no confusion as to who is the leader and who is the slave, we agree that the procedures described on one of the parties are implementations that are on the opposite side. That is, for example, it
(we use double parentheses because the preprocessor, when generating the argument for our MACRO (), expands the API described by us. It can be overcome, but I don’t know if it is necessary? ..)
From the macro call above, the code below the spoiler will be generated.
(My other project, YAS, is used as serialization )
As a bonus, a system procedure was added
As you may have noticed, on the names of the procedures described by us, on the side of their implementation, the
Class prefix is added
If you have the same class performing both roles, you can do this:
The final version looks like this:
The interface to the opposite side will be inherited from the ancestor
The answer we get in our implementation is
Everything!
Of the deficiencies, the following should be noted:
Commands cannot be used in type names describing procedures, because the preprocessor does not understand the context in which they are used. It will be fixed.
Ultimately, all this resulted in a project called YARMI (Yet Another RMI).
The described code generator is encoded in one file - yarmi.hpp . In total, it took one business day to implement the code generator.
An example of the use of this whole thing can be seen here and here . The first test project is still not completed, unfortunately.
In addition to the described, on the project page you will find codes of the asynchronous multi-user single-threaded server, and client codes.
Plans:
1. Generation of several interfaces
2. Describe the specification (although it is nowhere simpler)
3. The ability to use your own locator
I would be grateful for constructive criticism and suggestions.
PS
This code is used in several of our commercial projects, in game dev.
RMI is a very trivial task for introspection- enabled PLs . But, C ++, unfortunately, does not apply to them.
In this publication, I want to demonstrate the possibility of implementing a very usable RMI using the C ++ preprocessor.
Formulation of the problem
1. Provide the simplest syntax so that it is impossible to make a mistake.
2. The identification (linking) of procedures should be hidden from the user so that it is impossible to make a mistake.
3. The syntax should not impose restrictions on the used C ++ types.
4. There should be the possibility of versioned procedures, but so that compatibility with existing clients does not break.
Regarding the fourth point:
For example, we already have two add / div procedures, one version of each. We want add to add a new version. If you just add one more version, the procedure IDs that the client programs collected before the change was made will float.
Tool selection
Because The final result is supposed to be used in conjunction with C ++ code, I see three options:
- We invent the syntax and write our own code generator.
- We use the C ++ preprocessor.
- We are looking for something ready and finish it for ourselves (if necessary).
I will speak about each of the options, respectively:
- Why an additional stage of code generation?
- I love the preprocessor, and often use it.
- Waste of time and effort. And it’s not clear whether that makes sense.
Regarding the first, second and third paragraphs of the requirements - the preprocessor version is suitable.
So, the choice is made - we use the preprocessor. And yes, of course boost.preprocessor .
A little bit about the preprocessor
C ++ preprocessor data types:
- arrays :
(size, (elems, ....)) - lists :
(a, (b, (c, BOOST_PP_NIL))) - sequences :
(a)(b)(c) - tuples :
(a, b, c)
Types, as can be seen, are more than enough.
After a little thought, reading about the possibilities and limitations of each of them, and also taking into account the desired simplicity of the syntax and the inability to make a mistake - the choice was made in favor of sequences and tuples.
A few illustrative examples.
(a)(b)(c)- sequence. Here, we described a sequence consisting of three elements. (a)- also sequence, but consisting of one element. (attention!) (a)(b, c)(d, e, f)- again a sequence, but consisting of three tuples. (pay attention to the first element - a trick, however, but this is a true tuple) (a)(b, c)(d, (e, f))- again a sequence, and also consisting of three tuples. But! The last tuple consists of two elements: 1) any element, 2) tuple. And, finally, such an example:
(a)(b, c)(d, (e, (f)(g)))- here you can figure it out yourself;)Apparently, everything is really unrealistically simple.
Prototype syntax
After some thought, the following syntax emerges:
(proc_name0, // имя процедуры
(signature_arg0, signature_arg1, signature_argN) // первая версия процедуры
(signature_arg0) // вторая версия процедуры
)
(proc_name1, // имя процедуры
(signature_arg0, signature_arg1) // единственная версия этой процедуры
)
(proc_name2, // имя процедуры
() // единственная версия этой процедуры (без аргументов)
)
Well ... very preemptive, however.
Some implementation details
Because one of the requirements is the versioning of the procedures, and even such that the compatibility with existing clients does not break - we will need two IDs to identify the procedures. The first is the procedure ID, the second is the version ID.
I will explain with an example.
Let's say this is a description of the API of our service. Suppose we already have client programs using this API.
(proc_name0, // procID=0
(signature_arg0, signature_arg1) // sigID=0
)
(proc_name1, // procID=1
(signature_arg0, signature_arg1) // sigID=0
)
(proc_name2, // procID=2
() // sigID=0
)
Now, for
proc_name0()us we need to add another version with a different signature.(proc_name0, // procID=0
(signature_arg0, signature_arg1) // sigID=0
(signature_arg0, signature_arg1, signature_arg2) // sigID=1
)
(proc_name1, // procID=1
(signature_arg0, signature_arg1) // sigID=0
)
(proc_name2, // procID=2
() // sigID=0
)
Thus, we have a new ID version of the procedure, while the former remained unchanged.
It was: (0: 0), it became: (0: 0) (0: 1)
i.e. this is exactly what we tried to achieve. Previous clients both used (0: 0) and will continue to use these identifiers without worrying that new versions of these procedures have appeared.
We also agree that all new procedures must be added to the end.
Next, we need to make sure that IDs are automatically affixed on both sides of the service. Easy! - just use the same described sequence twice to generate client and server sides!
It's time to imagine how we want to see all this in the long run:
MACRO(
client_invoker, // name of the client invoker implementation class
((registration, // procedure name
((std::string, std::string)) // message : registration key
))
((activation,
((std::string)) // message
))
((login,
((std::string)) // message
))
((logout,
((std::string)) // message
))
((users_online,
((std::vector)) // without args
))
,
server_invoker, // name of the server invoker implementation class
((registration,
((std::string)) // username
))
((activation,
((std::string, std::string, std::string)) // registration key : username : password
))
((login,
((std::string, std::string)) // username : password
))
((logout,
(()) // without args
))
((users_online,
(()) // without args
((std::string)) // substring
))
)
So that there is no confusion as to who is the leader and who is the slave, we agree that the procedures described on one of the parties are implementations that are on the opposite side. That is, for example, it
client_invoker::registration(std::string, std::string)tells us that the implementation of this procedure will be on the server side, while the interface to this procedure will be on the client side, and vice versa. (we use double parentheses because the preprocessor, when generating the argument for our MACRO (), expands the API described by us. It can be overcome, but I don’t know if it is necessary? ..)
Total
From the macro call above, the code below the spoiler will be generated.
The code
namespace yarmi {
template
struct client_invoker {
client_invoker(Impl &impl, IO &io)
:impl(impl)
,io(io)
{}
void yarmi_error(const std::uint8_t &arg0, const std::uint8_t &arg1, const std::string &arg2) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(0)
& static_cast(0)
& arg0
& arg1
& arg2;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void registration(const std::string &arg0) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(1)
& static_cast(0)
& arg0;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void activation(const std::string &arg0, const std::string &arg1, const std::string &arg2) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(2)
& static_cast(0)
& arg0
& arg1
& arg2;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void login(const std::string &arg0, const std::string &arg1) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(3)
& static_cast(0)
& arg0
& arg1;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void logout() {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(4)
& static_cast(0);
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void users_online() {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(5)
& static_cast(0);
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void users_online(const std::string &arg0) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(5)
& static_cast(1)
& arg0;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void invoke(const char *ptr, std::size_t size) {
std::uint8_t call_id, call_version;
static const char* names[] = {
"yarmi_error"
,"registration"
,"activation"
,"login"
,"logout"
,"users_online"
};
static const std::uint8_t versions[] = { 0, 0, 0, 0, 0, 0 };
try {
yas::binary_mem_iarchive ia(ptr, size, yas::no_header);
ia & call_id
& call_version;
if ( call_id < 0 || call_id > 5 ) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"%s::%s(): bad call_id %d"
,"client_invoker"
,__FUNCTION__
,static_cast(call_id)
);
throw std::runtime_error(errstr);
}
if ( call_version > versions[call_id] ) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"%s::%s(): bad call_version %d for call_id %d(%s::%s())"
,"client_invoker"
,__FUNCTION__
,static_cast(call_version)
,static_cast(call_id)
,"client_invoker"
,names[call_id]
);
throw std::runtime_error(errstr);
}
switch ( call_id ) {
case 0: {
std::uint8_t arg0;
std::uint8_t arg1;
std::string arg2;
ia & arg0
& arg1
& arg2;
impl.on_yarmi_error( arg0 , arg1 , arg2);
};
break;
case 1: {
std::string arg0;
std::string arg1;
ia & arg0
& arg1;
impl.on_registration(arg0, arg1);
};
break;
case 2: {
std::string arg0;
ia & arg0;
impl.on_activation(arg0);
};
break;
case 3: {
std::string arg0;
ia & arg0;
impl.on_login(arg0);
};
break;
case 4: {
std::string arg0;
ia & arg0;
impl.on_logout(arg0);
};
break;
case 5: {
std::vector arg0;
ia & arg0;
impl.on_users_online(arg0);
};
break;
}
} catch (const std::exception &ex) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"std::exception is thrown when %s::%s() is called: '%s'"
,"client_invoker"
,names[call_id]
,ex.what()
);
yarmi_error(call_id, call_version, errstr);
} catch (...) {
char errstr[1024] = {0}; std::snprintf(
errstr
,sizeof(errstr)
,"unknown exception is thrown when %s::%s() is called"
,"client_invoker"
,names[call_id]
);
yarmi_error(call_id, call_version, errstr);
}
}
private:
Impl &impl;
IO &io;
}; // struct client_invoker
template
struct server_invoker {
server_invoker(Impl &impl, IO &io)
:impl(impl)
,io(io)
{}
void yarmi_error(const std::uint8_t &arg0, const std::uint8_t &arg1, const std::string &arg2) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(0)
& static_cast(0)
& arg0
& arg1
& arg2;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void registration(const std::string &arg0, const std::string &arg1) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(1)
& static_cast(0)
& arg0
& arg1;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void activation(const std::string &arg0) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(2)
& static_cast(0)
& arg0;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void login(const std::string &arg0) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(3)
& static_cast(0)
& arg0;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void logout(const std::string &arg0) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(4)
& static_cast(0)
& arg0;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void users_online(const std::vector &arg0) {
yas::binary_mem_oarchive oa(yas::no_header);
oa & static_cast(5)
& static_cast(0)
& arg0;
yas::binary_mem_oarchive pa;
pa & oa.get_intrusive_buffer();
io.send(pa.get_shared_buffer());
}
void invoke(const char *ptr, std::size_t size) {
std::uint8_t call_id, call_version;
static const char* names[] = {
"yarmi_error"
,"registration"
,"activation"
,"login"
,"logout"
,"users_online"
};
static const std::uint8_t versions[] = { 0, 0, 0, 0, 0, 1 };
try {
yas::binary_mem_iarchive ia(ptr, size, yas::no_header);
ia & call_id
& call_version;
if ( call_id < 0 || call_id > 5 ) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"%s::%s(): bad call_id %d"
,"server_invoker"
,__FUNCTION__
,static_cast(call_id)
);
throw std::runtime_error(errstr);
}
if ( call_version > versions[call_id] ) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"%s::%s(): bad call_version %d for call_id %d(%s::%s())"
,"server_invoker"
,__FUNCTION__
,static_cast(call_version)
,static_cast(call_id)
,"server_invoker"
,names[call_id]
);
throw std::runtime_error(errstr);
}
switch ( call_id ) {
case 0: {
std::uint8_t arg0;
std::uint8_t arg1;
std::string arg2;
ia & arg0
& arg1
& arg2;
impl.on_yarmi_error(arg0, arg1, arg2);
};
break;
case 1: {
std::string arg0;
ia & arg0;
impl.on_registration(arg0);
};
break;
case 2: {
std::string arg0;
std::string arg1;
std::string arg2;
ia & arg0
& arg1
& arg2;
impl.on_activation(arg0, arg1, arg2);
};
break;
case 3: {
std::string arg0;
std::string arg1;
ia & arg0
& arg1;
impl.on_login(arg0, arg1);
};
break;
case 4: {
impl.on_logout();
};
break;
case 5: {
switch ( call_version ) {
case 0: {
impl.on_users_online();
};
break;
case 1: {
std::string arg0;
ia & arg0;
impl.on_users_online(arg0);
};
break;
}
};
break;
}
} catch (const std::exception &ex) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"std::exception is thrown when %s::%s() is called: '%s'"
,"server_invoker"
,names[call_id]
,ex.what()
);
yarmi_error(call_id, call_version, errstr);
} catch (...) {
char errstr[1024] = {0};
std::snprintf(
errstr
,sizeof(errstr)
,"unknown exception is thrown when %s::%s() is called"
,"server_invoker"
,names[call_id]
);
yarmi_error(call_id, call_version, errstr);
}
}
private:
Impl &impl;
IO &io;
}; // struct server_invoker
} // ns yarmi
(My other project, YAS, is used as serialization )
As a bonus, a system procedure was added
yarmi_error()- it is used to inform the opposite side that an error occurred while trying to make a call. Look carefully, в client_invoker::invoke()deserialization and a call are wrapped in a try{}catch()block, and a catch()call is made in the blocks yarmi_error(). Thus, if an exception occurs when deserializing or calling a procedure, it will be successfully caughtcatch()block, and exception information will be sent to the caller. The same thing will happen in the opposite direction. Those. if the server called the client the procedure during the call of which an exception occurred - the client will send the error information to the server, also additionally reporting the ID and version of the call in which the exception occurred. But yarmi_error()you can use it yourself, none of this prohibits. Example: yarmi_error(call_id, version_id, "message");As you may have noticed, on the names of the procedures described by us, on the side of their implementation, the
on_Class prefix is added
client_invokerand server_invokertake two parameters. Their first bottom is the class in which the called procedures are implemented, the second is the class in which the method is implemented send(yas::shared_buffer buf). If you have the same class performing both roles, you can do this:
struct client_session: yarmi::client_base, yarmi::client_invoker {
client_session(boost::asio::io_service &ios)
:yarmi::client_base(ios, *this)
,yarmi::client_invoker(*this, *this) // <<<<<<<<<<<<<<<<<<<<<<<<<
{}
};
The final version looks like this:
struct client_session: yarmi::client_base, yarmi::client_invoker {
client_session(boost::asio::io_service &ios)
:yarmi::client_base(ios, *this)
,yarmi::client_invoker(*this, *this)
{}
void on_registration(const std::string &msg, const std::string ®key) {}
void on_activation(const std::string &msg) {}
void on_login(const std::string &msg) {}
void on_logout(const std::string &msg) {}
void on_users_online(const std::vector &users) {}
};
The interface to the opposite side will be inherited from the ancestor
yarmi::client_invoker. That is, for example, being in our constructor client_session, you can call the procedure registration()as follows:{
registration("niXman");
}
The answer we get in our implementation is
client_session::on_registration(std::string msg, std::string regkey)Everything!
Of the deficiencies, the following should be noted:
Commands cannot be used in type names describing procedures, because the preprocessor does not understand the context in which they are used. It will be fixed.
Ultimately, all this resulted in a project called YARMI (Yet Another RMI).
The described code generator is encoded in one file - yarmi.hpp . In total, it took one business day to implement the code generator.
An example of the use of this whole thing can be seen here and here . The first test project is still not completed, unfortunately.
In addition to the described, on the project page you will find codes of the asynchronous multi-user single-threaded server, and client codes.
Instead of a conclusion
Plans:
1. Generation of several interfaces
2. Describe the specification (although it is nowhere simpler)
3. The ability to use your own locator
I would be grateful for constructive criticism and suggestions.
PS
This code is used in several of our commercial projects, in game dev.