DYefimov November 18, 2014 at 19:07

My bike to reflection in c ++

From the sandbox

Inspired by the publication “Logger with Member Functions That Are Not There,” I decided to give my meta-bike to the public for reflection, which I use quite successfully, and not just for logging. But let's start with simple logging, continuing the aforementioned publication.

When implementing logging, the tasks for themselves were set as follows:

Solve the problem at the “meta level” in order to be detached from the final implementation
Frontend api for logging should be simple and transparent
Have the ability to turn off unnecessary logging levels at the compilation stage with one constant; for example: everything above LOG_NOTICE should not get into the resulting binari

The frontend looks like this:

1. In the CConnection constructor, we log in:

TestLog::Log() << *this << "connection created";

2. Where CConnection is inherited from

public TLogHeader<'c', CConnection>

3. Which, using CRTP , knows that in CConnection there is such:

	using this_t = CConnection;
	int m_id;           using m_id_t = TParamAccessor;
	std::string m_name; using m_name_t = TParamAccessor;
    char m_state;       using m_state_t = TParamAccessor;
	using log_header_param_list_t = std::tuple;

4. And turns this data into a log line:

c:23:test_conn 1:a:connection created

Where the colon is listed: c - the first parameter of the TLogHeader template, and then the values m_id, m_name, m_state

In my case, all this is further written to rsyslog and there it is parsed into the appropriate fields for MongoDB (depending on the type of header, i.e. the first parameter to the colon).

Now I have this approach in production on a highly loaded system, so that any structural stones will be in peak, or better for cat. If laziness, then here's a working example on http://coliru.stacked-crooked.com/ because under a cat of a code on 250 lines, but there are a lot of letters in the description.

Sorry, if there will be a lot of transliteration and English terms. The article is a complete gag, but for 20 years it somehow fell behind Russian terminology. Therefore, there will be no references to literature, although at one point they helped me at stackoverflow.com, but to give a link there means to burn my anonymity as a sand user, as I understand it.

So.

Part 1 - Logging

Let's start with a simple

TestLog::Log() << *this << "connection created";

"Template TestLog :: Log" in our case is a type (but it can also be implemented as a function that returns a type). As we see, a temporary object of this type is created. And, knowing that any object will be destroyed when leaving the scope, it is clear that for this type it will be ';'. Accordingly, the behavior model is simple - we merge data into this type, after which the destructor is called.

This is implemented in the TSink class.

template
struct TSink
{
    ~TSink()
    {
        std::cerr << "std::err log line flushed: \"" << stream.str() << " \"" << std::endl;
    }
    std::stringstream stream;
    template
    TSink& operator << (T&& val)
    {
        stream << val;
        return *this;
    }
};
template
struct TSink
{
    template
    TSink& operator << (T&& val)
    {
        return *this;
    }
};

Partial specialization is needed here in order to: “Have the ability to turn off unnecessary logging levels at the compilation stage with one constant; for example: everything above LOG_NOTICE should not get into the resulting binari. ” This is a present for us from the compiler (of which I am not 100% sure, because I am not really burning with the desire to decompile the code and see what we have at the output; hello stones!). The idea is simple: the operator of writing to the stream does not manipulate the data, and if we take into account that we had created a temporary object and all we do before deleting it is written to the stream, then the modern compiler will optimize the whole thing for us.

And this is how it will look if instead of cerr we merge into syslog

template
struct TSink
{
    ~TSink()
    {
        syslog(logLevel, stream.str().c_str(), "junk");
    }
    std::stringstream stream;
    template
    TSink& operator << (T&& val)
    {
        stream << val;
        return *this;
    }
};
template
struct TSink
{
    template
    TSink& operator << (T&& val)
    {
        return *this;
    }
};

It is worth mentioning that TSink is already the final implementation tied to a specific logging module (for example: standard output stream, file, syslog, etc.). The TLog class itself, which we will look at later, does not know how it will be implemented, it receives traits that will already refer to a specific sink implementation.

Let's see what TestLog :: Log is.

It:

static const int OUR_LOG_LEVEL = LOG_DEBUG;
using TestLog = TLog;

And, in fact, the implementation of TLog

template class log_traits>
struct TLog : public log_traits
{
	using trats_t = log_traits;
	template
	using sink_type = typename trats_t::template sink_type;
	template
	struct TGetLogOfLevel
	{
		using type = typename std::conditional<
			neededLogLevel == logLevel,
			sink_type logUpTo)>,
			typename TGetLogOfLevel::type
		>::type;
    };
	template
	struct TGetLogOfLevel
	{
        using type = void;
    };
	template
	using Log = typename TGetLogOfLevel::type;
};

At the “input of the template” we get two parameters: the maximum level of logging in this assembly of the program (OUR_LOG_LEVEL) and the template type log_traits, from which we will further inherit.

As you can see from the code, there are two requirements for the log_traits type:

the type must accept int logUpTo (which will be forwarded from our first TLog parameter)
the type must have inside the definition of another template type sink_type, where logLevel is the level of the current logging operation, and isNull is whether we are logging the current level at all (see the TSink partial specialization)

Everything works very simply inside: we define the Log type that is the result of calling the recursive meta-function TGetLogOfLevel (neededLogLevel, logLevel), which in the case of neededLogLevel == logLevel returns a sink_type of the current level, or calls itself recursively with a logging level of +1.

Based on all this, we will make the final implementation of log_traits

To do this, we wrap our ready-made TSink in a new TLogTraits class, taking into account the requirements for log_traits described above. Those. define type traits_type as

template
using sink_type = TSink;

We also add the OpenLog function and close it in the destructor (log).

Here is the TLogTraitsStdErr code

template
struct TLogTraitsStdErr
{
    void Open(const char* logName, int facility)
	{
    	std::cerr << "std::err log opened with logName: " << logName << std::endl;
	}
    ~TLogTraitsStdErr()
	{
        std::cerr << "std::err log closed" << std::endl;
	}
    template
    struct TSink
    {
        ~TSink()
        {
            std::cerr << "std::err log line flushed: \"" << stream.str() << " \"" << std::endl;
        }
        std::stringstream stream;
        template
        TSink& operator << (T&& val)
        {
            stream << val;
            return *this;
        }
    };
    template
    struct TSink
    {
        template
        TSink& operator << (T&& val)
        {
            return *this;
        }
    };
	template
	using sink_type = TSink;
};

And by analogy with TLogTraitsSyslog

template
struct TLogTraitsSyslog
{
	static void Open(const char* logName, int facility)
	{
		openlog(logName, LOG_PID | LOG_NDELAY, facility);
		setlogmask(LOG_UPTO(logUpTo));
	}
	~TLogTraitsSyslog()
	{
		closelog();
	}
    template
    struct TSink
    {
        ~TSink()
        {
            syslog(logLevel, stream.str().c_str(), "junk");
        }
        std::stringstream stream;
        template
        TSink& operator << (T&& val)
        {
            stream << val;
            return *this;
        }
    };
    template
    struct TSink
    {
        template
        TSink& operator << (T&& val)
        {
            return *this;
        }
    };
	template
	using sink_type = TSink;
};

And, of course, test this cat

Write testcase

static const int OUR_LOG_LEVEL 
    = LOG_DEBUG;
using TestLog = TLog;
struct CConnection
{
	CConnection()
    {
        TestLog::Log() << "connection created";
    }
    ~CConnection()
    {
        TestLog::Log() << "connection destroyed";
    }
    void DoSomething()
    {
        TestLog::Log() << "connection is doing something";
    }
};
class CServer : public TestLog
{
public:
    CServer()
	{
		TestLog::Open("test_log", 1);
	};
    int Run()
    {
        TestLog::Log() << "Creating connection";
        CConnection test_conn1;
        test_conn1.DoSomething();
        return 0;
    }
};
int main(int argc, char** argv)
{
    CServer server;
    return server.Run();
}

Compile and get this output:

clang++ -std=c++11 -O2 -Wall -pedantic  -pthread main.cpp  && ./a.out
std::err log opened with logName: test_log
std::err log line flushed: "Creating connection "
std::err log line flushed: "connection created "
std::err log line flushed: "connection is doing something "
std::err log line flushed: "connection destroyed "
std::err log closed

And let's play with it on (I don’t even know what to call this resource) Coliru

As a result, we got 25 lines of the TLog meta-class, and ~ 40 for each concrete implementation of log_traits. And the requirements are met: they turned off the unnecessary logging; simply? - like yes; transparent? - If you read the meta code is not difficult.
KISS from my point of view.
How about you? Hello stones 2!

Part 2 - Reflection

Extend CConnection in our testcase

Remember, in the beginning there were interesting buns:

TLogHeader <'c', CConnection>, using CRTP , knows that CConnection has this:

	using this_t = CConnection;
	int m_id;           using m_id_t = TParamAccessor;
	std::string m_name; using m_name_t = TParamAccessor;
    char m_state;       using m_state_t = TParamAccessor;
	using log_header_param_list_t = std::tuple;

Here is our advanced CConnection

struct CConnection : public TLogHeader<'c', CConnection>
{
	CConnection(int _id, const std::string& _name) : m_id(_id), m_name(_name), m_state('a')
    {
        TestLog::Log() << *this << "connection created";
        m_state = 'b';
    }
    ~CConnection()
    {
        TestLog::Log() << *this << "connection destroyed";
    }
    void DoSomething()
    {
        TestLog::Log() << *this << "connection is doing something";
        m_state = 'c';
    }
	using this_t = CConnection;
	int m_id;           using m_id_t = TParamAccessor;
	std::string m_name; using m_name_t = TParamAccessor;
    char m_state;       using m_state_t = TParamAccessor;
	using log_header_param_list_t = std::tuple;
};

Of course, all this can be wrapped in a simple macro, such as PARAM (int, m_id), but we will not for the sanity sake.

TParamAccessor is a type in which one static GetRef function is defined that returns a reference to a class variable.

Like this

struct TParamAccessorDefaultTraits
{
};
template 
struct TParamAccessor : public _traits_t
{
    using traits_t = _traits_t;
    using object_t = _object_t;
    using value_type = _value_type;
	static value_type& GetRef(object_t& data) { return data.*member_t; }
};

Accordingly, in order to push * this into the sink_type stream, we need to inherit CConnection :: log_header_param_list_t from inside TLogHeader <'c' and CConnection> and call sink_type << TParamAccessor :: GetRef (our object).

Warning: here it will be a little difficult to understand, until we look at the meta option for_each

We get the TLogHeader class

template
struct TLogHeader
{
	template
	struct TLogHeaderWriter
	{
		using type = TLogHeaderWriter;
		static void call(typename accessor_t::object_t& obj, std::stringstream& out)
		{
			typename accessor_t::value_type& val = accessor_t::GetRef(obj);
			out << val << ":";
		}
	};
	template
	friend sink_type& operator << (sink_type& sink, object_t& val)
	{
		std::stringstream header;
		using writers = typename for_each::template instantiate;
		for_each::call(*static_cast(&val), header);
		sink << moduleName << ":" << header.str();
		return sink;
	}
};

TLogHeader describes only one function - the overloaded write operator to the stream.
In which we see for_each in two guises:

calling a meta function to determine the type of writers, to which all instances of TLogHeaderWriter <from object_t :: log_header_param_list_t :: from each type inside>) return in the form of tuple))
call of the static call function for all types defined in writers

Let's do the meta hypostasis for_each

To do this, simplify our TLogHeader to call only meta-functions

template
struct TLogHeader
{
	template
	struct TLogHeaderWriter
	{
		using type = TLogHeaderWriter;
	};
	template
	friend sink_type& operator << (sink_type& sink, object_t& val)
	{
		using writers = typename for_each::template instantiate;
		return sink;
	}
};

And write a meta function.
Disclaimer: because for simplicity, cut the final code, did not test; The final one can later be tested live.

Just in case, I’ll clarify: in the terminology most pleasant to me, “a meta-function is a function performed at the stage of compiling the program, and the result is a new type” (in my own words) I will

comment in the code, because further a little confused will go.

For_each code

////////////////////////////////////////////////////////////////
// append to tuple
//мета-функция добавления к кортежу еще одного типа new_t
//можно вызывать как для tuple так и для template argument pack, как видно по использованию частичной специализации
template
struct append_to_tuple
{
	using type = std::tuple;
};
template
struct append_to_tuple>
{
	using type = std::tuple;
};
////////////////////////////////////////////////////////////////
// for_each_impl
//реализация for_each - рекурсивная метафункция, инстантиирующая объект func_t методом instatiate,
//который, в свою очередь дописывает новый тип в конец tuple методом append_to_tuple
//вынесена отдельно, для того, чтобы пробросить индекс и не указывать его в конечной реализации
template
struct for_each_impl
{
	using this_type = typename std::tuple_element>::type;
	using prev_type = for_each_impl;
	template class func_t>
	using instantiate = 
		typename append_to_tuple<
			typename func_t::type, 
			typename prev_type::template instantiate
		>::type;
};
// идем от последнего аргумента к первому
// данная частичная специализация является остановкой рекурсии
template
struct for_each_impl<0, args_t...>
{
	using this_type = typename std::tuple_element<0, std::tuple>::type;
	template class func_t>
	using instantiate = std::tuple::type>;
};
///////////////////////////////////////////////////////////////
// for_each
//конечная реализация.
//ссылается на for_each_impl, пробрасывая количество аргументов шаблона
// + есть две специализации для tuple<с параметрами> и tuple<> без оных 
template
struct for_each
{
	using prev_type = for_each_impl;
	template class func_t>
	using instantiate = typename prev_type::template instantiate;
};
template
struct for_each> : public for_each
{
};
template<>
struct for_each>
{
	template class func_t>
	using instantiate = std::tuple<>;
};

So we got using writers = typename for_each:: template instantiate; where writers will expand to:
std::tuple< TLogHeaderWriter<0, TParamAccessor>, TLogHeaderWriter<1, TParamAccessor>, TLogHeaderWriter<2, TParamAccessor> >;

It remains to go through these types, and call GetRef followed by writing to the stream.

Let's do a run-time hypostasis for_each

And immediately look at the extended for_each, which adds calls to the static function call (only you should pay attention to it, the rest is unchanged)
And here it is - the last piece of code
//////////////////////////////////////////////////////////////// // for_each_impl template struct for_each_impl { using this_type = typename std::tuple_element>::type; using prev_type = for_each_impl; template class func_t> using instantiate = typename append_to_tuple< typename func_t::type, typename prev_type::template instantiate >::type; template static void call(object_t&& obj, call_args_t&&... args) { prev_type::call(std::forward(obj), std::forward(args)...); this_type::call(std::forward(obj), std::forward(args)...); } }; template struct for_each_impl<0, args_t...> { using this_type = typename std::tuple_element<0, std::tuple>::type; template class func_t> using instantiate = std::tuple::type>; template static void call(object_t&& obj, call_args_t&&... args) { this_type::call(std::forward(obj), std::forward(args)...); } template static void call(object_t&& obj) { this_type::call(std::forward(obj)); } }; /////////////////////////////////////////////////////////////// // for_each template struct for_each { using prev_type = for_each_impl; template class func_t> using instantiate = typename prev_type::template instantiate; template static object_t call(object_t&& obj, call_args_t&&... args) { prev_type::call(std::forward(obj), std::forward(args)...); return obj; } template static object_t call(object_t&& obj) { prev_type::call(std::forward(obj)); return obj; } }; template struct for_each> : public for_each { }; template<> struct for_each> { template class func_t> using instantiate = std::tuple<>; template static object_t call(object_t&& obj, call_args_t&&... args) { return obj; } template static object_t call(object_t&& obj) { return obj; } };

I don’t think it is particularly necessary to comment here: call receives at least one parameter, which is our proxy object, and which returns upon completion of work - the rest of the parameters will be forwarded to call calls.
In our case, this is TLogHeaderWirter :: call:
static void call(typename accessor_t::object_t& obj, std::stringstream& out) { typename accessor_t::value_type& val = accessor_t::GetRef(obj); out << val << ":"; }
Where accessor_t :: object_t = CConnection
Well, of course, perfect forwarding is a thing!

Part 3 - Conclusion

Whole code
#include #include #include #include static const int OUR_LOG_LEVEL = LOG_DEBUG; // = LOG_NOTICE; // log up to LOG_DEBUG output: //std::err log opened with logName: test_log //std::err log line flushed: "s:1:test server:Creating connection " //std::err log line flushed: "c:23:test_conn 1:a:connection created " //std::err log line flushed: "c:23:test_conn 1:b:connection is doing something " //std::err log line flushed: "c:23:test_conn 1:c:connection destroyed " //std::err log closed // log up to LOG_NOTICE output: //std::err log opened with logName: test_log //std::err log line flushed: "c:23:test_conn 1:a:connection created " //std::err log line flushed: "c:23:test_conn 1:c:connection destroyed " //std::err log closed // ---------------------------- for_each.h start ----------------------------// #include #include namespace helpers { //////////////////////////////////////////////////////////////// // param accessor struct TParamAccessorDefaultTraits { }; template struct TParamAccessor : public _traits_t { using traits_t = _traits_t; using object_t = _object_t; using value_type = _value_type; static value_type& GetRef(object_t& data) { return data.*member_t; } }; //////////////////////////////////////////////////////////////// // append to tuple template struct append_to_tuple { using type = std::tuple; }; template struct append_to_tuple> { using type = std::tuple; }; //////////////////////////////////////////////////////////////// // for_each_impl template struct for_each_impl { using this_type = typename std::tuple_element>::type; using prev_type = for_each_impl; template class func_t> using instantiate = typename append_to_tuple< typename func_t::type, typename prev_type::template instantiate >::type; template static void call(object_t&& obj, call_args_t&&... args) { prev_type::call(std::forward(obj), std::forward(args)...); this_type::call(std::forward(obj), std::forward(args)...); } }; template struct for_each_impl<0, args_t...> { using this_type = typename std::tuple_element<0, std::tuple>::type; template class func_t> using instantiate = std::tuple::type>; template static void call(object_t&& obj, call_args_t&&... args) { this_type::call(std::forward(obj), std::forward(args)...); } template static void call(object_t&& obj) { this_type::call(std::forward(obj)); } }; /////////////////////////////////////////////////////////////// // for_each template struct for_each { using prev_type = for_each_impl; template class func_t> using instantiate = typename prev_type::template instantiate; template static object_t call(object_t&& obj, call_args_t&&... args) { prev_type::call(std::forward(obj), std::forward(args)...); return obj; } template static object_t call(object_t&& obj) { prev_type::call(std::forward(obj)); return obj; } }; template struct for_each> : public for_each { }; template<> struct for_each> { template class func_t> using instantiate = std::tuple<>; template static object_t call(object_t&& obj, call_args_t&&... args) { return obj; } template static object_t call(object_t&& obj) { return obj; } }; } //namespace helpers // ---------------------------- for_each.h end ----------------------------// using namespace helpers; ////////////////////////////////////////////////////////////////////////////////////////// template struct TLogTraitsStdErr { static void Open(const char* logName, int facility) { std::cerr << "std::err log opened with logName: " << logName << std::endl; } ~TLogTraitsStdErr() { std::cerr << "std::err log closed" << std::endl; } template struct TSink { ~TSink() { std::cerr << "std::err log line flushed: \"" << stream.str() << " \"" << std::endl; } std::stringstream stream; template TSink& operator << (T&& val) { stream << val; return *this; } }; template struct TSink { template TSink& operator << (T&& val) { return *this; } }; template using sink_type = TSink; }; //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// template struct TLogTraitsSyslog { static void Open(const char* logName, int facility) { openlog(logName, LOG_PID | LOG_NDELAY, facility); setlogmask(LOG_UPTO(logUpTo)); } ~TLogTraitsSyslog() { closelog(); } template struct TSink { ~TSink() { syslog(logLevel, stream.str().c_str(), "junk"); } std::stringstream stream; template TSink& operator << (T&& val) { stream << val; return *this; } }; template struct TSink { template TSink& operator << (T&& val) { return *this; } }; template using sink_type = TSink; }; ////////////////////////////////////////////////////////////////////////////////////////// template struct TLogHeader { template struct TLogHeaderWriter { using type = TLogHeaderWriter; static void call(typename accessor_t::object_t& obj, std::stringstream& out) { typename accessor_t::value_type& val = accessor_t::GetRef(obj); out << val << ":"; } }; template friend sink_type& operator << (sink_type& sink, object_t& val) { std::stringstream header; using writers = typename for_each::template instantiate; for_each::call(*static_cast(&val), header); sink << moduleName << ":" << header.str(); return sink; } }; ////////////////////////////////////////////////////////////////////////////////////////// template class log_traits = TLogTraitsSyslog> struct TLog : public log_traits { using trats_t = log_traits; template using sink_type = typename trats_t::template sink_type; template struct TGetLogOfLevel { using type = typename std::conditional< neededLogLevel == logLevel, sink_type logUpTo)>, typename TGetLogOfLevel::type >::type; }; template struct TGetLogOfLevel { using type = void; }; template using Log = typename TGetLogOfLevel::type; }; /////////////////////////////// //testcase using TestLog = TLog; struct CConnection : public TLogHeader<'c', CConnection> { CConnection(int _id, const std::string& _name) : m_id(_id), m_name(_name), m_state('a') { TestLog::Log() << *this << "connection created"; m_state = 'b'; } ~CConnection() { TestLog::Log() << *this << "connection destroyed"; } void DoSomething() { TestLog::Log() << *this << "connection is doing something"; m_state = 'c'; } using this_t = CConnection; int m_id; using m_id_t = TParamAccessor; std::string m_name; using m_name_t = TParamAccessor; char m_state; using m_state_t = TParamAccessor; using log_header_param_list_t = std::tuple; }; class CServer : public TLogHeader<'s', CServer>, public TestLog { public: CServer() : m_id(1), m_name("test server") { TestLog::Open("test_log", 1); }; int Run() { TestLog::Log() << *this << "Creating connection"; CConnection test_conn1(23, "test_conn 1"); test_conn1.DoSomething(); return 0; } using this_t = CServer; int m_id; using m_id_t = TParamAccessor; std::string m_name; using m_name_t = TParamAccessor; using log_header_param_list_t = std::tuple ; }; int main(int argc, char** argv) { CServer server; return server.Run(); }

Here is a ready-made example.
We added another 100 lines of implementation of 2 for_each options to it.
And if you looked at the example you can see that they are in a separate file for_each.h
Since I have been actively using it in 5 modules of my history, such as: a native binary format for communicating with postgres considering the endianess of the end systems, deploying packets from buffers and generating a message format on web sockets at the compilation stage, etc.

And as I said: Stones! Reception!

PS Writing the code took half a day, and the article on Habr’s day ... paradox!

Tags:
metaprogramming
c ++ 11