Semi-automatic registration of unit tests in pure C

    After reading the book Test Driven Development for Embedded C, I began my introduction to the world of unit testing with the cppUtest framework. Not least because in it a freshly written test is registered and launched independently. You have to pay for it - using C ++, dynamic memory allocation somewhere in the depths of the framework. Maybe it can be somehow simpler?
    Most recently, I learned about the minimalistic minUnit framework , which fits in just 4 lines.

    I will give them here for clarity:

    #define mu_assert(message, test) do { if (!(test)) return message; } while (0)
     #define mu_run_test(test) do { char *message = test(); tests_run++; \
                                    if (message) return message; } while (0)
     extern int tests_run;
    

    Simple and beautiful. At the same time, writing the test looks like this:

    static char * test_foo() {
         mu_assert("error, foo != 7", foo == 7);
         return 0;
     }
    

    Unfortunately, when I tried to use this framework, I quickly realized that I was terribly lazy to register each test with my hands. This is because you need to have a header file for the file with the tests, add a declaration to each test in this file, then go to main and register a call!

    I looked at other frameworks written in pure C: the same thing almost everywhere. As an alternative, separate programs are offered that scan source codes with tests and generate code to run.
    But maybe it can be easier? This post

    instilled confidence in me , where a linker is used to register tests. But I did not want to get attached to the linker and compiler-specific attributes.
    As far as I know, in pure C it is impossible to do a full test registration. What about semi-automatic?

    The idea took shape as follows. For each module, the module_tests.c file is written, all tests for this module are written in it. These tests form a group. In the same file, a magic function is written to run all the tests in a group.
    And in main, you only need to register with your hands the launch of the group, and not of each test individually.
    This boils down to the following task: you need to somehow get a list of all the functions in the file. In C, this can only be done using a preprocessor. But how? For example, if functions will be called somehow monotonously.

    The “service” names of the tests may well be anything, if only the title of the test was intelligible!
    This means that with the help of a preprocessor it is necessary to generate names for test functions, moreover, uniformly and according to a single template. For example, like this:

    #define UMBA_TEST_COUNTER        BOOST_PP_COUNTER
    #define UMBA_TEST_INCREMENT()    BOOST_PP_UPDATE_COUNTER()
    #define UMBA_TOKEN(x, y, z)  x ## y ## z
    #define UMBA_TOKEN2(x, y, z) UMBA_TOKEN(x,y,z)
    #define UMBA_TEST( description )      static char * UMBA_TOKEN2(umba_test_, UMBA_TEST_COUNTER, _(void) )
    

    Frankly, I used boost for the first time in my life and was amazed to the core by the power of the C preprocessor!
    Now you can write tests as follows:

    UMBA_TEST("Simple Test") // получается static char * umba_test_0_(void)
    {
        uint8_t a = 1;
        uint8_t b = 2;    
        UMBA_CHECK(a == b, "MATHS BROKE");    
        return 0;    
    }
    #include UMBA_TEST_INCREMENT()
    

    After this inclusion, the counter is incremented and the name for the next test will be generated name static char * umba_test_1_ (void).

    It remains only to generate a function that will run all the tests in the file. To do this, an array of pointers to functions is created and populated with pointers to tests. Then the function simply calls each test from the array in a loop.
    This function will need to be written at the end of the test file so that the value of UMBA_TEST_COUNTER equals the number of the last test.
    To generate an array of pointers, I first followed a simple path and wrote a helper file like this:

    #if   UMBA_TEST_COUNTER == 1
    	#define UMBA_LOCAL_TEST_ARRAY  UmbaTest umba_local_test_array[ UMBA_TEST_COUNTER ] = {umba_test_0_};
    #elif UMBA_TEST_COUNTER == 2
    	#define UMBA_LOCAL_TEST_ARRAY  UmbaTest umba_local_test_array[ UMBA_TEST_COUNTER ] = {umba_test_0_, umba_test_1_};
    …
    

    In principle, it’s quite possible to get by with this by generating ads for several hundred tests. Then only one file will be needed from boost'a - boost / preprocessor / slot / counter.hpp.
    But, since I started using boost, why not continue?

    #define UMBA_DECL(z, n, text) text ## n ## _,
    #define UMBA_LOCAL_TEST_ARRAY  UmbaTest umba_local_test_array[ UMBA_TEST_COUNTER ] = { BOOST_PP_REPEAT( UMBA_TEST_COUNTER, UMBA_DECL, umba_test_ ) }
    

    Only two lines, but what power is hidden behind them!
    Add the trivial code for the group launch function itself:

    #define UMBA_RUN_LOCAL_TEST_GROUP( groupName )         UMBA_LOCAL_TEST_ARRAY; \
                                                           char * umba_run_test_group_ ## groupName ## _(void) \
                                                           { \
                                                               for(uint32_t i=0; i < UMBA_TEST_COUNTER; i++) \
                                                               { \
                                                                   tests_run++; \
                                                                   char * message = umba_local_test_array[i](); \
                                                                   if(message) \
                                                                       return message; \
                                                               } \
                                                               return 0; \
                                                           } \
    													   

    And to start it from main:

    #define UMBA_EXTERN_TEST_GROUP( groupName )       char * umba_run_test_group_ ## groupName ## _(void);                                  
    #define UMBA_RUN_GROUP( groupName )     do { \
                                                char *message = umba_run_test_group_ ## groupName ## _(); \
                                                tests_run++; \
                                                if (message) return message; \
                                             } while (0)
    

    Voila. Now starting a group with any number of tests looks the same:

    UMBA_EXTERN_TEST_GROUP( SimpleGroup )
    static char * run_all_tests(void)
    {
        UMBA_RUN_GROUP( SimpleGroup );    
        return 0;
    }
    int main(void)
    {	
        char *result = run_all_tests();
        if (result != 0 ) 
        {
            printf("!!!!!!!!!!!!!!!!!!!\n");
            printf("%s\n", result);
        }
        else 
        {
            printf("ALL TESTS PASSED\n");
        }    
        printf("Tests run: %d\n", tests_run-1);
    	return 0;
    }
    


    I am quite pleased with this result. Mechanical actions when writing a test are now noticeably less.
    All the mentioned macros fit in 40-50 lines, which, unfortunately, is slightly larger than minUnit (and much less obvious).
    The entire code.

    Yes, there isn’t a lot of functionality from large frameworks, but honestly, I’ve never had time to use in the test something other than a simple check like CHECK (if true).
    Description for the test is simply thrown out, but to do something useful with it seems to be easy if you suddenly want to.

    What I would like to find out:
    1. Have I invented something new or have I been using this trick for many years?
    2. Can this be improved somehow? I don’t really like the need to write some strange include after each test, but I did not find other counter implementations on the preprocessor (__COUNT__ does not support my compiler).
    3. Should I use a makeshift framework in production?
    4. How the hell does BOOST_PP_COUNTER work ?! Even on stackoverflow, the answer to the corresponding question is “magic”.

    Also popular now: