Why developers do not like Unit Tests

    Maybe they just don’t know how to “cook” them?

    Intro


    On duty, I am involved in the development of applications for microcontrollers. But it so happened that I was engaged in various kinds of testing (both my own and someone else's code) more than, in fact, development. Far from the first attempt, I managed to master TDD. Now the volumes of the test and “combat” code have more or less leveled :)
    I hope that after reading this article the question “Why not the first time?” Will be removed.

    Facts


    In my professional career, I often hear statements of the following nature:
    • “Why are we going to spend time on unit tests, are we not yet in time to complete the project on time?”
    • “Why do tests dictate how to write code?”
    • “Let's just write the code, testers will find all the defects. Then we’ll fix it. ”
    • “Here, colleagues implemented a new feature, we need to cover it with unit tests”

    Even supporters of flexible development methodologies do not always understand the value of this type of testing. Actually, the Agile article from the point of view of the programmer triggered this publication.

    As it usually happens


    Let's imagine that in the process of developing a certain system, a need arose for implementing a linked list. For simplicity, I will limit myself to just push and pop (FIFO) functions and an integer as payload.
    Without additional requirements for this list, we can expect that an experienced developer Maxim will first study the examples that are on the Internet and take one of them as a basis.

    As a result, we have the following implementation option:
    file my_list.h
    #ifndef MY_LIST_H
    #define MY_LIST_H
    #ifndef NULL    /* just for this example */
    #define NULL 0
    #endif
    void list_push( int val );
    int list_pop( void );
    #endif
    


    file my_list.c
    #include "my_list.h"
    #include 
    typedef struct node
    {
        int val;
        struct node * next;
    } node_t;
    static node_t * list_head;
    void list_push( int val )
    {
        node_t * current = list_head;
        if(list_head == NULL)
        {
            list_head = malloc(sizeof(node_t));		
            list_head->val = val; 
            list_head->next = NULL;
        }
        else
        {
            while (current->next != NULL) 
            {
                current = current->next;
            }
            current->next = malloc(sizeof(node_t));
            current->next->val = val;
            current->next->next = NULL;
        }
    }
    int list_pop( void )
    {
        int retval = -1;
        node_t * next_node = NULL;
        if (list_head == NULL)
        {
            return -1;
        }
        next_node = list_head->next;
        retval = list_head->val;
        free(list_head);
        list_head = next_node;
        return retval;
    }
    


    Well, there is an implementation. Integrated the code into the system, “lamented” everything works.
    Then someone recalls that linked lists are a very responsible matter, address arithmetic is there ... memory leaks ... And we should write unit tests, at least for this module - well, that would be safe to fall into.
    And I'm almost 100% sure that another developer will be engaged in this - Andrei. Andrey is a novice developer and he just needs to gain experience. And since the development of the system is not finished yet, the guys with experience still have something to do.

    Andrew: “And how to test this?”
    Maxim: “Well, look into the code, figure out how it is implemented, and cover all branches of the code with tests so that you don’t miss anything”
    Andrei: “I want to start testing with the list_pop () function. It allocates memory for the new item and adds it to the list. But there is static and I can’t get to the list from the test code. ”
    static node_t * list_head;
    void list_push( int val )
    {
        node_t * current = list_head;
        if(list_head == NULL)
        {
            list_head = malloc(sizeof(node_t));		
            list_head->val = val; 
            list_head->next = NULL;
        }
    ...
    

    Maxim: “Ah ... well, let me make a crutch specifically for your tests. It will not work in the production build, but it will help you. Externally in the test and that’s it. ”
    #ifdef UNIT_TEST
    node_t * list_head;
    #else
    static node_t * list_head;
    #endif
    


    It is natural to expect such an implementation of the test:
    file test_my_list.c
    #include "unity.h"
    #include "my_list.h"
    void setUp(void)
    {
    } 
    void tearDown(void)
    {
    }
    typedef struct node
    {
        int val;
        struct node * next;
    } node_t;
    extern node_t * list_head;
    void test_1( void )
    {
        list_push( 1 );
        TEST_ASSERT_NOT_NULL( list_head );  /* Check that memory is allocated */
        TEST_ASSERT_EQUAL_INT( 1, list_head->val );  /* Check that value is set*/
        TEST_ASSERT_NULL( list_head->next );  /* Check that the next pointer has appropriate value */
    }
    


    I think the further expansion of the code coverage with new tests is obvious to the reader. The result is achieved - the module is tested with unit tests, coverage is 100%. You can sleep peacefully.

    What's wrong with that?


    Of course, the story described above may have another development. I'm just trying to say that unit tests are different.
    In this case, the tests have the following disadvantages:
    • Tests test the code (however strange it sounds)
    • Tests force the developer to make crutches
    • Tests require titanic efforts to support them even in the case of refactoring, not to mention significant changes.
    • “Failed” tests do not mean at all that some functionality does not work


    And if you write tests first, and then code. Will this help?


    Unfortunately not. Or not always.
    I am not an ardent supporter of the basic principle of TDD, which forces you to first write a test for non-existent code, and then write code to ensure that this test passes. Sometimes, I write a small piece of code before testing for it.

    The main thing is different. In my opinion, it is very important to consider each module as an independent system:
    • Try to formulate the requirements for this system that it must meet
    • It is the compliance with these requirements that I try to check with unit tests
    • Try not to delve into the implementation of this system and use only its external API for testing


    Someone, probably, will notice "so it is BDD". And most likely he will be right. But it doesn’t matter what is primary in your development: tests, or behavior, or the code itself, which has already been written very, very much. It is important how you write unit tests.

    For example, the first test for the list implemented above may be as follows:
    /*
    * Given the list is empty
    * When I push 1 to the list
    * Then the pop function shall return 1
    */
    void test_simple( void )
    {
        list_push( 1 );
        TEST_ASSERT_EQUAL_INT( 1, list_pop() );
    }
    

    Second test:
    /*
    * Given the list is empty
    * When I push 1 to the list
    * And I push 2 to the list
    * Then the first call of the pop function shall return 1
    * And the second call of the pop function shall return 2
    */
    void test_order( void )
    {
        list_push( 1 );
        list_push( 2 );
        TEST_ASSERT_EQUAL_INT( 1, list_pop() );
        TEST_ASSERT_EQUAL_INT( 2, list_pop() );
    }
    

    In the first test, we checked that the module APIs are in principle functional. We also made sure that what we save in the list can be retrieved later.
    In the second test, we checked that the items are retrieved from the list in the order in which they were placed there.
    And it was just such functionality that we were initially interested in when designing the entire software package, but by no means the way in which it was implemented.

    Benefits


    With this approach, the test disadvantages described above are eliminated:
    Tests test code
    Tests test module behavior without knowing anything about its implementation (black-box)
    Tests force the developer to make crutches
    when testing through the API, the need for this is extremely rare
    Tests require titanic efforts to support them even in the case of refactoring, not to mention significant changes.
    in our example, the implementation can be completely changed (an array instead of a linked list, a bidirectional list instead of unidirectional, etc.), which should not affect its behavior
    “Failed” tests do not mean at all that some functionality does not work
    since refactoring the code (if successful) does not affect the test results in any way, there is only one reason for the test failures - something really doesn’t work

    Extra buns


    In addition to the above advantages, unit tests have another, in my opinion, very important advantage - they improve the quality of the code.
    Whether we like it or not, the test code (the one that can be physically tested) is more flexible, more portable, more scalable. Maybe somehow (I'm afraid to praise).

    Unfortunately, the list implemented above has still not been tested for memory leaks. But this moment was far from the last in the list of fears, which made the team even think about unit tests for a linked list.

    In order to verify that there are no leaks, we must control the allocation / release of memory. And to make mock on the functions of the standard library is not an easy task.

    There is a solution - add an abstraction layer between the module and the standard library with the following interface:
    file my_list_mem.h
    #ifndef MY_LIST_MEM
    #define MY_LIST_MEM
    void * list_alloc_item( int size );
    void list_free_item( void * item );
    #endif
    


    Then, the implementation of the list will take the form:
    file my_list.c
    #include "my_list.h"
    #include "my_list_mem.h"
    typedef struct node
    {
        int val;
        struct node * next;
    } node_t;
    static node_t * list_head;
    void list_push( int val )
    {
        node_t * current = list_head;
        if(list_head == NULL)
        {
            // list_head = malloc(sizeof(node_t));		
            list_head = (node_t*)list_alloc_item( sizeof(node_t) );
            list_head->val = val; 
            list_head->next = NULL;
        }
        else
        {
            while (current->next != NULL) 
            {
                current = current->next;
            }
            // current->next = malloc(sizeof(node_t));
            current->next = (node_t*)list_alloc_item( sizeof(node_t) );
            current->next->val = val;
            current->next->next = NULL;
        }
    }
    int list_pop( void )
    {
        int retval = -1;
        node_t * next_node = NULL;
        if (list_head == NULL)
        {
            return -1;
        }
        next_node = list_head->next;
        retval = list_head->val;
        // free(list_head);
        list_free_item( list_head );
        list_head = next_node;
        return retval;
    }
    


    Already implemented tests will not change in any way, with the exception of adding mock-s:
    file test_my_list.c
    #include "unity.h"
    #include "my_list.h"
    #include "mock_my_list_mem.h"
    #include 
    static void * list_alloc_item_mock( int size, int numCalls )
    {
        return malloc( size );
    }
    static void list_free_item_mock( void * item, int numCalls )
    {
        free( item );
    }
    void setUp(void)
    {
        list_alloc_item_StubWithCallback( list_alloc_item_mock );
        list_free_item_StubWithCallback( list_free_item_mock );
    }
    void tearDown(void)
    {
    }
    /*
    * Given the list is empty
    * When I push 1 to the list
    * Then the pop function shall reutrn 1
    */
    void test_nominal( void )
    {
        list_push( 1 );
        TEST_ASSERT_EQUAL_INT( 1, list_pop() );
    }
    /*
    * Given the list is empty
    * When I push 1 to the list
    * And I push 2 to the list
    * Then the first call of the pop function shall return 1
    * And the second call of the pop function shall return 2
    */
    void test_order( void )
    {
        list_push( 1 );
        list_push( 2 );
        TEST_ASSERT_EQUAL_INT( 1, list_pop() );
        TEST_ASSERT_EQUAL_INT( 2, list_pop() );
    }
    


    And finally, new memory management tests:
    file test_my_list_mem_leak.c
    #include "unity.h"
    #include "my_list.h"
    #include "mock_my_list_mem.h"
    #include 
    static int mallocCounter;
    static int freeCounter;
    static void * list_alloc_item_mock( int size, int numCalls )
    {
        mallocCounter++;
        return malloc( size );
    }
    static void list_free_item_mock( void * item, int numCalls )
    {
        freeCounter++;
        free( item );
    }
    void setUp(void)
    {
        list_alloc_item_StubWithCallback( list_alloc_item_mock );
        list_free_item_StubWithCallback( list_free_item_mock );
        mallocCounter = 0;
        freeCounter = 0;
    }
    void tearDown(void)
    {
    }
    /*
    * Given the list is empty
    * When I push an item to the list
    * Then one part of mеmory shall be allocated
    * And no part of memory shall be released
    */
    void test_push( void )
    {
        list_push( 1 );
        TEST_ASSERT_EQUAL_INT( 1, mallocCounter );
        TEST_ASSERT_EQUAL_INT( 0, freeCounter );
    }
    /*
    * Given the list is empty
    * When get the item from the list pushed before
    * Then one part of mеmory shall be released
    * And no part of memory shall be allocated
    */
    void test_pop( void )
    {
        list_pop();
        TEST_ASSERT_EQUAL_INT( 0, mallocCounter );
        TEST_ASSERT_EQUAL_INT( 1, freeCounter );
    }
    


    As a result, on the one hand, we checked the correctness of working with memory, on the other, we implemented an additional layer containing wrappers for the malloc () and free () functions. And if in the future the memory allocation mechanism is changed (a staic array of elements of a fixed size, some RTOS memory_pools) - our code is ready for these changes, and the list itself and tests for its functionality will not be affected in any way.

    Conclusions


    Yes, ... conclusions, only two
    1. unit tests are good, the main thing is to write them correctly.
    2. And in order to make this possible, you should think about testing when developing code.

    PS


    All coincidences with real people are random.
    The material www.learn-c.org was used as the basis for the implementation of the speaker .
    All tests were written using Unity / CMock / Ceedling tools.

    Also popular now: