PHPUnit && ordered tests

    All programmers are lazy. And everyone wants not to write additional code, but to use the already ready one. Moreover, this is a good practice.

    So I had a problem, in which I wanted not to do copy-paste, but to run several tests. But, each subsequent test depended on the data of the previous one, and so on and so forth ... As a result, I needed a strict sequence of tests and the ability to respond to dependencies. What a solution, look under the cut ...


    Precondition

    There is a facade of the system that can perform any action. There are dependent actions, but there are independent actions. Dependent actions require an independent action (and sometimes more than one). Therefore, in the end, we should get the so-called ordered list of actions.

    Since all the actions that are called need to be tested, you must accordingly get an ordered list of tests. PHPUnit was used as a test framework, as its conditions were enough for a certain type of action.

    Condition

    So, now I obviously want to throw tomatoes and say that there are dependency in PHPUnit . And indeed it is. BUT, as written by the author himself, dependencies do not allow you to specify a strict test execution order.
    PHPUnit supports the declaration of explicit dependencies between test methods. Such dependencies do not define the order in which the test methods are to be executed but they allow the returning of an instance of the test fixture by a producer and passing it to the dependent consumers.

    In addition, as I found out later, to indicate the dependence of one test method in the descendant class on the test method of the parent class, this is the same as immediately indicating that the test should be ignored.
    For instance:
    Class ParentTestCase extends PHPUnit_Framework_TestCase 
    {
        public function testOne()
        {
            self::assertTrue(true);
        }
        /**
         * @depends testOne
         */
        public function testTwo()
        {
            self::assertTrue(true);
        }
    }
    

    class ChildTestCase extends ParentTestCase
    {
        /**
         * @depends testTwo
         */
        public function testThree()
        {
            self::assertTrue(true);
        }
    }
    

    I will point out that this happens due to the use of reflection , because first methods from the descendant class are collected, and then from the ancestor class.

    In this regard, I came up with the idea to modify PHPUnit so that it supports the sequence of tests and that the basis for this sequence lies in indicating the dependence on the execution of the test.

    Decision

    First, let's determine where we are going to store the order of the test dependencies. To do this, create a descendant of TestCase and add functions to control this order.
    class MagicTestCase extends PHPUnit_Framework_TestCase
    {
        /**
         * @var    array
         */
        protected $order = array();
        /**
         * Sets the orderSet of a TestCase.
         *
         * @param  array $orderSet
         */
        public function setOrderSet(array $orderSet)
        {
            $this->order = $orderSet;
        }
        /**
         * Get the orderSet of a TestCase.
         *
         * @return  array $order
         */
        public function getOrderSet()
        {
            return $this->order;
        }
    }
    


    After that, you must specify the initial dependence for each test. To do this, redefine the addTestMethod method for the PHPUnit_Framework_TestSuite class in the inherited class. Install the dependency:
    $test->setOrderSet(PHPUnit_Util_Test::getDependencies($class->getName(), $name));

    Now it is necessary to supplement the constructor of our TestSuite so that it sorts all the tests in the order we set. There, we define a recursive order for each test.
            foreach($this->tests as $test) {
                $test->setOrderSet(
                    array_unique($this->getRecursiveOrderSet($test, $test->getName()))
                );
            }
            usort($this->tests, array("MagicUtilTest ", "compareTestOrder"));
    


    In addition, the addTestSuite function creates an instance of itself ( PHPUnit_Framework_TestSuite ), and it is necessary that it creates a modified TestSuite , because in the first case, it does not use the modified TestSuite constructor . It all depends on one line, which we redefine inside the method:
    $this->addTest(new MagicTestSuite($testClass));


    The resulting class:
    class MagicTestSuite extends PHPUnit_Framework_TestSuite
    {
        /**
         * Constructs a new TestSuite:
         *
         * @param  mixed  $theClass
         * @param  string $name
         * @throws InvalidArgumentException
         */
        public function __construct($theClass = '', $name = '')
        {
            parent::__construct($theClass, $name);
            foreach($this->tests as $test) {
                $test->setOrderSet(
                    array_unique($this->getRecursiveOrderSet($test, $test->getName()))
                );
            }
            usort($this->tests, array("MagicUtilTest ", "compareTestOrder"));
        }
        /**
         * @param  $object
         * @param  $methodName
         * @return array
         */
        protected function getRecursiveOrderSet($object, $methodName)
        {
            $orderSet = array();
            foreach($this->tests as $test) {
                if ($test->getName() == $methodName && get_class($object) == get_class($test)) {
                    $testOrderSet = $test->getOrderSet();
                    if (!empty($testOrderSet)) {
                        foreach($testOrderSet as $orderMethodName) {
                            if(!in_array($orderMethodName, $orderSet)) {
                                $orderResult = $this->getRecursiveOrderSet($test, $orderMethodName);
                                $orderSet = array_merge($orderSet, $orderResult);
                            }
                        }
                    }
                    $orderSet = array_merge($orderSet, $testOrderSet);
                }
            }
            return $orderSet;
        }
        /**
         * @param ReflectionClass  $class
         * @param ReflectionMethod $method
         */
        protected function addTestMethod(ReflectionClass $class, ReflectionMethod $method)
        {
            $name = $method->getName();
            if ($this->isPublicTestMethod($method)) {
                $test = self::createTest($class, $name);
                if ($test instanceof PHPUnit_Framework_TestCase ||
                    $test instanceof PHPUnit_Framework_TestSuite_DataProvider) {
                    $test->setDependencies(
                      PHPUnit_Util_Test::getDependencies($class->getName(), $name)
                    );
                }
                $test->setOrderSet(PHPUnit_Util_Test::getDependencies($class->getName(), $name));
                $this->addTest($test, PHPUnit_Util_Test::getGroups(
                  $class->getName(), $name)
                );
            }
            else if ($this->isTestMethod($method)) {
                $this->addTest(
                  self::warning(
                    sprintf(
                      'Test method "%s" is not public.',
                      $name
                    )
                  )
                );
            }
        /**
         * Adds the tests from the given class to the suite.
         *
         * @param  mixed $testClass
         * @throws InvalidArgumentException
         */
        public function addTestSuite($testClass)
        {
            if (is_string($testClass) && class_exists($testClass)) {
                $testClass = new ReflectionClass($testClass);
            }
            if (!is_object($testClass)) {
                throw PHPUnit_Util_InvalidArgumentHelper::factory(
                  1, 'class name or object'
                );
            }
            if ($testClass instanceof PHPUnit_Framework_TestSuite) {
                $this->addTest($testClass);
            }
            else if ($testClass instanceof ReflectionClass) {
                $suiteMethod = FALSE;
                if (!$testClass->isAbstract()) {
                    if ($testClass->hasMethod(PHPUnit_Runner_BaseTestRunner::SUITE_METHODNAME)) {
                        $method = $testClass->getMethod(
                          PHPUnit_Runner_BaseTestRunner::SUITE_METHODNAME
                        );
                        if ($method->isStatic()) {
                            $this->addTest(
                              $method->invoke(NULL, $testClass->getName())
                            );
                            $suiteMethod = TRUE;
                        }
                    }
                }
                if (!$suiteMethod && !$testClass->isAbstract()) {
                    $this->addTest(new MagicTestSuite($testClass));
                }
            }
            else {
                throw new InvalidArgumentException;
            }
        }
    }
    

    Well and accordingly, add a utility function that will compare two tests and indicate in which direction to sort them.
    class MagicUtilTest 
    {
        /**
         * @static
         * @param PHPUnit_Framework_TestCase $object1
         * @param PHPUnit_Framework_TestCase $object2
         * @return int
         */
        public static function compareTestOrder(PHPUnit_Framework_TestCase $object1, PHPUnit_Framework_TestCase $object2)
        {
            if (in_array($object2->getName(), $object1->getOrderSet())) {
                return 1;
            }
            if (in_array($object1->getName(), $object2->getOrderSet())) {
                return -1;
            }
            return 0;
        }
    }
    


    Application

    Create a test run file:
    require dirname(__FILE__) . DIRECTORY_SEPARATOR.' runSuite.php';
    PHPUnit_Util_Filter::addFileToFilter(__FILE__, 'PHPUNIT');
    require_once 'PHPUnit/TextUI/Command.php';
    $tests= runSuite::suite();
    PHPUnit_TextUI_TestRunner::run($tests);
    exit;
    


    We create the general TestSuite class which will directly launch tests. We inherit it from our created:
    require_once ‘MagicTestSuite.php’;
    class runSuite extends MagicTestSuite
    {
    	public static function suite()
           {
    		$suite  = new self();
    		$suite->addTestSuite(“ChildTestCase”);
           }
    }
    

    Well, TestCases, which include tests and dependencies:
    require_once ‘MagicTestCase.php’;
    class ParentTestCase extends MagicTestCase
    {
        public function testOne()
        {
            self::assertTrue(true);
        }
        /**
         * @depends testOne
         */
        public function testTwo()
        {
            self::assertTrue(true);
        }
    }
    

    require_once ‘ParentTestCase.php’;
    class ChildTestCase extends ParentTestCase
    {
        /**
         * @depends testTwo
         */
        public function testThree()
        {
            self::assertTrue(true);
        }
    }
    


    Limitations

    The current solution does not allow breaking interlocking dependencies. That is, if we have two tests that will depend on each other, we get an endless loop.

    Of course, it was possible to build a tree of test dependencies ... but to be honest I didn’t think up in which case (besides a human error) you can use closure dependencies for tests.

    Also popular now: