October 7, 2014 at 13:40

Test :: Spec: Pros, Cons, and Features

  • Tutorial
image

Test :: Spec ( https://metacpan.org/pod/Test::Spec ) - a module for declaratively writing unit tests in Perl. We at REG.RU actively use it, so I want to tell you why it is needed, how it differs from other modules for testing, point out its advantages, disadvantages and implementation features.

This article is not an introduction to unit testing in general, nor to the use of Test :: Spec in particular. Information on working with Test :: Spec can be obtained from the documentation ( https://metacpan.org/pod/Test::Spec and https://metacpan.org/pod/Test::Spec::Mocks ). In the article, we will focus on the specifics and nuances of this module.

Table of contents:
Specifications for the tested code
Unit testing using mock-objects
More minor usefulness.
Clear exception output.
Automatically imports strict / warnings;
Simple and convenient selective launch of tests.
Alternatives are not visible.
Features and rakes.
Displaying test names in ok / is / other does not work.
You cannot place tests inside before / after. Before / after
blocks change the code structure
. Local operator no longer works.
    DSL
    Global caches
    Global variables
    And how differently?
More about it and local
General code
It's hard to write helpers working both with Test :: Spec and Test :: More.
The with function works only for classes.
The with function does not see the difference between a hash and an array
Problems with testing things like memory leaks
The use_ok function is out of place
Interesting
About how to technically fake objects using the expects method
Conclusions
Test :: Spec is good for unit testing of high-level code


Test code specifications


Take a simple test for Test :: More.

Test code:
package MyModule;
use strict;
use warnings;
sub mid {
    my ($first, $last) = @_;
    $first + int( ($last - $first) / 2 );
}
1;

Test itself:
use strict;
use warnings;
use Test::More;
use MyModule;
is MyModule::mid(8, 12), 10, "mid should work";
is MyModule::mid(10, 11), 10, "mid should round the way we want";
done_testing;

work result:
ok 1 - mid should work
ok 2 - mid should round the way we want
1..2

Equivalent test for Test :: Spec:
use Test::Spec;
use MyModule;
describe "MyModule" => sub {
    describe "mid" => sub {
        it "should work" => sub {
            is MyModule::mid(8, 12), 10;
        };
        it "should round the way we want" => sub {
            is MyModule::mid(10, 11), 10;
        };
    };
};
runtests unless caller;

and the result of his work:
ok 1 - MyModule mid should work
ok 2 - MyModule mid should round the way we want
1..2

Everything is very similar. Differences in the structure of the test.

Test :: Spec is a way to declaratively describe specifications for the tested code. This module is created in the image of the well-known RSpec package from the Ruby world, which, in turn, works in accordance with the principles of TDD and BDD. The test code specification describes the functional behavior of the code unit under test ( http://en.wikipedia.org/wiki/Behavior-driven_development#Story_versus_specification ). It makes it easier to read the source code of the test and understand what and how we test. At the same time, description lines of the behavior and entities to which this behavior corresponds are used to display information about successful or failed tests.

Compare these entries:

Ruby:
describe SomeClass do
    describe :process do
        @instance = nil
        before :all do
            @instance = SomeClass.new(45)
        end
        it "should return to_i" do
            @instance.to_i.should == 45
        end
    end
end

Perl:
describe SomeClass => sub {
    describe process  => sub {
        my $instance;
        before all => sub {
            $instance = SomeClass->new(45);
        };
        it "should return to_i" => sub {
            is $instance->to_i, 45;
        };
    };
};

describe - the block where the tests are located (should describe what we are testing). The nesting of describe blocks is not limited, which allows you to structurally declare the desired behavior in the test and set test scripts.

it is one separate test (should describe what should do what we are testing). Testing itself takes place inside the “it” blocks, it is implemented by the familiar ok / is / like functions (by default, all functions from Test :: More, Test :: Deep and Test :: Trap are imported).

before / after - allow you to perform various actions before each test, or before each block of tests.


Unit testing using mock objects


Test :: Spec is ideal for unit testing using mock objects ( https://metacpan.org/pod/Test::Spec::Mocks#Using-mock-objects ). This is its main advantage over other libraries for tests.
image
In order to implement unit testing on the principle “only one module / function is tested at a time”, it is practically necessary to actively use mock-objects.

For example, the following method of the User module is the implementation of the business logic for providing discounts on purchase:
sub apply_discount {
    my ($self, $shopping_cart) = @_;
    if ($shopping_cart->total_amount >= Discounts::MIN_AMOUNT
            && Discounts::is_discount_date) {
        if ($shopping_cart->items_count > 10) {
            $self->set_discount(DISCOUNT_BIG);
        }
        else {
            $self->set_discount(DISCOUNT_MINI);
        }
    }
}

One of the options for testing it could be this: creating a User ($ self) object with all the dependencies, creating a basket with the right amount of goods and with the right amount and testing the result.

In the case of a unit test, only this section of the code is tested, while creating the User and Shopping cart is avoided.

The test (for one “if” branch) looks something like this:
describe discount => sub {
    it "should work" => sub {
        my $user = bless {}, 'User';
        my $shopping_cart = mock();
        $shopping_cart->expects('total_amount')->returns(4_000)->once;
        Discounts->expects('is_discount_date')->returns(1)->once;
        $shopping_cart->expects('items_count')->returns(11);
        $user->expects('set_discount')->with(Discounts::DISCOUNT_BIG);
        ok $user->apply_discount($shopping_cart);
    };
};

The functions used here are Test :: Spec :: Mocks: expects , returns , with , once .

The following happens: the User :: apply_discount method is called, the mock object $ shopping_cart is passed to it. It is checked that the total_amount method of the $ shopping_cart object is called exactly once (in fact, no real code will be called - instead, this method will return the number 4000). Similarly, the class method Discounts :: is_discount_date should be called once, and will return one. The items_count method of the $ shopping_cart object is called at least once and returns 11. Finally, $ user-> set_discount should be called with the Discounts :: DISCOUNT_BIG argument

That is, in fact, we most naturally check each branch of the logic.

This approach gives us the following advantages:

  1. The test is easier to write.
  2. It is less fragile: if we completely tried to recreate the User object in the test, we would have to deal with breakdowns associated with the fact that the details of the implementation of something that was not used at all in the tested function changed.
  3. The test works faster.
  4. The business logic is more clearly stated (documented) in the test.
  5. If the bug is in the code, then not 100500 different tests fall, but some one, and from it you can definitely understand what exactly is violated.

If the equivalent unit test had to be written in pure Perl and Test :: More, it would look something like this:
use strict;
use warnings;
use Test::More;
my $user = bless {}, 'User';
my $shopping_cart = bless {}, 'ShoppingCart';
no warnings 'redefine', 'once';
my $sc_called = 0;
local *ShoppingCart::total_amount = sub { $sc_called++; 4_000 };
my $idd_called = 0;
local *Discounts::is_discount_date = sub { $idd_called++; 1 };
my $sc2_called = 0;
local *ShoppingCart::items_count = sub { $sc2_called++; 11 };
my $sd_called = 0;
local *User::set_discount = sub {
    my ($self, $amount) = @_;
    is $amount, Discounts::DISCOUNT_BIG;
    $sd_called = 1;
};
ok $user->apply_discount($shopping_cart);
is $sc_called, 1;
is $idd_called, 1;
ok $sc2_called;
is $sd_called, 1;
done_testing;

Here it is obvious that there is a lot of routine work on faking functions that could be automated.


More minor utility


Clear exception output


use Test::Spec;
describe "mycode" => sub {
    it "should work" => sub {
        is 1+1, 2;
    };
    it "should work great" => sub {
        die "WAT? Unexpected error";
        is 2+2, 4;
    };
};
runtests unless caller;

produces:
ok 1 - mycode should work
not ok 2 - mycode should work great
# Failed test 'mycode should work great' by dying:
# WAT? Unexpected error
# at test.t line 8.
1..2
# Looks like you failed 1 test of 2.
what contains, besides the line number, the name of the test - “mycode should work great”. The naked Test :: More cannot and cannot boast of this, since the name of the test is not yet known while preparations are being made for it.

Automatically imports strict / warnings;


That is, in fact, writing them is not necessary. But be careful if you have adopted a different code requirements module, such as Modern :: Perl. In this case, include it after Test :: Spec.

Simple and convenient custom test run


By simply setting the environment variable SPEC = pattern on the command line, you can run only a few tests. Which is extremely convenient when you are debugging one test and you do not need screen output from the rest.

Example:
use Test::Spec;
describe "mycode" => sub {
    it "should add" => sub {
        is 1+1, 2;
    };
    it "should substract" => sub {
        is 4-2, 2;
    };
};
runtests unless caller;

If you run it as SPEC = add perl test.t, then only the “mycode should add” test will be executed.

More details: https://metacpan.org/pod/Test::Spec#runtests-patterns .


No alternatives visible


Modules that allow structured organization of test code, like RSpec, of course, exist. But the alternatives, in terms of working with mock-objects, are not visible.

imageThe creator of the Test :: MockObject module is Chromatic https://metacpan.org/author/CHROMATIC (author of Modern Perl, participated in the development of Perl 5, Perl 6 and many popular modules on CPAN), does not recognize unit testing in the documentation for mock objects are described as “Test :: MockObject - Perl extension for emulating troublesome interfaces” (the keyword troublesome interfaces), which he even wrote a post about: http://modernperlbooks.com/mt/2012/04/mock-objects -despoil-your-tests.html

His approach is clearly not for us.

He also noted: “Note: See Test :: MockModule for an alternate (and better) approach.”

Test :: MockModule is extremely inconvenient, not supported (the author has not been seen since 2005) and broken in perl 5.21 ( https://rt.cpan.org/Ticket/Display.html?id=87004 )


Features of work and rake


Displaying test names in ok / is / others does not work


More precisely, it works, but it spoils the logic of forming test names in Test :: Spec.
describe "Our great code" => sub {
    it "should work" => sub {
        is 2+2, 4;
    };
};

outputs:
ok 1 - Our great code should work
1..1

and the code:
describe "Our great code" => sub {
    it "should work" => sub {
        is 2+2, 4, "should add right";
    };
};

Outputs:
ok 1 - should add right
1..1

As you can see, “Our great code” was lost, which negates the use of text in describe / it.

It turns out that the messages in ok and is better not to use.

But what if we want two tests in the it block?

describe "Our great code" => sub {
    it "should work" => sub {
        is 2+2, 4;
        is 10-2, 8;
    };
};

will output:
ok 1 - Our great code should work
ok 2 - Our great code should work
1..2

As you can see, there are no individual messages for each test. If you carefully look at the examples in the documentation of Test :: Spec, you can see that each individual test should be in a separate it:
describe "Our great code" => sub {
    it "should add right" => sub {
        is 2+2, 4;
    };
    it "should substract right" => sub {
        is 10-2, 8;
    };
};

will output:
ok 1 - Our great code should add right
ok 2 - Our great code should substract right
1..2

Which, however, is not very convenient and cumbersome for some cases.

There are problems with other modules made for Test :: More, for example, https://metacpan.org/pod/Test::Exception by default sets an automatically generated message for ok, respectively, you must explicitly specify an empty string instead.

You cannot place tests inside before / after


You will have to use the before block very often, it will initialize the variables before the tests. The after block is mainly needed to undo changes made in the outside world, including global variables, etc.

You do not need to try to place the tests themselves, which should be in it. For instance:
use Test::Spec;
describe "mycode" => sub {
    my $s;
    before each => sub {
        $s = stub(mycode=>sub{42});
    };
    after each => sub {
        is $s->mycode, 42;
    };
    it "should work" => sub {
        is $s->mycode, 42;
    };
};
runtests unless caller;

It produces an error:
ok 1 - mycode should work
not ok 2 - mycode should work
# Failed test 'mycode should work' by dying:
# Can't locate object method "mycode" via package "Test::Spec::Mocks::MockObject"
# at test.t line 9.
1..2
# Looks like you failed 1 test of 2.

As you can see, in the after block, the mock object created in the before block no longer works. So, if you have a lot of it blocks, and at the end of each block you want to carry out the same tests, then putting them into the after block will fail. You can put them in a separate function, and call it from each it, but this already looks like duplication of functionality.

image

Before / after blocks change the structure of the code


In the example below, we need to initialize a new Counter object for each test (let's imagine that it is complicated and takes a lot of lines of code, so copy / paste is not an option). It will look like this:
use Test::Spec;
use Counter;
describe "counter" => sub {
    my $c;
    before each => sub {
        $c = Counter->new();
    };
    it "should calc average" => sub {
        $c->add(2);
        $c->add(4);
        is $c->avg, 3;
    };
    it "should calc sum" => sub {
        $c->add(2);
        $c->add(4);
        is $c->avg, 3;
    };
};
runtests unless caller;

That is, the lexical variable $ c is used, which will be available in the scope of the “it” blocks. Before each of them, the before block is called, and the variable is reinitialized.

If you write a similar test without Test :: Spec, it turns out like this:
use strict;
use warnings;
use Test::More;
use Counter;
sub test_case(&) {
    my ($callback) = @_;
    my $c = Counter->new();
    $callback->($c);
}
test_case {
    my ($c) = @_;
    $c->add(2);
    $c->add(4);
    is $c->avg, 3, "should calc average";
};
test_case {
    my ($c) = @_;
    $c->add(2);
    $c->add(4);
    is $c->sum, 6, "should calc sum";
};
done_testing;

That is, a callback is passed to the test_case function, then test_case creates a Counter object and calls a callback, passing the created object as a parameter.

In principle, in Test :: More you can organize a test as your heart desires, but the example above is a universal, scalable solution.

If you try to make tracing paper with Test :: Spec, a lexical variable that is initialized before each test, you get something “not very correct”:
use strict;
use warnings;
use Test::More;
use Counter;
my $c;
sub init {
    $c = Counter->new();
}
init();
$c->add(2);
$c->add(4);
is $c->avg, 3, "should calc average";
init();
$c->add(2);
$c->add(4);
is $c->sum, 6, "should calc sum";
done_testing;

In this code, the function modifies a variable that is not passed to it as an argument, which is already considered a bad style. However, technically, this is the same as in the version with Test :: Spec (there, too, the code in the before block modifies a variable that was not passed to it explicitly), but in it it is considered “normal”.

We see that in Test :: More and Test :: Spec the code is organized differently. Different language features are used to organize the work of the test.

The local statement no longer works


More precisely, it works, but not always.

This does not work:
use Test::Spec;
our $_somevar = 11;
describe "foo" => sub {
    local $_somevar = 42;
    it "should work" => sub {
        is $_somevar, 42;
    };
};
runtests unless caller;

not ok 1 - foo should work
# Failed test 'foo should work'
# at test-local-doesnt-work.t line 8.
# got: '11'
# expected: '42'
1..1
# Looks like you failed 1 test of 1.

So - it works:
use Test::Spec;
our $_somevar = 11;
describe "foo" => sub {
    it "should work" => sub {
        local $_somevar = 42;
        is $_somevar, 42;
    };
};
runtests unless caller;

ok 1 - foo should work
1..1

The thing is that it does not fulfill the callback passed to it (or rather, this can already be considered a closure), but remembers the link to it. It is executed during a call to runtests. And as we know, local, unlike my, acts “in time”, and not “in space”.

What problems can this cause? local in tests may be needed for two things - fake a function and fake a variable. Now this is not so easy.

In principle, the fact that it is impossible to fake a function using local (and without it it is not practical - you have to manually return the old function back) is only good. Test :: Spec has its own mechanism for faking functions (it was mentioned above), and the other is not worth supporting.

But the impossibility of resetting a variable is worse.

If you do not use local in Perl right now, this does not mean that you will not need it in the tests. In the next three paragraphs I will explain why it may be needed.

DSL


The fact is that DSL ( http://www.slideshare.net/mayperl/dsl-perl ) in Perl is very often done using local variables.

For example, we need to receive data from a database in a Web application, in controllers. In this case, we have configured master / slave replication. By default, data must be received from slave servers, but in case we are going to modify the received data and write it to the database, the initial data must be received from the master server before modification.

Thus, we need to transfer information to all of our functions to receive data from the database: to take data from the slave server or from master. You can just pass them a database connection, but this is too cumbersome - there can be many such functions, they can call each other.

Suppose the code for obtaining data from the database is as follows:
sub get_data {
    mydatabase->query("select * from ... ");
}

Then we can do the following API: mydatabase will return the connection to the database slave, mydatabase inside the with_mysql_master block will return the connection to the master database.

This is how reading data from slave looks like:
$some_data = get_data();
$even_more_data = mydatabase->query("select * from anothertable … ");

This is how reading data from master and writing to master look like:
with_mysql_master {
    $some_data = get_data();
    mydatabase->query("insert into … ", $some_data);
};

The with_mysql_master function is easiest to implement with local:
our $_current_db = get_slave_db_handle();
sub mydatabase { $_current_db }
sub with_mysql_master(&) {
    my ($cb) = @_;
    local $_current_db = get_master_db_handle();
    $cb->();
}

Thus, mydatabase inside the with_mysql_master block will return a connection to the master database, since it is in the "range" of the local override of $ _current_db, and outside this block is a connection to the database slave.

So, in Test :: Spec with all similar constructions there can be difficulties.

Test :: Spec is made in the image of Ruby libraries, there DSL is organized without local (and there is no analog of local there at all), so this nuance was not provided.

Global caches


Look for “state” in your code. Any use of it can usually be classified as a global cache of something. When they say that global variables are bad, this often applies to such a “non-global” state.

The problem with state is that it cannot be tested at all (see http://perlmonks.org/?node_id=1072981 ). You cannot call a function from one process many times where something is cached using state and flush caches. We'll have to replace state with the good old our. And just when testing, reset it:
local %SomeModule::SomeData;

If you need to test such a function with Test :: Spec, and the cache will interfere, you can replace it with two separate functions - the first returns data without caching (say, get_data), the second only deals with caching (cached_get_data). And test only the first of them. This will be a unit test (tests one function separately). The second of these functions cannot be tested at all, but this is not particularly necessary: ​​it is simple - you have to believe that it works.

If you have an integration test that tests a whole call stack, then you have to fake a cached_get_data call in it and replace it with get_data without caching.

Global variables


Some% SomeModule :: CONFIG is a completely normal use-case for using global variables. Using local is convenient to replace the config before calling functions.

If there is a problem with Test :: Spec, it is better to make a function that returns CONFIG and fake it.

How else?


It should be noted that there are modules in which the same structural description of the tests is available (even with the same “describe” and “it”), but without this problem with local, for example https://metacpan.org/pod/Test :: Kantan . However, this module, in addition to the structural description of the tests, does not provide any capabilities.

More about it and local


At the beginning of the article, we found out that in each “it” there should be one test. This was originally conceived, and only this way it works normally. What to do if we have a whole cycle, where in each iteration of the test?

The correct way to do this is supposed to be:
use Test::Spec;
describe "foo" => sub {
    for my $i (1..7) {
        my $n = $i + 10;
        it "should work" => sub {
            is $n, $i + 10;
        };
    }
};
runtests unless caller;

But since each “it” only remembers the closure, but does not execute it right away, local code can no longer be used in this code, such a test completely fails:
use Test::Spec;
our $_somevar = 11;
describe "foo" => sub {
    local $_somevar = 42;
    for my $i (1..7) {
        my $n = $i + $_somevar;
        it "should work" => sub {
            is $n, $i + $_somevar;
        };
    }
};
runtests unless caller;

not ok 1 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '43'
# expected: '12'
not ok 2 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '44'
# expected: '13'
not ok 3 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '45'
# expected: '14'
not ok 4 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '46'
# expected: '15'
not ok 5 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '47'
# expected: '16'
not ok 6 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '48'
# expected: '17'
not ok 7 - foo should work
# Failed test 'foo should work'
# at t6-local.pl line 9.
# got: '49'
# expected: '18'
1..7
# Looks like you failed 7 tests of 7.

Yes, and each “describe” also remembers the closure, and does not perform it, so everything that applies to it is the same as “it”.

Common code


There is a mechanism for connecting common code that can be run in different tests. Here is the documentation: https://metacpan.org/pod/Test::Spec#spec_helper-FILESPEC , and here is the implementation: https://metacpan.org/source/PHILIP/Test-Spec-0.47/lib/Test/Spec. pm # L354 .

How does he work?

  1. Searches for the included file on disk using File :: Spec (bypassing the perl @ INC mechanism and the require file upload mechanism).
  2. It loads the file into memory, compiles a line with perl code, where the package first changes, then the contents of the read file “as is” are simply included.
  3. Executes this line as eval EXPR.
  4. The downloaded file itself has the extension .pl, it works, but it may not be a valid perl file, it may lack use, it may contain the wrong paths and so on, respectively, from the point of view of perl there are syntax errors. In general, this is a broken piece of code that needs to be stored in a separate file.

That is - an absolute hack.

However, it is quite possible to write a common code in the usual way - format it as a function, put it into modules:

Test code:
package User;
use strict;
use warnings;
sub home_page {
    my ($self) = @_;
    "http://www.example.com/".$self->login;
}
sub id {
    my ($self) = @_;
    my $id = $self->login;
    $id =~ s/\-/_/g;
    $id;
}
1;

Our module with common code for tests:
package MyTestHelper;
use strict;
use warnings;
use Test::Spec;
sub fake_user {
    my ($login) = @_;
    my $user = bless {}, 'User';
    $user->expects("login")->returns($login);
    $user;
}
1;

Test itself:
use Test::Spec;
use User;
use MyTestHelper;
describe user  => sub {
    it "login should work" => sub {
        my $user = MyTestHelper::fake_user('abc');
        is $user->home_page, 'http://www.example.com/abc';
    };
    it "should work" => sub {
        my $user = MyTestHelper::fake_user('hello-world');
        is $user->id, 'hello_world';
    };
};
runtests unless caller;

The fake_user function creates a User object, while faking the login method of this object so that it returns the login that we now want (also passed to fake_user). In tests, we check the logic of the User :: home_page and User :: id methods (knowing the login, we know that we must return these methods). Thus, the fake_user function is an example of reusing code to create a User object and configure fake methods.

It's hard to write helpers working simultaneously with Test :: Spec and Test :: More


As you can see, the test build order for Test :: Spec and Test :: More is very different. Usually, we can’t write a library that works in both test environments (we don’t take any tricks into account).

For example, we have a helper for Test :: More, which helps in the test to contact Redis. This is necessary for integration testing of code that works with this Redis, and is also convenient for some other tests (for example, tests with fork, where Redis is used to exchange test data between different processes).

This helper gives the following DSL:
redis_next_test $redis_connection => sub {
    ...
}

This function executes the code passed as the last argument. Inside the code, the namespace function is available. Inside each redis_next_test block, namespace is unique. It can and should be used to name Redis keys. At the end of the block, all keys with this prefix are deleted. All this is necessary so that the tests can be executed simultaneously with themselves on the CI server, and at the same time not spoil each other's keys, and also so as not to clutter up the developers' machines with unnecessary keys.

A simplified version of this helper:
package RedisUniqueKeysForTestMore;
use strict;
use warnings;
use Exporter 'import';
our @EXPORT = qw/
    namespace
    redis_next_test
/;
our $_namespace;
sub namespace() { $_namespace };
sub redis_next_test {
    my ($conn, $cb) = @_;
    local $_namespace = $$.rand();
    $cb->();
    my @all_keys = $conn->keys($_namespace."*");
    $conn->del(@all_keys) if @all_keys;
}
1;

An example of a test with it:
use strict;
use warnings;
use Test::More;
use RedisUniqueKeysForTestMore;
my $conn = connect_to_redis(); # external sub
redis_next_test $conn => sub {
    my $key = namespace();
    $conn->set($key, 42);
    is $conn->get($key), 42;
};
done_testing;

For Test :: Spec, this will no longer work, since:

  1. The concept of "inside redis_next_test" is quite naturally implemented using local, and with local in Test :: Spec problems, as we saw above.
  2. Even if redis_next_test didn't have local, and instead of local $ _namespace = $$. Rand () it would just be $ _namespace = $$. Rand () (which would make nested calls to redis_next_test impossible), it still wouldn't work, since $ conn-> del (@all_keys) if @all_keys; It would be executed not after the test, but after the callback of the test is added to the internal structures of Test :: Spec (in fact, the same story as with local).

A function that accepts a callback and executes it inside the describe block, with before (generates namespace) and after (removes keys) blocks, will do. Here she is:
package RedisUniqueKeysForTestSpec;
use strict;
use warnings;
use Test::Spec;
use Exporter 'import';
our @EXPORT = qw/
    describe_redis
    namespace
/;
my $_namespace;
sub namespace() { $_namespace };
sub describe_redis {
    my ($conn, $example_group) = @_;
    describe "in unique namespace" => sub {
        before each => sub {
            $_namespace = $$.rand();
        };
        after each => sub {
            my @all_keys = $conn->keys($_namespace."*");
            $conn->del(@all_keys) if @all_keys;
        };
        $example_group->();
    };
}

And so the test with her looks:
use Test::Spec;
use RedisUniqueKeysForTestSpec;
my $conn = connect_to_redis();
describe "Redis" => sub {
    describe_redis $conn => sub {
        it "should work" => sub {
            my $key = namespace();
            $conn->set($key, 42);
            is $conn->get($key), 42;
        };
    };
};
runtests unless caller;

The with function only works for classes.


Mymodule.pm
package MyModule;
use strict;
use warnings;
sub f2 { 1 };
sub f1 { f2(42); };
1;

Test:
use Test::Spec;
use MyModule;
describe "foo" => sub {
    it "should work with returns" => sub {
        MyModule->expects("f2")->returns(sub { is shift, 42});
        MyModule::f1();
    };
    it "should work with with" => sub {
        MyModule->expects("f2")->with(42);
        MyModule::f1();
    };
};
runtests unless caller;

Result:
ok 1 - foo should work with returns
not ok 2 - foo should work with with
# Failed test 'foo should work with with' by dying:
# Number of arguments don't match expectation
# at /usr/local/share/perl/5.14.2/Test/Spec/Mocks.pm line 434.
1..2
# Looks like you failed 1 test of 2.

Thus, it can be used only for working with class methods; if perl package is used not as a class, but as a module (procedural programming), this does not work. Test :: Spec simply waits for the first argument to $ self, always.

The with function does not see the difference between a hash and an array


MyClass.pm:
package MyClass;
use strict;
use warnings;
sub anotherfunc {
    1;
}
sub myfunc {
    my ($self, %h) = @_;
    $self->anotherfunc(%h);
}
1;

Test:
use Test::Spec;
use MyClass;
describe "foo" => sub {
    my $o = bless {}, 'MyClass';
    it "should work with with" => sub {
        MyClass->expects("anotherfunc")->with(a => 1, b => 2, c => 3);
        $o->myfunc(a => 1, b => 2, c => 3);
    };
};
runtests unless caller;

Result:
not ok 1 - foo should work with with
# Failed test 'foo should work with with' by dying:
# Expected argument in position 0 to be 'a', but it was 'c'
Expected argument in position 1 to be '1', but it was '3'
Expected argument in position 2 to be 'b', but it was 'a'
Expected argument in position 3 to be '2', but it was '1'
Expected argument in position 4 to be 'c', but it was 'b'
Expected argument in position 5 to be '3', but it was '2'
# at /usr/local/share/perl/5.14.2/Test/Spec/Mocks.pm line 434.
1..1
# Looks like you failed 1 test of 1.

Actually, Perl also does not see the difference. And the order of the elements in the hash is not defined. This could be taken into account when developing the with function API and to make a method that facilitates hash checking.

You can get around this flaw by using returns and checking the data in his callback. For the example above, this would be:
MyClass->expects("anotherfunc")->returns(sub { shift; cmp_deeply +{@_}, +{a => 1, b => 2, c => 3}  });


Problems testing things like memory leaks


imageFor example, the stub () function itself is a leak (apparently, stubs are stored somewhere). So this test doesn’t work:

MyModule.pm:
package MyModule;
sub myfunc {
    my ($data) = @_;
    ### Memory leak BUG
    #$data->{x} = $data;
    ### /Memory leak BUG
    $data;
}
1;

Test:
use Test::Spec;
use Scalar::Util qw( weaken );
use MyModule;
describe "foo" => sub {
    it "should not leak memory" => sub {
        my $leakdetector;
        {
            my $r = stub( service_id => 1 );
            MyModule::myfunc($r);
            $leakdetector = $r;
            weaken($leakdetector);
        }
        ok ! defined $leakdetector;
    }
};
runtests  unless caller;

This test shows a memory leak, even when it is not there.

A test written without stub works fine (it fails only if you uncomment a line with a bug in MyModule.pm):
use Test::Spec;
use Scalar::Util qw( weaken );
use MyModule;
describe "foo" => sub {
    it "should not leak memory" => sub {
        my $leakdetector;
        {
            my $r = bless { service_id => 1 }, "SomeClass";
            MyModule::myfunc($r);
            $leakdetector = $r;
            weaken($leakdetector);
        }
        ok ! defined $leakdetector;
    }
};
runtests  unless caller;

In any case, since “describe” and “it” remember closures, this in itself can interfere with the search for leaks, since a closure can contain links to all the variables that are used in it.

The use_ok function is no longer in place


If you used use_ok in tests before, now you can say goodbye to it. Judging by the documentation, it can only be used in the BEGIN block (see https://metacpan.org/pod/Test::More#use_ok ), and this is correct, since outside of BEGIN it may not work exactly as in reality ( for example, do not import function prototypes), and it makes no sense to use such a “correct” construction to test import from modules, violating this very import.

So, in Test :: Spec it’s not customary to write tests outside of “it”, but inside “it” the BEGIN block will execute ... as if it were outside of “it”.

So to do everything “beautifully and correctly” will not work, but if “beautifully and correctly” is not interested, then normal use will do.


Interesting


How technically faked objects using the expects method

image
Separately, it is worth noting how technically it is possible to achieve the overlap of the expects method for any object or class.

This is done by creating a method (surprise!) Expects in the code of the UNIVERSAL package.

Let's try to do the same trick:
package User;
use strict;
use warnings;
sub somecode {}
package main;
use strict;
use warnings;
{
    no warnings 'once';
    *UNIVERSAL::expects = sub { 
        print "Hello there [".join(',', @_)."]\n";
    };
}
User->expects(42);
my $u =bless { x => 123}, 'User';
$u->expects(11);

will output:
Hello there [User,42]
Hello there [User=HASH(0x8a6688),11]
that is, everything works - it was possible to shut down the method.


conclusions


Test :: Spec is good for unit testing high-level code


Test :: Spec is good for unit tests, that is, when only one “layer” is tested, and the rest of the function stack is faked.

For integration tests, when we are more interested in not quick, convenient and correct testing of a code unit and all border cases in it, but if everything works, and if everything is correctly “connected” - then Test :: More and its analogues are more suitable.

Another criterion is high-level vs low-level code. In high-level code, you often have to test business logic; mock objects are ideal for this. Everything except the logic itself is faked, the test becomes simple and straightforward.

For a low-level code, sometimes it makes no sense to write separately a “real” unit test, separately “integration”, since in a low-level code there is usually one “layer” and there is nothing to fake. The unit test will also be integration. Test :: More in these cases is preferable because in Test :: Spec there are things that are not very well transferred from the Ruby world, without taking into account the realities of Perl, and the methods of constructing the code change for no good reason.

Unit tests of high-level code are pretty similar, so for them the limitations and listed disadvantages of Test :: Spec are not a very big problem, and for low-level code and integration tests it is better to leave room for maneuver and use Test :: More.

This article was prepared with the active participation of the REG.RU development department. Special thanks to SFXfor numerous additions, akzhan for expertise and information from the world of Ruby and dmvaskin for three bugs found in the Test :: Spec module, as well as imagostorm , Chips , evostrov , TimurN , nugged , vadiml .
Tags:

Also popular now: