Testing in Java. Spock framework

  • Tutorial

In previous articles on the examples of JUnit and TestNG, I mentioned test-driven development (TDD) and data-driven testing (DDT) . But there is another approach that is actively gaining popularity, behavior-driven development (BDD) . This is such a development of TDD technology, in which the test is viewed not as testing some system components, but as functional requirements. If a TDD uses concepts such as a test or a method, then for BDD it is a specification and requirements. About this technique already spoke on a habr earlier:

This approach is applicable using both JUnit and TestNG. But there are other tools tailored specifically for BDD. In this article I will talk about such a framework. It is called the Spock Framework and combines not only the principles of BDD, but also the merits of Groovy . Yes, it’s Groovy. Although Groovy is used, it is also used to test Java code. Examples of use are Spring, Grails, Tapestry5. Interesting? Then read on.


Behavior-driven development


So, let me remind you what it is. Consider an example. Is there a utility that works with ant patterns (these are the ones for fetching files).? - any 1 and only 1 character, * - any number of any characters, ** - any path. It looks something like this:

public abstract class PathUtils {
  public static boolean matchAntPath(final String path, final String pattern) {
    // ...
  }
  public static boolean matchAntPattern(final String path, final String pattern) {
    // ...
  }
}

Both methods check whether the passed string matches the pattern or not, but the matchAntPattern method only considers the local pattern without considering the path, matchAntPath takes the full path. Following the principles of TDD, a test for each method will be created with some set of input data and some set of expected results.

public class TestPathUtils extends Assert {
  @Test(dataProvider = "matchAntPatternData")
  public void testMatchAntPattern(final String pattern, final String text, final boolean expected) {
    final boolean actual = PathUtils.matchAntPattern(text, pattern);
    assertEquals(actual, expected);
  }
  @Test(dataProvider = "matchAntPathData")
  public void testMatchAntPath(final String pattern, final String path, final boolean expected) {
    final boolean actual = PathUtils.matchAntPath(path, pattern);
    assertEquals(actual, expected);
  }
}

Perhaps tests for incorrect parameters will also be added here when exceptions should be thrown. Now let's look at it from the point of view of BDD.
A test is not just a test, but a specification, and does not consist of methods, but of requirements. Highlight the requirements for PathUtils :
  • Symbol? in the template should be equivalent to any character in the string being checked
  • Symbol? in the template should be equivalent to 1 and only 1 character in the string being checked
  • The * character in the pattern must be equivalent to any character in the string being checked
  • The * character in the pattern must be equivalent to any number of characters in the string to be checked
  • The values ​​of the template and the string to be checked must not be null

Further, each requirement has its own verification script, usually the terms given-when-then are used for this. Given - settings for the start of the script, when - the reason, then - the condition for the script. For instance:

Given:
PathUtils
---
When:
matchAntPattern(null, "some string")
---
Then:
NullPointerException should be thrown

Thus, the test will look something like this:

public class PathUtilsSpec extends Assert {
  @Test
  public void question_character_should_mean_any_character() {
    assertTrue(PathUtils.matchAntPattern("abb", "a?b"));
    assertTrue(PathUtils.matchAntPattern("a1b", "a?b"));
    assertTrue(PathUtils.matchAntPattern("a@b", "a?b"));
    assertTrue(PathUtils.matchAntPath("abb", "a?b"));
    assertTrue(PathUtils.matchAntPath("a1b", "a?b"));
    assertTrue(PathUtils.matchAntPath("a@b", "a?b"));
    // ...
  }
  @Test
  public void question_character_should_mean_only_one_character() {
    assertFalse(PathUtils.matchAntPattern("ab", "a?b"));
    assertFalse(PathUtils.matchAntPattern("aabb", "a?b"));
    assertFalse(PathUtils.matchAntPath("ab", "a?b"));
    assertFalse(PathUtils.matchAntPath("aabb", "a?b"));
    // ...
  }
  @Test
  public void asterisk_character_should_mean_any_character() {
    assertTrue(PathUtils.matchAntPattern("abb", "a*b"));
    assertTrue(PathUtils.matchAntPattern("a1b", "a*b"));
    assertTrue(PathUtils.matchAntPattern("a@b", "a*b"));
    assertTrue(PathUtils.matchAntPath("abb", "a*b"));
    assertTrue(PathUtils.matchAntPath("a1b", "a*b"));
    assertTrue(PathUtils.matchAntPath("a@b", "a*b"));
    // ...
  }
  @Test
  public void asterisk_character_should_mean_any_number_of_characters() {
    assertTrue(PathUtils.matchAntPattern("ab", "a*b"));
    assertTrue(PathUtils.matchAntPattern("aabb", "a*b"));
    assertTrue(PathUtils.matchAntPath("ab", "a*b"));
    assertTrue(PathUtils.matchAntPath("aabb", "a*b"));
    // ...
  }
  @Test
  public void double_asterisk_character_should_mean_any_path() {
    assertTrue(PathUtils.matchAntPath("aaa/bbb", "aaa/**/bbb"));
    assertTrue(PathUtils.matchAntPath("aaa/ccc/bbb", "aaa/**/bbb"));
    assertTrue(PathUtils.matchAntPath("aaa/c/c/c/bbb", "aaa/**/bbb"));
    // ...
  }
}

Now more about the Spock Framework.

Key features


As I said, scripts are written in Groovy. Is this good or bad? Decide for yourself, beginners can read Groovy in 15 minutes - a quick overview .

The specification should be inherited from spock.lang.Specification . It may contain fields, fixture methods, feature scripts, helper methods.

Fields by default are not shared between scripts, i.e. field changes from one scenario will not be visible from another scenario. To share, you can annotate using @Shared .

Installation methods are:
  • setup () - analogue of @Before in JUnit, executed before each script
  • cleanup () - analogue of @After in JUnit, executed after each script
  • setupSpec () - an analogue of @BeforeClass in JUnit, executed before the first script in the specification
  • cleanupSpec () - an analogue of @AfterClass in JUnit, executed after the last script in the specification

As in other test frameworks, these methods are used in order not to write the same installation code for each script.

Requirement scenarios are the main part of the specification. This is where component behavior is described. It is customary to call them using string literals, and you can use any characters, the main thing is that this name as clearly as possible describes what this script does. For example, in our case:

class PathUtilsSpec extends Specification {
  def "? character should mean any character"() {
    // ...
  }
  def "? character should mean only one character"() {
    // ...
  }
  def "* character should mean any character"() {
    // ...
  }
  def "* character should mean any number of characters"() {
    // ...
  }
  def "** character should mean any path"() {
    // ...
  }
}

Each scenario consists of blocks, which are indicated by labels:
  • setup is the same as the setup () installation method , only applies to a specific scenario. Must be located before the remaining blocks and should not be repeated. The setup label may be missing, you can also write given instead of setup , this is done for better readability (given-when-then)
  • cleanup is the same as the cleanup () method , only applies to a specific script. Must be at the end of the script before the where block , if any, and should not be repeated
  • when-then is the reason and condition for execution. In the when part, variables are usually declared, the necessary actions are performed, in the part then some conditions are checked. This can be checking conditions, checking for throwing an exception, or waiting for some methods to execute on mock objects. This block can be repeated, but the authors of the framework recommend not to get involved, a good script should contain from 1 to 5 such blocks
  • expect is a simplified when-then block where action and validation are in the same expression
  • where is an analog of @DataProvider from TestNG, designed to create a dataset for a test

Now about everything in more detail. Consider another example. PathSearcher , designed to search files, uses ant patterns as a filter for files.

public class PathSearcher {
  public PathSearcher(final String path) {...}
  public PathSearcher include(final String... patterns) {...}
  public PathSearcher exclude(final String... patterns) {...}
  public Set search() {...}
}

We write the requirement “must search for files on the file system”:

class PathSearcherSpec extends Specification {
  def "it should search files under the file system"() {
    given:
    def searcher = PathSearcher.create(inClasspath("test1"))
    when:
    def results = searcher.search();
    then:
    results.containsAll(["1.txt", "2.txt"]);
    results.size() == 2
  }
  private String inClasspath(path) {
    return ClassLoader.getSystemResource(path).toExternalForm()
  }
}

So, it is given - a search engine that searches in the test1 folder from the classpath , check the search, execution condition - the search engine should find our files. inClasspath is a helper method that returns the absolute path of a file from classpath .

Another example for PathUtils "the values ​​of the template and the string to be checked must not be null"

class PathUtilsSpec extends Specification {
  def "null parameter values are not allowed"() {
    when:
    PathUtils.matchAntPattern(null, "some string")
    then:
    thrown(NullPointerException)
    when:
    PathUtils.matchAntPattern("some string", null)
    then:
    thrown(NullPointerException)
    when:
    PathUtils.matchAntPath(null, "some string")
    then:
    thrown(NullPointerException)
    when:
    PathUtils.matchAntPath("some string", null)
    then:
    thrown(NullPointerException)
  }
}

Here we see the thrown (...) method , this is the expectation of the specified exception, there is also the notThrown (...) method and noExceptionThrown () . They are to verify that a given / no exception is thrown. Also in the then part, there may be expectations of some methods for mock objects, but about them a little later. One more example:

class PathUtilsSpec extends Specification {
  def "? character should mean any character"() {
    expect:
    PathUtils.matchAntPattern("abb", "a?b")
    PathUtils.matchAntPattern("a1b", "a?b")
    PathUtils.matchAntPattern("a@b", "a?b")
    PathUtils.matchAntPath("abb", "a?b")
    PathUtils.matchAntPath("a1b", "a?b")
    PathUtils.matchAntPath("a@b", "a?b")
  }
}

As you can see from the example, if both when and then parts can be combined into one condition, it is more convenient to use the expect block . This scenario can be improved by making it parameterizable using the where block :

class PathUtilsSpec extends Specification {
  def "? character should mean any character"() {
    expect:
    PathUtils.matchAntPattern(text, pattern)
    PathUtils.matchAntPath(text, pattern)
    where:
    pattern | text
    "ab?"   | "abc"
    "ab?"   | "ab1"
    "ab?"   | "ab@"
    "a?b"   | "abb"
    "a?b"   | "a1b"
    "a?b"   | "a@b"
    "?ab"   | "aab"
    "?ab"   | "1ab"
    "?ab"   | "@ab"
  }
}

Or so:

class PathUtilsSpec extends Specification {
  def "? character should mean any character"() {
    expect:
    PathUtils.matchAntPattern(text, pattern)
    PathUtils.matchAntPath(text, pattern)
    where:
    pattern << ["ab?", "ab?", "ab?", "a?b", "a?b", "a?b", "?ab", "?ab", "?ab"]
    text    << ["abc", "ab1", "ab@", "abb", "a1b", "a@b", "aab", "1ab", "@ab"]
  }
}

Or so:

class PathUtilsSpec extends Specification {
  def "? character should mean any character"() {
    expect:
    PathUtils.matchAntPattern(text, pattern)
    PathUtils.matchAntPath(text, pattern)
    where:
    [pattern, text] << [
        ["ab?", "abc"],
        ["ab?", "ab1"],
        ["ab?", "ab@"],
        ["a?b", "abb"],
        ["a?b", "a1b"],
        ["a?b", "a@b"],
        ["?ab", "aab"],
        ["?ab", "1ab"],
        ["?ab", "@ab"]
    ]
  }
}

Or even like this:

class PathUtilsSpec extends Specification {
  def "? character should mean any character"() {
    expect:
    PathUtils.matchAntPattern(text, pattern)
    PathUtils.matchAntPath(text, pattern)
    where:
    [pattern, text] = sql.execute("select pattern, text from path_utils_test")
  }
}

I think everything is clear from the examples, so I will not focus on this. I just note that in the where block you cannot use fields that are not marked as @Shared .

Interactions


Among other things, the framework allows you to work with mock objects without additional dependencies. You can create mokas for interfaces and non-final classes. The creation looks like this:

    def dao1 = Mock(UserDAO)
    UserDAO dao2 = Mock()

You can override return values ​​or the methods of such objects themselves. Authors call this interactions.

    dao1.findAll() >> [
        new User(name: "test1", description: "Test User"),
        new User(name: "test2", description: "Test User"),
        new User(name: "test3", description: "Test User")
    ]
    dao2.findAll() >> { throw new UnsupportedOperationException() }

Interactions are local (defined in the then block) and global (defined elsewhere). Local are available only in the then block, global are available everywhere, starting from the point of their definition. Also for local interactions, you can specify their power, this is the expected number of method calls.

class UserCacheSpec extends Specification {
  def users = [
      new User(name: "test1", description: "Test User"),
      new User(name: "test2", description: "Test User"),
      new User(name: "test3", description: "Test User")
  ]
  def "dao should be used only once for all user searches until invalidated"() {
    setup:
    def dao = Mock(UserDAO)
    def cache = new UserCacheImpl(dao)
    when:
    cache.getUser("test1")
    cache.getUser("test2")
    cache.getUser("test3")
    cache.getUser("test4")
    then:
    1 * dao.findAll() >> users
  }
}

In this example, we create a mock for UserDAO and a real UserCache object using this mock (setup block). Then we search for several users by name ( when- block) and finally check that the findAll method that returns the prepared result is called only 1 time.
Describing interactions, you can use templates:

    1 * dao.findAll() >> users
    (1..4) * dao.findAll() >> users
    (2.._) * dao.findAll() >> users
    (_..4) * dao.findAll() >> users
    _.findAll() >> users
    dao./find.*/(_) >> users

Read more here .

Additional features


As you can see, the framework already has a lot of features. But like other frameworks, there is the possibility of expanding the functionality. Examples are built-in extensions:
  • @Timeout - sets the maximum timeout for the script, an analog of the timeout attribute of @Test from JUnit
  • @Ignore - disables the script, analogue of @Ignore from JUnit
  • @IgnoreRest - disables all script except annotated, useful if you need to check only 1 test
  • @FailsWith - sets the expected exception, analog of the expected attribute at @Test from JUnit
  • @Unroll - indicates that parameterized scripts should be specified as separate scripts for each iteration, here you can also specify a template for the requirement name, by default it is "#featureName [#iterationCount]"

class InternalExtensionsSpec extends Specification {
  @FailsWith(NumberFormatException)
  @Unroll("#featureName (#data)")
  def "integer parse method should throw exception for wrong parameters"() {
    Integer.parseInt(data)
    where:
    data << ["Hello, World!!!", "0x245", "1798237199878129387197238"]
  }
  @Ignore
  @Timeout(3)
  def "temporary disabled feature"() {
    setup:
    sleep(20000)
  }
}

Integrations with other frameworks are made in separate modules:
  • Spring - the specification is annotated using @ContextConfiguration (locations = "application_context_xml") and dependencies can be injected into fields using @Autowired

    @ContextConfiguration(locations = "context.xml")
    class SpringIntegrationSpec extends Specification {
      @Autowired
      String testSymbol
      def "test-symbol should be spring"() {
        expect:
        testSymbol == "spring"
      }
    }
    

  • Guice - the specification is annotated using @UseModules (guice_module_class) and dependencies can be injected into fields using @Inject

    public class GuiceModule extends AbstractModule {
      @Override
      protected void configure() {
        bind(String.class).annotatedWith(Names.named("test-symbol")).toInstance("guice");
      }
    }
    @UseModules(GuiceModule)
    class GuiceIntegrationSpec extends Specification {
      @Inject
      @Named("test-symbol")
      String testSymbol
      def "test-symbol should be guice"() {
        expect:
        testSymbol == "guice"
      }
    }
    

  • Tapestry - the specification is annotated using @SubModule (tapestry_module_class) and dependencies can be embedded into fields using the @Inject annotation

    public class TapestryModule {
      public void contributeApplicationDefaults(final MappedConfiguration configuration) {
        configuration.add("test-symbol", "tapestry");
      }
    }
    @SubModule(TapestryModule)
    class TapestryIntegrationSpec extends Specification {
      @Inject
      @Symbol("test-symbol")
      String testSymbol
      def "test-symbol should be tapestry"() {
        expect:
        testSymbol == "tapestry"
      }
    }
    


Well and most importantly, if you need your own functionality, you can add your own extensions. Key classes for expanding functionality:
  • IMethodInterceptor, IMethodInvocation - the first for proxying specification methods, allows you to add your code before and after the method is called, to simplify the work, you can use the AbstractMethodInterceptor class . The second is available from the first, serves to work with the original (proxied) method
  • IGlobalExtension - allows you to work with specification metadata ( SpecInfo ), here you can see metadata for any specification components (fields, installation methods, requirements scripts) and add your own interceptors to them
  • IAnnotationDrivenExtension - the same as the previous one, only simplifies the task, if our extension is tied to a specific annotation, you can use the AbstractAnnotationDrivenExtension class to simplify the work

To create your own extension, you need to create an IGlobalExtension or IAnnotationDrivenExtension descendant class , in which your IMethodInterceptor will most likely be added to the specification components , and finally , add the spi extension to META-INF / services / org.spockframework.runtime.extension.IGlobalExtension for IGlobalExtension , for IAnnotationDrivenExtension, our annotation needs to be annotated using @ExtensionAnnotation (extension_class) .
An example of an extension that runs a script a specified number of times:

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
@ExtensionAnnotation(RepeatExtension.class)
public @interface Repeat {
  int value() default 1;
}
public class RepeatExtension extends AbstractAnnotationDrivenExtension {
  @Override
  public void visitFeatureAnnotation(Repeat annotation, FeatureInfo feature) {
    feature.addInterceptor(new RepeatInterceptor(annotation.value()));
  }
}
public class RepeatInterceptor extends AbstractMethodInterceptor{
  private final int count;
  public RepeatInterceptor(int count) {
    this.count = count;
  }
  @Override
  public void interceptFeatureExecution(IMethodInvocation invocation) throws Throwable {
    for (int i = 0; i < count; i++) {
      invocation.proceed();
    }
  }
}


class CustomExtensionsSpec extends Specification {
  @Repeat(10)
  def "custom extension"() {
    expect:
    Integer.parseInt("123") == 123
  }
}


Running tests


Due to the fact that Spock tests are run using the JUnit launcher ( Sputnik ), they work fine under various IDEs (as the authors say, I checked only under the idea). You can also configure the launch of tests from ant, maven, gradle. All the necessary information about the settings can be found here .
I’ll add that for myself, I a little bit shamanized the configuration under maven, because proposed by the authors did not work under maven3. Here is my configuration option:

4.0.0com.exampletesting-example1.0-SNAPSHOTcom.exampletesting-spock1.0-SNAPSHOTjarTesting Spock Framework Example
    This is an example application that demonstrates Spock Framework usage.
  org.codehaus.groovygroovy-all${groovy-version}testorg.spockframeworkspock-core${spock.version}testsrc/test/groovysrc/test/resourcesorg.apache.maven.pluginsmaven-surefire-plugin**/*Spec.groovyorg.codehaus.gmavengmaven-plugin${gmaven-version}${gmaven-provider}testCompileorg.codehaus.groovygroovy-all${groovy-version}1.7.101.31.70.5-groovy-1.7


Conclusion


Despite the fact that I met this wonderful framework recently, I have practically no experience using it, I can say with confidence that it is not inferior in capabilities, and in some aspects even surpasses other frameworks. I really liked writing tests on Groovy, I liked writing tests guided by BDD. Therefore, I advise you to try.

Examples can be found here .

Literature



Also popular now: