Testing Bash Applications

Overview of existing solutions

1. Dependencies: if we take the test framework on Bash, then I would like it to not pull for itself: Python, Lua, and a couple more system packages (and there are some).
2. The complexity of the installation: since one of the tasks was the deployment of continuous-development and continuous-integration in Travis , it was important for me that the installation could be done in a sane time and number of steps. Ideal options: package managers, acceptable: git clone, wget.
3. Documentation and support: the application should work on different unix distributions, respectively, and the tests should work everywhere, taking into account the number of different platforms, shells, their combinations and the speed of updating them, I would not want to stay without the community and experience of other users.
4. The presence of fixtures in any form and / or (at least!) setup()And teardown()functions.
5. Sane syntax for writing new tests. In the world of Bash, a very important requirement.
6. My usual conclusion about the results of the tests: how much has passed, what and where has fallen, in which line (preferably).

assert.sh

. assert.sh
# `echo test` is expected to write "test" on stdout
assert "echo test" "test"
# `seq 3` is expected to print "1", "2" and "3" on different lines
assert "seq 3" "1\n2\n3"
# exit code of `true` is expected to be 0
assert_raises "true"
# exit code of `false` is expected to be 1
assert_raises "false" 1
# end of test suite
assert_end examples

$ ./tests.sh
all 4 examples tests passed in 0.014s.

1. Ease of syntax and use.
2. Good documentation, examples of use.
3. The ability to do conditional or unconditional skip tests.
4. Opportunity fail-fast or run-all.
5. It is possible to make the error output detailed (if you use the flag -v), initially it does not say which tests fall.

1. At the time of writing the article on github, the red "build failing" icon was on, it looks scary.
2. The framework positions itself as light, it lacks methods for me setup()and teardown()so that it is possible to prepare the necessary data for each test and delete it at its completion.
3. There is no way to run all test files from a specific folder.

shunit2

testAdding()
{
  result=`expr 1 + 2`
  assertEquals \
      "the result of '${result}' was wrong" \
      3 "${result}"
}

$ /bin/bash math_test.sh
testAdding
Ran 1 test.
OK

1. The ability to create suites within the code, such a function can be useful, there are tests for specific platforms or shells. Then you can use your namespaces, like zsh_, debian_etc.
2. There are functions setUpand tearDownthat are performed for each test, and another oneTimeSetUp, and oneTimeTearDownthat are performed at the beginning and end of the test.
3. A wide selection of different ones assert, it is possible to display line numbers where the test falls using the design ${_ASSERT_EQUALS_}, but only in shells where line numbering is supported. From the documentation: bash(> = 3.0) ksh, pdkshand zsh.
4. It is possible to skip tests.

1. There is no activity in the project, all the latest errors in Google.Code since 2012 hang without a solution, there have not been any commits to the repository for three years now. In general, trouble.
2. It is not clear what and how to put, the last release was in 2011. Associated with the last paragraph.
3. The number of functions is even slightly redundant, so there are two ways to check equality: assertEqualsand assertSame. A trifle, but surprising.
4. There is no way to run all files from a folder.

roundup

describe "roundup(5)"
before() {
    foo="bar"
}
after() {
    rm -f foo.txt
}
it_runs_before() {
    test "$foo" "=" "bar"
}

1. Each test runs inside its sandbox, which is very convenient.
2. Easy to use.
3. Installation through git cloneand ./configure && make, can be installed in a local directory with the addition of $PATH.

1. There is no way to do sourcesome common functions for all tests, but in fairness it’s worth saying that with the help of a hack - you can.
2. There is no way to run all test files from a folder.
3. Documentation is replete TODO, but work has not been carried out for a couple of years.
4. You can not miss the test.

bats

#!/usr/bin/env bats
@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
}
@test "addition using dc" {
  result="$(echo 2 2+p | dc)"
  [ "$result" -eq 4 ]
}

$ bats addition.bats
 ✓ addition using bc
 ✓ addition using dc
2 tests, 0 failures

• The command runallows you to run the command and then test its output code and text output: for which there are special variables: $statusand$output
• The command loadallows you to load a common code base for use.
• The command skipallows you to skip the test if necessary.
• Functions setup()and teardown()allow you to customize the environment and clean up for yourself.
• There are a number of special environment variables .
• It is possible to run all test files inside a folder.
• Active community.

• batsdeparts from the valid bash. Tests must be written in files with permission .bats, use another shebang.

PS