Testing Bash Applications

Recently, I was faced with the task of testing an application written in Bash. Initially, I decided to use unit tests in Python, however, I did not want to add extra technology to the project. And I had to choose a test framework whose mother tongue is the long-suffering Bash.

Overview of existing solutions

When I turned to Google asking: what there is already a choice, the answer was not so many options. Here I will consider some of them.

What criteria will I pay attention to?

  1. Dependencies: if we take the test framework on Bash, then I would like it to not pull for itself: Python, Lua, and a couple more system packages (and there are some).
  2. The complexity of the installation: since one of the tasks was the deployment of continuous-development and continuous-integration in Travis , it was important for me that the installation could be done in a sane time and number of steps. Ideal options: package managers, acceptable: git clone, wget.
  3. Documentation and support: the application should work on different unix distributions, respectively, and the tests should work everywhere, taking into account the number of different platforms, shells, their combinations and the speed of updating them, I would not want to stay without the community and experience of other users.
  4. The presence of fixtures in any form and / or (at least!) setup()And teardown()functions.
  5. Sane syntax for writing new tests. In the world of Bash, a very important requirement.
  6. My usual conclusion about the results of the tests: how much has passed, what and where has fallen, in which line (preferably).


One of the first options that I noticed was a small framework assert.sh. Fairly good solution: easy to install, easy to use. In order to write the first tests you need to create a file tests.shand write just something into it (example from the documentation):

. assert.sh
# `echo test` is expected to write "test" on stdout
assert "echo test" "test"
# `seq 3` is expected to print "1", "2" and "3" on different lines
assert "seq 3" "1\n2\n3"
# exit code of `true` is expected to be 0
assert_raises "true"
# exit code of `false` is expected to be 1
assert_raises "false" 1
# end of test suite
assert_end examples

Then you can run the tests and see the results:

$ ./tests.sh
all 4 examples tests passed in 0.014s.

Of the advantages, you can further highlight:

  1. Ease of syntax and use.
  2. Good documentation, examples of use.
  3. The ability to do conditional or unconditional skip tests.
  4. Opportunity fail-fast or run-all.
  5. It is possible to make the error output detailed (if you use the flag -v), initially it does not say which tests fall.

There are several serious disadvantages:

  1. At the time of writing the article on github, the red "build failing" icon was on, it looks scary.
  2. The framework positions itself as light, it lacks methods for me setup()and teardown()so that it is possible to prepare the necessary data for each test and delete it at its completion.
  3. There is no way to run all test files from a specific folder.

Conclusion: a good tool that I would recommend using if you need to write a couple of simple tests for a script. For more serious tasks - not suitable.


Things shunit2are worse with the installation . I could not find an adequate repository: there is a certain project on Google.Code, there are several projects on github of various neglect (3 years and 5 years), there are even several svn repositories. Accordingly, it is unrealistic to understand which release is the latest and where to download it from. But then the little things. What do the tests themselves look like? Here is a slightly simplified example from the documentation :

  result=`expr 1 + 2`
  assertEquals \
      "the result of '${result}' was wrong" \
      3 "${result}"


$ /bin/bash math_test.sh
Ran 1 test.

This framework has a number of unique features in its class:

  1. The ability to create suites within the code, such a function can be useful, there are tests for specific platforms or shells. Then you can use your namespaces, like zsh_, debian_etc.
  2. There are functions setUpand tearDownthat are performed for each test, and another oneTimeSetUp, and oneTimeTearDownthat are performed at the beginning and end of the test.
  3. A wide selection of different ones assert, it is possible to display line numbers where the test falls using the design ${_ASSERT_EQUALS_}, but only in shells where line numbering is supported. From the documentation: bash(> = 3.0) ksh, pdkshand zsh.
  4. It is possible to skip tests.

But there are a number of significant drawbacks that pushed me as a result:

  1. There is no activity in the project, all the latest errors in Google.Code since 2012 hang without a solution, there have not been any commits to the repository for three years now. In general, trouble.
  2. It is not clear what and how to put, the last release was in 2011. Associated with the last paragraph.
  3. The number of functions is even slightly redundant, so there are two ways to check equality: assertEqualsand assertSame. A trifle, but surprising.
  4. There is no way to run all files from a folder.

Conclusion: a serious tool that can be flexibly configured and turned into an indispensable part of the project, but the lack of a clear system for managing the project itself scares shunit2. I decided to search further.


I was initially interested in this framework, because it was written by the author Sinatrafor Ruby. I also liked the test syntax, which resembles the familiar and familiar Mocha. By default, all functions that start with the it_inside of the file are launched . Interestingly, all tests are run inside their own sandbox, which helps prevent unnecessary errors. And here are the tests themselves, an example from the documentation:

describe "roundup(5)"
before() {
after() {
    rm -f foo.txt
it_runs_before() {
    test "$foo" "=" "bar"

There are no examples of the conclusion to look - you need to put and check, bad. Here are the advantages:

  1. Each test runs inside its sandbox, which is very convenient.
  2. Easy to use.
  3. Installation through git cloneand ./configure && make, can be installed in a local directory with the addition of $PATH.

And there were enough minuses:

  1. There is no way to do sourcesome common functions for all tests, but in fairness it’s worth saying that with the help of a hack - you can.
  2. There is no way to run all test files from a folder.
  3. Documentation is replete TODO, but work has not been carried out for a couple of years.
  4. You can not miss the test.

Conclusion: such an average thing is absolutely average, one cannot say that it is bad. But you can’t call her good either. The functionality is similar to assert.sh, only a little more. Where to use? If there is enough functionality assert.sh, but you need a function before()or after().


I have to say right away, I chose this framework. Liked a lot. First of all - excellent documentation: examples of use, semantic versioning, separately pleased with the list of projects that use bats.

batsuses the following approach: a test is considered passed if all the commands inside it return a code 0(how set -e). That is, each line is a truth check. Here's what the tests written in bats:

#!/usr/bin/env bats
@test "addition using bc" {
  result="$(echo 2+2 | bc)"
  [ "$result" -eq 4 ]
@test "addition using dc" {
  result="$(echo 2 2+p | dc)"
  [ "$result" -eq 4 ]

And the conclusion:

$ bats addition.bats
 ✓ addition using bc
 ✓ addition using dc
2 tests, 0 failures

The output of information about tests using the flag ( --tap) can be represented in the form of text compatible with Test Anything Protocol, for which there are plugins for more programs: Jenkins, Redmine and others.

In batsaddition to the special syntax for writing a test, there are many interesting things:

  • The command runallows you to run the command and then test its output code and text output: for which there are special variables: $statusand$output
  • The command loadallows you to load a common code base for use.
  • The command skipallows you to skip the test if necessary.
  • Functions setup()and teardown()allow you to customize the environment and clean up for yourself.
  • There are a number of special environment variables .
  • It is possible to run all test files inside a folder.
  • Active community.

Objectively, there are batsa lot of pluses , and I have already listed them, but minus I could notice only one:

  • batsdeparts from the valid bash. Tests must be written in files with permission .bats, use another shebang.

Conclusion: a quality tool, with virtually no weaknesses. I advise to use.


If you are interested to see what happened in the end, here is a link to the tests for my free-time project git-secret.

Also popular now: