Git Product home page Git Product logo

testing-in-bash's Introduction

In search of the best test framework for Bash

There are many testing frameworks for Bash. However, many of them are not actively maintained, or are used only by a small group of people.

In this comparision I've included well-established and new testing frameworks that had at least recent release and have more than 50 Stars on GitHub:

  • bashunit, a fast, simple and flexible testing library for your bash scripts
  • Bats, a mature testing library with a bit of a custom syntax for more concise tests
  • shUnit2, possibly the oldest Bash test framework around, supporting test files in pure Bash
  • bash_unit, a new and featureful test framework where test cases can be written in pure Bash
  • ShellSpec, a BDD style testing framework with its own DSL
  • shpec, a BDD style testing framework similar to RSpec, Jasmine, and mocha

Test drive

To start fiddling with any of the test projects just run the install.sh to download and extract the particular testing framework to a local directory, then execute test.sh to start the suite.

Detailed comparision

bashunit Bats shUnit2 bash_unit ShellSpec shpec
Discover and run all tests
in the project
✔️ ✔️ ? ? ✔️ ? ✔️
Run subset of tests or
a specific test
✔️ ✔️ ? ? ✔️ ✔️ issue
Parallel test execution ✔️ ✔️
Skip or ignore tests ✔️ ✔️ ? ✔️ ? ✔️ issue
Rich assertion library ✔️ ? ? ? ✔️ ? ✔️ ?
Extensible with custom assertions ✔️ ✔️ ? ✔️ ? ✔️ ? ✔️ ? ✔️ ?
Before / After ✔️ ✔️ issue ✔️ ✔️ issue
BeforeAll / AfterAll ✔️ issue issue ✔️ ✔️ ? issue
Parameterized tests
and test data
✔️ issue ✔️
Mocking * issue ✔️ ? ? ✔️ ? ✔️ ? ✔️ ?
Isolation between tests ** ✔️ ✔️ doc ✔️ ✔️
Nice local report ✔️ ? ✔️ ✔️ ✔️ ✔️
CI compatible report ✔️ TAP issue ✔️ TAP ✔️ TAP
Zero Dependency and
implemented in Bash ***
✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Easy to install **** ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Tests written in Bash ✔️ almost ✔️ ✔️ ? ?
Compatible with the
Unofficial Bash Strict Mode *****
✔️ ? ✔️ issue ✔️
Maintained ✔️ ? ? ✔️ ? ✔️ ? ?
Mature since 2023 ✔️ since 2011 ✔️ since 2007 ✔️ since 2016 since 2019 ✔️ since 2013
Good documentation ✔️ ? ? ✔️ ? ✔️ ? ✔️ ? ✔️ ?

* Mocking:
The testing framework is compatible with the common mocking techniques.

** Isolation between tests:
Aliases, custom function definitions, Bash options and variables are not leaking from one test to another.

*** Zero Dependency and implemented in Bash:
Apart from a few common tools like coreutils and grep, you don't need anything to use the framework.
The source code of the framework is itself implemented as Bash scripts. If you are comfortable with Bash, you can read and even modify it.

**** Easy to install:
Many of the frameworks support more than one alternatives on how to set it up quickly. It's possible to install it with your favourite package manager and some of them even have an official Docker image. Personally, I try to avoid installing project dependencies globally as much as I can, so I just decided to grab a tar.gz file with the latest release from GitHub.

***** Compatible with the Unofficial Bash Strict Mode:
Sourcing a script file that use the Unofficial Bash Strict Mode should not break the test framework. Of course, custom assertions and the test code should also work with strict mode but that's up for the user. Also, the test framework shouldn't set Bash options as well to ensure that executing sourced functions work similarly to when they are not sourced.

Related posts, other comparisions

Contribution guide

If you know a testing framework that should be mentioned here or find that there's room for improvement feel free to open an issue or Pull Request. (See CONTRIBUTING.md.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.