Git Product home page Git Product logo

testing-in-bash's Introduction

In search of the best test framework for Bash

There are many testing frameworks for Bash. However, many of them are not actively maintained, or are used only by a small group of people.

In this comparision I've included well-established and new testing frameworks that had at least recent release and have more than 50 Stars on GitHub:

  • bashunit, a fast, simple and flexible testing library for your bash scripts
  • Bats, a mature testing library with a bit of a custom syntax for more concise tests
  • shUnit2, possibly the oldest Bash test framework around, supporting test files in pure Bash
  • bash_unit, a new and featureful test framework where test cases can be written in pure Bash
  • ShellSpec, a BDD style testing framework with its own DSL
  • shpec, a BDD style testing framework similar to RSpec, Jasmine, and mocha

Test drive

To start fiddling with any of the test projects just run the install.sh to download and extract the particular testing framework to a local directory, then execute test.sh to start the suite.

Detailed comparision

bashunit Bats shUnit2 bash_unit ShellSpec shpec
Discover and run all tests
in the project
✔️ ✔️ ? ? ✔️ ? ✔️
Run subset of tests or
a specific test
✔️ ✔️ ? ? ✔️ ✔️ issue
Parallel test execution ✔️ ✔️
Skip or ignore tests ✔️ ✔️ ? ✔️ ? ✔️ issue
Rich assertion library ✔️ ? ? ? ✔️ ? ✔️ ?
Extensible with custom assertions ✔️ ✔️ ? ✔️ ? ✔️ ? ✔️ ? ✔️ ?
Before / After ✔️ ✔️ issue ✔️ ✔️ issue
BeforeAll / AfterAll ✔️ issue issue ✔️ ✔️ ? issue
Parameterized tests
and test data
✔️ issue ✔️
Mocking * issue ✔️ ? ? ✔️ ? ✔️ ? ✔️ ?
Isolation between tests ** ✔️ ✔️ doc ✔️ ✔️
Nice local report ✔️ ? ✔️ ✔️ ✔️ ✔️
CI compatible report ✔️ TAP issue ✔️ TAP ✔️ TAP
Zero Dependency and
implemented in Bash ***
✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Easy to install **** ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Tests written in Bash ✔️ almost ✔️ ✔️ ? ?
Compatible with the
Unofficial Bash Strict Mode *****
✔️ ? ✔️ issue ✔️
Maintained ✔️ ? ? ✔️ ? ✔️ ? ?
Mature since 2023 ✔️ since 2011 ✔️ since 2007 ✔️ since 2016 since 2019 ✔️ since 2013
Good documentation ✔️ ? ? ✔️ ? ✔️ ? ✔️ ? ✔️ ?

* Mocking:
The testing framework is compatible with the common mocking techniques.

** Isolation between tests:
Aliases, custom function definitions, Bash options and variables are not leaking from one test to another.

*** Zero Dependency and implemented in Bash:
Apart from a few common tools like coreutils and grep, you don't need anything to use the framework.
The source code of the framework is itself implemented as Bash scripts. If you are comfortable with Bash, you can read and even modify it.

**** Easy to install:
Many of the frameworks support more than one alternatives on how to set it up quickly. It's possible to install it with your favourite package manager and some of them even have an official Docker image. Personally, I try to avoid installing project dependencies globally as much as I can, so I just decided to grab a tar.gz file with the latest release from GitHub.

***** Compatible with the Unofficial Bash Strict Mode:
Sourcing a script file that use the Unofficial Bash Strict Mode should not break the test framework. Of course, custom assertions and the test code should also work with strict mode but that's up for the user. Also, the test framework shouldn't set Bash options as well to ensure that executing sourced functions work similarly to when they are not sourced.

Related posts, other comparisions

Contribution guide

If you know a testing framework that should be mentioned here or find that there's room for improvement feel free to open an issue or Pull Request. (See CONTRIBUTING.md.)

testing-in-bash's People

Contributors

antonio-gg-dev avatar chemaclass avatar dodie avatar heilig avatar ko1nksm avatar schorsch3000 avatar solt87 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

testing-in-bash's Issues

Add bashunit

There's a new library that has gotten over 50 stars ⭐ in its GiHub repository in its first week:

  • Click here to see the bashunit repository.

Its aim is to create a simple testing library for bash scripts, and I think it would make a good fit for this repository, so people could compare it with the others when making their choice. 🐱

Clarify standard for "Zero Dependency and implemented in Bash" section?

Nice comparison! I stumbled on this while searching around to look for a pure bash test framework.

I've done a little bit of preliminary packaging work on both bats and shunit2, and noticed that the Zero Dependency section didn't quite square with my experience--IIRC they both depend on at least coreutils and grep.

I don't mean to suggest that you need to census all of the frameworks for these--they probably will be available just about everywhere--but a little context about the baseline might help someone.

Use bats for docker utilitary functions

Hi, fellows. I am a software developer apprentice. In this journey, I learned docker is a great tool for apps and isolated images in containers. Since I am a Linux user, I use docker on shell.

I wrote some scripts with docker commands to both train my shell and docker skills here: https://github.com/web-needle/sappio/tree/main/src%2Fscripts .

It is kind of you in case you may help me to cover somr use cases. I wrote some easy use cases like wrong command run.

Thank you. :-)

Function mocking by export in shunit2

Your shunit2 description mentions that it can't be used to mock functions by export. In my experience it's working though.
Looks like alias in your unit test example is still in effect and break function export test.
This patch makes tests work for me.

diff --git a/example-shunit2/test/unit_test.sh b/example-shunit2/test/unit_test.sh
index 82176e0..38d64ed 100755
--- a/example-shunit2/test/unit_test.sh
+++ b/example-shunit2/test/unit_test.sh
@@ -20,12 +20,10 @@ function test_there_should_be_20_percent_discount_for_large_quantities_on_odd_da
 
   assertEquals 0 $?
   assertContains "${result}" "Total 320"
+  unalias getDay
 }
 
 function test_there_should_be_no_discount_even_for_large_quantities_on_even_days() {
-  # Mocking by exporting functions does not work for shunit2
-  startSkipping
-
   source src/ice_cream_price.sh
 
   # Mocking a function by exporting a function

And by the way, thanks for the great guide!

[Feature Request] shpec

Hi,
im using shpec for my bash-testing-purpose.
Would you be willing to either add that to your by your own or have me do that and accept a PR?

Thanks, schorsch

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.