jasonkarns / bats-mock Goto Github PK

View Code? Open in Web Editor NEW

80.0 3.0 35.0 139 KB

Mocking/stubbing library for BATS (Bash Automated Testing System)

License: MIT License

Shell 100.00%

bash bats mock shell stub testing

bats-mock's Introduction

bats-mock

Mocking/stubbing library for BATS (Bash Automated Testing System)

Installation

git

The recommended installation is with git-subtree. Assuming your project's bats tests are in test/:

# v1 can be any valid ref (tag, branch, commit) of the bats-mock repo
git subtree add --prefix test/helpers/bats-mock https://github.com/jasonkarns/bats-mock v1 --squash

# pull a future bats-mock release (say, v2) with:
# git subtree pull --prefix test/helpers/bats-mock https://github.com/jasonkarns/bats-mock v2 --squash

# bats-mock keeps a "major version" tag pinned to the highest corresponding tag

then in test/test_helper.bash:

load helpers/bats-mock/stub

npm

Also available as an npm module if you're into that sort of thing.

npm install --save-dev bats-mock

then in test/test_helper.bash:

load ../node_modules/bats-mock/stub

Usage

After loading bats-mock/stub you have two new functions defined:

stub: for creating new stubs, along with a plan with expected args and the results to return when called.
unstub: for cleaning up, and also verifying that the plan was fullfilled.

Stubbing

The stub function takes a program name as its first argument; any remaining arguments go into the stub plan.

Each plan line represents an expected invocation: a list of expected arguments followed by the command to execute, separated with a colon.

arg1 arg2 ... : only_run if args matched

The expected args (and the colon) is optional.

So, in order to stub date, we could use something like this in a test case (where format_date is the function under test, relying on data from the date command):

load helper

# this is the "code under test"
# it would normally be in another file
format_date() {
  date -r 222
}

setup() {
  _DATE_ARGS='-r 222'
  stub date \
      "${_DATE_ARGS} : echo 'I am stubbed!'" \
      "${_DATE_ARGS} : echo 'Wed Dec 31 18:03:42 CST 1969'"
}

teardown() {
  unstub date
}

@test "date format util formats date with expected arguments" {
  result="$(format_date)"
  [ "$result" == 'I am stubbed!' ]

  result="$(format_date)"
  [ "$result" == 'Wed Dec 31 18:03:42 CST 1969' ]
}

This verifies that format_date indeed called date using the args defined in ${_DATE_ARGS} (which can not be declared in the test-case with local), and made proper use of the output of it.

The plan is verified, one by one, as the calls come in. Finally, when the stub is removed with unstub, there is a final check to ensure there are no remaining un-met plans (which would indicated an expected invocation that did not occur).

Unstubbing

Once the test case is done, you should call unstub <program> in order to clean up the temporary files, and make a final check that all the plans have been met for the stub.

How it works

Under the covers, bats-mock uses three scripts to manage the stubbed programs/functions.

First, it is the command (or program) itself, which when the stub is created is placed in (or rather, the binstub script is sym-linked to) ${BATS_MOCK_BINDIR}/${program} (which is added to your PATH when loading the stub library). Secondly, it creates a stub plan, based on the arguments passed when creating the stub. And finally, during execution, the command invocations are tracked in a stub run file which is checked once the command is unstub'ed. The ${program}-stub-[plan|run] files are both in ${BATS_MOCK_TMPDIR}.

Caveat

If you stub functions, make sure to unset them, or the stub script wan't be called, as the function will shadow the binstub script on the PATH.

Credits

Extracted from the ruby-build test suite. Many thanks to its author and contributors: Sam Stephenson and Mislav Marohnić.

bats-mock's People

Contributors

Stargazers

Watchers

bats-mock's Issues

Print missing invocations on failure

Would be great if unstub would not only return 1 but also print the missing invocations to &3 for easier error fixing.

tests

Stubbing w/ more than 2 threads doesn't work

If you stub a function with 3 mocks e.g.

stub someFunc \
"echo 1 >> somefile.txt" \
"echo 2 >> somefile.txt" \
"echo 3 >> somefile.txt";
sut_dosomething_thatwillcall_someFunc;
unstub someFunc

This will work if [sut_dosomething_thatwillcall_someFunc] is single threated. However,
if [sut_dosomething_thatwillcall_someFunc] has 3 threads, each thread calls [someFunc] once, the 3rd mock will never be executed (the 2nd mock "echo 2..." gets executed repetitively).

New to bats, can anyone share a real world project using bats and bats-mock

Hello, I have some scripts that depend on hardware and hence the need for my to use mock/stubs. Can anyone please share some insight as to how to setup a bats project, do all stub files go into a certain folder?

Stub fails to recognise call when arguments contain an asterisk character

If the arguments string includes an asterisk then the bats-mock stub appears to fail to recognise the call.

Reduced test case:

#!/usr/bin/env bats

load '../vendor/bats-mock/stub'

teardown() {
  unstub aws
}

# fails
@test "test with asterisk" {
  stub aws \
    "ec2 describe-snapshots --query Snapshots[*].{Time:StartTime} : echo 'foo'"

  run aws ec2 describe-snapshots --query Snapshots[*].{Time:StartTime}
  [ "$output" = "foo" ]
}

# passes
@test "test without asterisk" {
  stub aws \
    "ec2 describe-snapshots --query Snapshots[].{Time:StartTime} : echo 'foo'"

  run aws ec2 describe-snapshots --query Snapshots[].{Time:StartTime}
  [ "$output" = "foo" ]
}

Output:

$ vendor/bats/bin/bats test/test_case.bats
 ✗ test with asterisk
   (in test file test/test_case.bats, line 14)
     `[ "$output" = "foo" ]' failed
 ✓ test without asterisk

2 tests, 1 failure

`stub` not concurrent-safe with parallel test execution (with `--jobs`)

It appears that a race to write stub test plans happens when you run bats with --jobs n set.

Example:

load "test_helpers/bats-mock/stub"
load "test_helpers/bats-support/load"
load "test_helpers/bats-assert/load"
# 1_stub_stat_succeeding.bats
@test "Stub stat" {
  stub stat "fake-dir: echo 'Yup' && exit 0"
  run ./example.sh
  assert_success
  unstub stat
}

# 2_stub_stat_failing.bats
load "test_helpers/bats-mock/stub"
load "test_helpers/bats-support/load"
load "test_helpers/bats-assert/load"
@test "Stub stat failing" {
  stub stat "fake-dir: echo 'Nope' && exit 1"
  run ./example.sh
  assert_failure
  unstub stat
}

# example.sh
stat fake-dir

When you run this with --bats jobs 8 (for example), one of these things happens:

A failure occurs when trying to unstub stat (since /tmp/stat-stub-plan will have two entries in its plan when there should only ever be one
One of these tests produces an invalid result (because both plans share the same arguments)

Sorry for the toy example. I hope this makes sense.

documentation

spy-like behavior

presently stubs behavior is supported (creating a plan); and mock behavior is supported (via unstub, which fails if the plan wasn't followed as expected). It would be nice to be able to make limited assertions against the test run, instead of requiring full mock verification

`read` fails to stub, halts test runner

When I attempt to run a test with a stub, the test runner gets stuck without throwing an error. It must be exited with CTRL + C

$ find -type f | entr ./node_modules/bats/bin/bats -r ./helpers/tests
 ✓ a/b matcher to ensure the asserts are working
 ✓ createConfigFile
 ✓ toLowercase
   gatherConfigOptions                    4/4

I need to stub read to allow automated testing to work. But this code is causing the halt:

# tests.bats
load "batsMock/stub" # saved via git submodule into this folder

@test "gatherConfigOptions" {
   # setup
   stub read \
      "echo 'returnValue1'" \
      "echo 'returnValue2'"

   # test execution & assert
   local options=$(gatherConfigOptions)
   assert_equal $options "wrongValue"

   # teardown
   unstub read
}

Also tried

Commenting out the unstub statement. No effect

I suspect that read is running unstubbed and causing the infinite delay. I could be wrong though.

Anyone able to tell what the issue is and how to overcome it?

Here is the function I am attempting to stub the internals of:

# ./helpers/helpers.sh

function gatherConfigOptions {
   declare -A configData

   read -p "prompt text here" "configData[languageCode]"
   read -p "prompt text here" "configData[numberOfRepeats]"

   echo "${configData[*]}"
}

cleanup function

presently, stub just appends to the stub-plan file(s). And typical teardown functions only cleanup TMP, not BATS_TMPDIR. It would be nice for stubs to have their own directory under BATS_TMPDIR. Or alternatively keep a list of stubs created and expose a teardown function to remove them (which could then be invoked from userland teardown)

Support for calling the same mocked call - argument multiple times

This will fail:

@test "test calling the same mocked program multiple times" { 
    stub curl \
          "-s -m 2 http://www.google.com : echo called_me" 
    run bash -c "test.sh" 
    unstub curl 
}

test.sh:

!#/bin/bash
for i in `seq 1 10`;
do
     MYVAR=$(curl -s -m 2 http://www.google.com)
     echo ${MYVAR}
done

The error:

 ✗ test calling the same mocked program multiple times
   (from function `unstub' in file ./helpers/mocks/stub.bash, line 34,
    in test file test-m.bats, line 16)
     `unstub curl' failed

Is there a way to support calling the same call multiple times?

Question: Advantage of bats-mock over stub.sh?

I am currently looking into unit-testing my BASH code. The bats-core project seems the most likely candidate to write tests in. As part of writing unit-test is mocking/stubbing, I went looking for which project offers the best functionality.

Besides this project I also came across https://github.com/jimeh/stub.sh, which looks like it has more functionality/features.

I was wondering, is there any advantage in using bats-mock over stub.sh?