cloudkick / whiskey Goto Github PK

Whiskey is a powerful test runner for Node.js applications and a process orchestration framework which makes running integration tests with a lot of service / process dependencies easier.

License: Apache License 2.0

JavaScript 92.86% CSS 2.08% Shell 5.06%

whiskey's People

Contributors

Stargazers

Watchers

whiskey's Issues

Coverage output of missed lines

Right now the coverage only output the number of lines tested and missed. Can you also make an option to output the lines of code actually missed. I think expresso does this.

Replace optparse-js with an option parser package from NPM

Possible candidates:

optimist
...

add test.disable()/test.ignore() ability

would be nice to have the ability within a setUp method to specify that the tests in this file should be ignored because of some environmental condition that would be checked and then asserted. Using test.ignore/disable would not effect the exit code, it would also provide the ability to specify why it was ignored, to be sent to stdout:

test.ignore('server was not running.')

the logs would print [IGNORED] in the same place you see [TIMEOUT] and [OK]

Coverage population should support other source types

This boils down to appending '.html' on multiple file types. Otherwise it is a pain getting mime right in webservers.

assert.response example

Hello! I'd really love to see an example of how assert.response works -- I'm trying to get it to work, but having zero luck. Even rudimentary documentation would really be appreciated.

Thanks!

Process Orchestration

It would be helpful if Whiskey could coordinate spawning of processes for test cases.

We already do this by hand in several projects, where its redundant and painful.

Be consistent with other test frameworks and run setUp and tearDown before and after every test

Be consistent with other test frameworks and run setUp and tearDown before and after every test instead of doing it once per test file.

Helper function for figuring out double callbacks

A helper functions which wraps an original callback and returns a new function. This function should throw an error if it's called more then X times.

Something like:

myFunc(1, 2, 3 assert.callCount(myCallback), 1));

Add docs with examples

Need to wait for @tripngroove to create a kick-ass documentation template first :)

'make test' doesn't pass

Hi. I have a pull request coming... and wanted to make sure that whiskey's own test suite still passed. However, it fails on the current HEAD, or at least looks like it does. I did the following (on Mac). Suggestions on how what best to do? Thanks.

$ git clone https://github.com/cloudkick/whiskey.git
...
$ cd whiskey/
$ npm install
[email protected] ./node_modules/sprintf
[email protected] ./node_modules/rimraf
[email protected] ./node_modules/async
[email protected] ./node_modules/magic-templates
$ make test
./test/run.sh
test-success.js
  test_two_plus_two_equals_four                                              [OK]
  test_true_equals_true                                                      [OK]
  tearDown                                                                   [OK]
---------------------------------------------------------------------------------
Ran 3 tests in 0.185s

Successes: 3
Failures: 0
Timeouts: 0

PASSED
test-failure.js
  test_false_is_not_equal_true                                             [FAIL]

Exception:
AssertionError: true == false
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-failure.js:19:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_one_is_not_equal_two                                                [FAIL]

Exception:
AssertionError: 2 == 1
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-failure.js:24:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:499:23
    at finishCallback (/Users/trentm/src/whiskey/lib/common.js:106:19)
    at finishFunc (/Users/trentm/src/whiskey/lib/common.js:123:5)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:139:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 2 tests in 0.086s

Successes: 0
Failures: 2
Timeouts: 0

FAILED
test-timeout.js
  test_success_1                                                             [OK]
  test_success_2                                                             [OK]
  test_success_3                                                             [OK]
  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 3 tests in 1.006s

Successes: 3
Failures: 0
Timeouts: 1

FAILED
test-timeout-blocking.js
  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 0 tests in 1.005s

Successes: 0
Failures: 0
Timeouts: 1

FAILED
test-setup-and-teardown.js
  setUp                                                                      [OK]
  test_throw                                                                 [OK]
  test_indexOf                                                               [OK]
  tearDown                                                                   [OK]
---------------------------------------------------------------------------------
Ran 4 tests in 0.136s

Successes: 4
Failures: 0
Timeouts: 0

PASSED
test-inexistent.js
  file_does_not_exist                                                      [FAIL]

Exception:
Error: Cannot find module 'test-inexistent'
    at Function._resolveFilename (module.js:320:11)
    at Function._load (module.js:266:25)
    at require (module.js:348:19)
    at /Users/trentm/src/whiskey/lib/common.js:266:29
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 1 tests in 0.085s

Successes: 0
Failures: 1
Timeouts: 0

FAILED
test-setup-fail.js
  setUp                                                                    [FAIL]

Exception:
AssertionError: true == false
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-setup-fail.js:19:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at /Users/trentm/src/whiskey/lib/common.js:302:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
    at /Users/trentm/src/whiskey/lib/common.js:290:7
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_1                                                                   [FAIL]

Exception:
AssertionError: true == false
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-setup-fail.js:24:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_2                                                                   [FAIL]

Exception:
AssertionError: true == false
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-setup-fail.js:29:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:499:23
    at finishCallback (/Users/trentm/src/whiskey/lib/common.js:106:19)
    at finishFunc (/Users/trentm/src/whiskey/lib/common.js:123:5)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:139:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 3 tests in 0.085s

Successes: 0
Failures: 3
Timeouts: 0

FAILED
test-success.js
  test_two_plus_two_equals_four                                              [OK]
  test_true_equals_true                                                      [OK]
  tearDown                                                                   [OK]
---------------------------------------------------------------------------------
Ran 3 tests in 0.184s

Successes: 3
Failures: 0
Timeouts: 0

PASSED
test-success.js
  test_two_plus_two_equals_four                                              [OK]
  test_true_equals_true                                                      [OK]
  tearDown                                                                   [OK]
test-failure.js
  test_false_is_not_equal_true                                             [FAIL]

Exception:
AssertionError: true == false
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-failure.js:19:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_one_is_not_equal_two                                                [FAIL]

Exception:
AssertionError: 2 == 1
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-failure.js:24:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:499:23
    at finishCallback (/Users/trentm/src/whiskey/lib/common.js:106:19)
    at finishFunc (/Users/trentm/src/whiskey/lib/common.js:123:5)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:139:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 5 tests in 0.282s

Successes: 3
Failures: 2
Timeouts: 0

FAILED
test-success.js
  test_two_plus_two_equals_four                                              [OK]
  test_true_equals_true                                                      [OK]
  tearDown                                                                   [OK]
---------------------------------------------------------------------------------
Ran 3 tests in 0.416s

Successes: 3
Failures: 0
Timeouts: 0

PASSED
test-uncaught.js
  test_uncaught_exception                                                  [FAIL]

Exception:
Error: Testing uncaught exception
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-uncaught.js:22:9)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_uncaught_exception_2                                                [FAIL]

Exception:
Error: failure!
    at Object.throwException (/Users/trentm/src/whiskey/example/some-file.js:19:11)
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-uncaught.js:28:12)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:499:23
    at finishCallback (/Users/trentm/src/whiskey/lib/common.js:106:19)
    at finishFunc (/Users/trentm/src/whiskey/lib/common.js:123:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_unknown_method                                                      [FAIL]

Exception:
TypeError: Object #<Object> has no method 'unknownMethod'
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-uncaught.js:34:12)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:499:23
    at finishCallback (/Users/trentm/src/whiskey/lib/common.js:106:19)
    at finishFunc (/Users/trentm/src/whiskey/lib/common.js:123:5)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:139:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_uncaught_nested_functions                                           [FAIL]

Exception:
Error: woho, failure!
    at /Users/trentm/src/whiskey/example/some-file.js:26:23
    at /Users/trentm/src/whiskey/example/some-file.js:29:13
    at /Users/trentm/src/whiskey/example/some-file.js:32:9
    at Object.throwNestedFunctions (/Users/trentm/src/whiskey/example/some-file.js:35:5)
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-uncaught.js:40:12)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test1_require_throws                                                     [FAIL]

Exception:
Error: timeout
    at Object.secondTimeout (/Users/trentm/src/whiskey/example/timeout-throws.js:3:15)
    at Timer.callback (timers.js:83:39)
----------------------------------------
    at Object.setTimeout
    at initSecondTimeout (/Users/trentm/src/whiskey/example/timeout-throws.js:2:5)
    at Object.firstTimeout (/Users/trentm/src/whiskey/example/timeout-throws.js:9:9)
    at Timer.callback (timers.js:83:39)
----------------------------------------
    at Object.setTimeout
    at onload (/Users/trentm/src/whiskey/example/timeout-throws.js:8:4)
    at Object.<anonymous> (/Users/trentm/src/whiskey/example/timeout-throws.js:13:1)
    at Module._compile (module.js:404:26)
    at Object..js (module.js:410:10)
    at Module.load (module.js:336:31)
    at Function._load (module.js:297:12)
    at require (module.js:348:19)
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-uncaught.js:46:3)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 5 tests in 0.390s

Successes: 0
Failures: 5
Timeouts: 0

FAILED
test-chdir.js
  test_chdir                                                               [FAIL]

Exception:
AssertionError: true == false
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-chdir.js:20:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 1 tests in 0.089s

Successes: 0
Failures: 1
Timeouts: 0

FAILED
test-chdir.js
  test_chdir                                                                 [OK]
---------------------------------------------------------------------------------
Ran 1 tests in 0.085s

Successes: 1
Failures: 0
Timeouts: 0

PASSED
test-init-function.js
  test_folder_exists                                                         [OK]
---------------------------------------------------------------------------------
Ran 1 tests in 0.087s

Successes: 1
Failures: 0
Timeouts: 0

PASSED
test-failure.js
  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 0 tests in 2.005s

Successes: 0
Failures: 0
Timeouts: 1

FAILED
test-setup-timeout.js
  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 0 tests in 1.006s

Successes: 0
Failures: 0
Timeouts: 1

FAILED
test-teardown-timeout.js
  test_one_equals_two                                                      [FAIL]

Exception:
AssertionError: 2 == 1
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-teardown-timeout.js:19:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 1 tests in 2.006s

Successes: 0
Failures: 1
Timeouts: 1

FAILED
test-custom-assert-functions.js
  test_assertThrow                                                         [FAIL]

Exception:
TypeError: Object #<Object> has no method 'assertThrow'
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-custom-assert-functions.js:2:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_assertNoop                                                          [FAIL]

Exception:
TypeError: Object #<Object> has no method 'assertNoop'
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-custom-assert-functions.js:6:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Object.process (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:571:27
    at onTestDone (/Users/trentm/src/whiskey/lib/common.js:234:5)
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:499:23
    at finishCallback (/Users/trentm/src/whiskey/lib/common.js:106:19)
    at finishFunc (/Users/trentm/src/whiskey/lib/common.js:123:5)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:139:5)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

---------------------------------------------------------------------------------
Ran 2 tests in 0.084s

Successes: 0
Failures: 2
Timeouts: 0

FAILED
test-custom-assert-functions.js
  test_assertThrow                                                         [FAIL]

Exception:
Error: Assert thrown
    at Object.assertThrow (/Users/trentm/src/whiskey/example/custom-assert-functions.js:2:9)
    at Test._testFunction (/Users/trentm/src/whiskey/example/test-custom-assert-functions.js:2:10)
    at Test.run (/Users/trentm/src/whiskey/lib/common.js:131:10)
    at taskFunc (/Users/trentm/src/whiskey/lib/common.js:312:14)
    at Array.<anonymous> (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:565:21)
    at EventEmitter._tickCallback (node.js:126:26)
----------------------------------------
    at EventEmitter.nextTick
    at Object.push (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:558:23)
    at /Users/trentm/src/whiskey/lib/common.js:327:15
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:129:25
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:160:17
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:458:34
----------------------------------------
    at EventEmitter.on
    at TestFile._getConnection (/Users/trentm/src/whiskey/lib/common.js:352:14)
    at /Users/trentm/src/whiskey/lib/common.js:240:12
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:453:21
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:158:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:118:13
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:134:9
    at /Users/trentm/src/whiskey/node_modules/async/lib/async.js:157:9
    at Object.mapSeries (/Users/trentm/src/whiskey/node_modules/async/lib/async.js:147:23)

  test_assertNoop                                                            [OK]
---------------------------------------------------------------------------------
Ran 2 tests in 0.085s

Successes: 1
Failures: 1
Timeouts: 0

FAILED
jscoverage not installed, skipping coverage tests
test-timeout-after-finish.js
  test_timeout_after_finish                                                  [OK]
  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 1 tests in 1.005s

Successes: 1
Failures: 0
Timeouts: 1

FAILED

test should fail if the test throws

this wil timeout, instead of failing:

exports["test failure"] = function(test, assert){
  throw "blaaaaah";
  assert.ok(true);
  test.finish();
}

While i agree that we should use assert instead of throw, it makes developing tests more of a pain in the ass.

get "TIMEOUT" when testing with cassandra?

my code is easy:

exports.testCassandra = function(test, assert) {
var cql = require('node-cassandra-cql');
var client = new cql.Client({hosts: ['127.0.0.1:9042'], keyspace: 'schema1'});
client.execute('SELECT user_id, email FROM users WHERE user_id=?', ['baby'] ,function(err, result) {
// baby's email is [email protected] in cassandra ;
assert.equal("[email protected]", result.rows[0].email);
});
test.finish();
};

when i run " whiskey -t testCassandra.js", display "OK " but "TIMEOUT".

Is there anything wrong with my configuration or my test case?

exit non-zero on bad usage

$ ./node_modules/.bin/whiskey Usage: whiskey [options] --tests "files" $ echo $? 0

When whiskey exits 0, tools like scons won't notice it has failed to run.

Add support for other coverage arguments

currently the only coverage arguments supported are --exclude and --encoding could you also support --no-instrument as this is different from --exclude. No instrument means that the code is loaded but not included in the coverage. Exclude means the code isn't loaded, so if you use exclude on a file that is required, it will break. One example is to not instrument files generated by code generators.

Replace sprintf dependency with built-in or sprintf-js

The owner of sprintf (https://github.com/maritz/node-sprintf) says that node-sprintf is no longer maintained. He suggests replacing your sprintf with either the built-in Javascript methods or use the sprintf-js package.

My issue with Whiskey's usage of sprintf is that its package.json file does not contain a license declaration. This is a problem for my legal department. I asked the maintainer to update the file as a separate issue, but it is unclear if he'll do it. He recommended that I create an issue with Whiskey asking the project to remove the dependency. Discussion here: maritz/node-sprintf#18

Whiskey doesn't report exceptions thrown in child processes that originate outside of a test.

These exceptions cause the test runner to time out, without providing a sensible error message.

Note to myself on how to implement (current brain-dump):

To fix, it looks like we need to insert code somewhere in lib/run_test_file.js close to lines 98-102, to catch any exception, stringify it, and send it along the IPC channel back to the test runner proper. Then, inside the test runner, we need to somehow decode this and provide a sensible (and visible) error diagnostic for the user. Such errors should stop the test run cold as well.

Test All the files in a folder

Lets say my tests files are located at 'tests' folder.
There are thousands of test files out there. How should I test them with a single bash command?

Uncaught exception logic is faulty.

While attempting to add a means of signalling an exception generated outside the context of a test to Whiskey, I happened upon another bug with respect to exceptions within the context of a test.

To reproduce:

Edit lib/common.js and insert this text on line 425: throw { message: 'hello world' };
Invoke Whiskey (see results below for how I invoked it).

Apparently, the _testObj member of the TestFile class is not being initialized.

$ NODE_PATH=$(pwd)/lib-cov bin/whiskey --coverage --coverage-reporter cli --coverage-file /tmp/hooey --coverage-dir /tmp -t example/test-bdd.js                                                                                                                                                                   /Users/sfalvo/git/whiskey :: Mon Jun 10 15:32:34 PDT 2013
path.existsSync is now called `fs.existsSync`.
/Users/sfalvo/git/whiskey/example/test-bdd.js

Stderr:

/Users/sfalvo/git/whiskey/lib/common.js:553
    test._testObj.finish();
                  ^
TypeError: Cannot call method 'finish' of null
    at TestFile.addUncaughtException (/Users/sfalvo/git/whiskey/lib/common.js:553:19)
    at process.<anonymous> (/Users/sfalvo/git/whiskey/lib/run_test_file.js:116:12)
    at process.EventEmitter.emit (events.js:95:17)
    at process._fatalException (node.js:272:26)

Make it work with Node 0.5 branch

Use relative imports
Remove all the require.path mucking

Ability to throttle test concurrency?

I have a feeling my tests are saturating my CouchDB instance, I am getting test errors that occur only when I run the entire suite as a group, in isolation things run fine. Is there a setting I can use to throttle the test runners a bit?

Abort noisily if test.finish() is called more than once in a case

Test cases can present mysterious race conditions (even where --sequential is used) if test.finish() is called more than once in a test case.

Whiskey should abort with a very prominent error if test.finish() is called more than once in a test case.

Report stack trace for thrown objects which are not instanceof Error

hide passed tests, require asserts, print expected and actual

Just a note about a couple of features I'd like to add as a patch some time:

I'd like to hide pass tests via command line argument.
I'd like to print the actual and expected inputs in the error report.
I'd like to be able to tell a test how many asserts to expect and to error out if it doesn't get that many.

Just things I want to add at some point.

Can't get a simple assert.response example to work

I'm writing a very simple assert.response example and it's failing with the following error.

Exception
ReferenceError: client is not defined
at Array. (/Users/matt/.nave/installed/0.4.2/lib/node_modules/whiskey/lib/assert.js:208:7)

Not sure what I'm doing wrong. I checked assert.js:208 and it wasn't obvious where 'client' was defined.

test code is here:
https://gist.github.com/1022251

Support for --teamcity reporter

mocha has cool support for teamcity reporter, it should not be hard to port it to whisky.

btw as I understand using mocha as bdd framework with whiskey is not a good solution?

feature: specify loadpath for tests?

It'd be nice if there was -i or something for me to specify the loadpath for my require statements, much like the node executable itself.

Add --failfast option back

Do it!

How to test with Code Coverage

Hi Can someone please point me out how to do code coverage with whiskey

test names

do test names have to have the word "test".

this is not working:

exports['foo'] = function(done,assert){
assert.ok(true); done();
}

whereas this test works:

exports['test_foo']....[insert same code here.

long-stack-traces causes event listeners to be non removable

It looks like long-stack-traces wraps events.addListener in some sort of tracing function. This makes the listener undiscoverable but events.removeListener as its call back is a different function. This also make events.once just a standard listener. You can see what happens in the test below.

  var TestClass, events, sys, _;
  _ = require("underscore");
  events = require("events");
  sys = require("sys");


  TestClass = (function() {
    function TestClass(test, assert) {
      _.extend(this, new events.EventEmitter);
      this.test = test;
      this.assert = assert;
      this.count = 0;
      return this;
    }

    TestClass.prototype.doFailTime = function() {

      this.once('reply', function(name) {
        this.count++;
        return console.log("" + name + " part 1 real awesome time!");
      });
      this.emit('reply', "first");
      console.log(sys.inspect(this.listeners('reply')));

      this.once('reply', function(name) {
        this.count++;
        return console.log("" + name + " real awesome time!");
      });
      this.emit('reply', "second");
      console.log(sys.inspect(this.listeners('reply')));

      this.once('reply', function(name) {
        this.count++;
        console.log("" + name + " real super awesome time!");
        this.assert.equal(this.count, 3);
        return this.test.finish();
      });

      console.log(sys.inspect(this.listeners('reply')));
      return this.emit('reply', "third");
    };
    return TestClass;
  })();

  exports["test for event emitter once"] = function(test, assert) {
    var myfail;
    myfail = new TestClass(test, assert);
    return myfail.doFailTime();
  };

exports['tearDown

exports['tearDown'] usage with SocketIO server testing

I want to be able to write the following code:

exports['setUp'] = (cb) ->
  SocketServer.control.startServer(cb)

exports['tearDown'] = (cb) ->
  SocketServer.control.startServer(cb)

Where setUp happens before all tests run, and tearDown happens after all the tests have run. Ideally, we are only running the server startup once, so all tests can async happen against the same server. Let me know if I can provide more help on this issue.

Better Test Debugging Experience

Hi,

When we have a test failure, it is very difficult to debug why a test is failing.

In traditional frameworks, you could add a debugger; statement inline a test and continue from there.

But since whiskey is spawning a subprocess, and doing more magic, this debugger statement doesn't work.

Maybe we need a whiskey-single-malt which runs in a single process (the parent) and lets you use the debugger on a single test file?

(or we need a better plan in general for debugging failing test cases)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.