Documentation suggests that recent versions of wdio-jasmine-framework allow for retries of flaky tests.

The most recent available version of wdio-jasmine-framework in node is 0.2.5, and the documentation states that this functionality is available in 0.3.0 or greater (unavailable? mistake in documentation?).

Anyway, there are tests in release 0.2.5 for the retry functionality, but the retries do not work on the tests I am running.

Version 2.20.1 of np was just published.

This version is covered by your current version range and after updating it in your project the build failed.

np is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Using afterEach like this:

  afterEach(function (done) {
  })

Should fail the test because done is never called, but it passes.

The problem

When a describe block is disabled using xdescribe, the beforeAll hook still gets executed.

Environment

• WebdriverIO version: Latest
• Node.js version: 4.2.8
• wdio test runner
• sync
• wdio-jasmine-framework (latest)

Code To Reproduce Issue [ Good To Have ]

Checkout the below branch:
https://github.com/patthiel/wdio-test-playground/tree/disabled-test-issue

Run: gulp wdio

Example Spec:
https://github.com/patthiel/wdio-test-playground/blob/disabled-test-issue/specs/test.spec.js

cc. @christian-bromann

Using xdescribe to ignore tests does not work.
Seeing the logs it seems that it runs the test but discards the result.

Version 4.18.1 of eslint was just published.

This version is covered by your current version range and after updating it in your project the build failed.

eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

☝️ Greenkeeper’s updated Terms of Service will come into effect on April 6th, 2018.

Version 6.0.1 of eslint-plugin-node was just published.

This version is covered by your current version range and after updating it in your project the build failed.

eslint-plugin-node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

see webdriverio/webdriverio#1073

Error: Cannot generate spec hash for runner with no 'specs' key
    at Error (native)
    at ReporterStats.getSpecHash (/Users/justin.williams/.nvm/versions/node/v5.0.0/lib/node_modules/webdriverio/build/lib/utils/ReporterStats.js:212:42)
    at ReporterStats.getSpecStats (/Users/justin.williams/.nvm/versions/node/v5.0.0/lib/node_modules/webdriverio/build/lib/utils/ReporterStats.js:227:33)
    at ReporterStats.suiteStart (/Users/justin.williams/.nvm/versions/node/v5.0.0/lib/node_modules/webdriverio/build/lib/utils/ReporterStats.js:239:18)
    at BaseReporter.<anonymous> (/Users/justin.williams/.nvm/versions/node/v5.0.0/lib/node_modules/webdriverio/build/lib/utils/BaseReporter.js:134:25)
    at emitOne (events.js:77:13)
    at BaseReporter.emit (events.js:169:7)
    at BaseReporter.handleEvent (/Users/justin.williams/.nvm/versions/node/v5.0.0/lib/node_modules/webdriverio/build/lib/utils/BaseReporter.js:326:27)
    at Launcher.messageHandler (/Users/justin.williams/.nvm/versions/node/v5.0.0/lib/node_modules/webdriverio/build/lib/launcher.js:469:28)
    at emitTwo (events.js:87:13)

Version 4.2.1 of sinon was just published.

This version is covered by your current version range and after updating it in your project the build failed.

sinon is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

In the README.

grep is said to take a String[] value, but in adapter.js it appears to expect grep to be a string version of a RegExp. https://github.com/webdriverio/wdio-jasmine-framework/blob/master/lib/adapter.js#L210

invertGrep is said to take a String[] value. This seems to be a copy & paste error. invertGrep is a boolean option.

I'm trying to write an async spec using Jasmine's done callback, but it complains that done is undefined. This works find with Jasmine 2.5.2 without the wdio wrapper.

describe('jasmine async', function () {
  it('works with a done callback', function (done) {
    setTimeout(function () {
      done.fail();
    }, 10);
  });
});

Which results in

․/home/adborden/projects/cg/new/973ade9632c4629c753422755ddacafe-106fc7d391d5f1d75bf190fa1023197320d39d6d/spec.js:5
      done.fail();
          ^

TypeError: Cannot read property 'fail' of undefined
    at Timeout._onTimeout (/home/adborden/projects/cg/new/973ade9632c4629c753422755ddacafe-106fc7d391d5f1d75bf190fa1023197320d39d6d/spec.js:5:11)
    at tryOnTimeout (timers.js:232:11)
    at Timer.listOnTimeout (timers.js:202:5)


1 passing (2.80s)


npm ERR! Test failed.  See above for more details.

But I would expect to see 1 failure.

I tried to setup a minimal example to demonstrate this in case there's something wrong with my setup. You can try this out locally by downloading this gist, unpack, and run npm install && npm test.

Async code (https request) is not executed within the test.

Steps:

code:

var http = require("https");
...

function api_request(options, to_write) {
  return new Promise(resolve => {
    var req = http.request(options, function(res) {
      var chunks = [];
      var cookies = res.headers['set-cookie'];

      res.on("data", function(chunk) {
        chunks.push(chunk);
      });
      res.on("end", function() {
        var body = Buffer.concat(chunks).toString();
        if (!isValidJSON(body)) {
          body = "{}";
        }
        resolve({ body: JSON.parse(body), cookies: cookies });
      });
    });

    if (options.method == "POST") {
      req.write(JSON.stringify(to_write));
    }
    req.end();
  });
}
describe('test', function() {
  beforeAll(function(done) {
    api_request(login_options, login_to_write).then(function(data) {
      var session_options = {
        "method": "GET",
        ...
      };
      return api_request(session_options);
    }).then(function(data) {
      var profile_show_options = {
        "method": "GET",
        ...
      };
      token = data.body.token;
      return token;
    }).then(function(data) {
      var profile_update_options = {
        "method": "POST",
        ...
      };
      var profile_update_to_write = {
        name: 'FirstName LastName',
        ...
      };
      return api_request(profile_update_options, profile_update_to_write);
    }).then(function(data) {
      done();
    });
  });

This code perfectly works (i.e. it successfully logs in to server, gets necessary data, then updates the required fields) in a separate .js file when run from node as
node test.js

This code also perfectly works if it is run within a standalone jasmine as
jasmine ./test.js

However it seems like this code is totally skipped when executed using wdio as:
./node_modules/.bin/wdio ./my/autotest/wdio.conf.test.js --spec ./my/autotest/specs/test.js

Setup:

"wdio-jasmine-framework": "^0.3.2",
"wdio-selenium-standalone-service": "0.0.9",
"jasmine": "~2.8.0",

Ive just had the following crash

/Users/chrismatheson/Code/teachersregister/web/node_modules/webdriverio/build/lib/utils/BaseReporter.js:336
                        throw _iteratorError;
                        ^

TypeError: Cannot read property 'split' of undefined
    at /Users/chrismatheson/Code/teachersregister/web/node_modules/wdio-spec-reporter/build/reporter.js:267:43
    at Array.forEach (native)
    at SpecReporter.getFailureList (/Users/chrismatheson/Code/teachersregister/web/node_modules/wdio-spec-reporter/build/reporter.js:265:22)
    at SpecReporter.getSuiteResult (/Users/chrismatheson/Code/teachersregister/web/node_modules/wdio-spec-reporter/build/reporter.js:318:28)
    at SpecReporter.printSuiteResult (/Users/chrismatheson/Code/teachersregister/web/node_modules/wdio-spec-reporter/build/reporter.js:326:30)
    at SpecReporter.<anonymous> (/Users/chrismatheson/Code/teachersregister/web/node_modules/wdio-spec-reporter/build/reporter.js:109:18)
    at emitOne (events.js:77:13)
    at SpecReporter.emit (events.js:169:7)
    at BaseReporter.handleEvent (/Users/chrismatheson/Code/teachersregister/web/node_modules/webdriverio/build/lib/utils/BaseReporter.js:324:35)
    at Launcher.messageHandler (/Users/chrismatheson/Code/teachersregister/web/node_modules/webdriverio/build/lib/launcher.js:590:28)

i think this is from your latest change 64373dd

where if an error does not contain a stack it will error with an internal error rather than the original

maybe this is something to do with warnings vs errors?

I'm getting this as the report for a failing test.

1) Form List Pagesuite1 should have the right title:
Expected 'Forms | AwesomeForm' to equal 'Forms | MyAwesomeFormApp'.

See the 'suite1' string above?

This is my test code.

describe('Form List Page', () => {
    beforeAll(() => {
        browser.login()
        browser.url('/#/forms/list')
    })

    it('should have the right title', () => {
        const title = browser.getTitle()
        expect(title).toEqual('Forms | MyAwesomeFormApp')
    })
})

My test suite is simply named 'Form List Page' and yet it gets displayed as 'Form List Pagesuite1'.

Here are my dependencies used:

"wdio-jasmine-framework": "^0.3.1"
"wdio-spec-reporter": "^0.1.0"
"webdriverio": "^4.8.0"

node v8.3.1

When calling pending('My reason here') in a jasmine spec, the reason is never displayed in WDIO (standalone mode) results. Small change in https://github.com/webdriverio/wdio-jasmine-framework/blob/master/lib/reporter.js is likely all that's needed. I can PR this change next week.

Reproduceable Spec:
https://gist.github.com/patthiel/7f68db05a21fdbd3cd80a2b7b66f8eb1

Version 2.3.2 of sinon just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As sinon is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

Version 4.2.0 of sinon was just published.

This version is covered by your current version range and after updating it in your project the build failed.

sinon is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 2.14.1 of np just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As np is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

Hi, I have an it that contains multiple expects and when one of them fails, I can't determine which one because in the command line it will only display one error for the whole it block so i need to manually check which of the expects failed. Is there a way to display all the failed in expects?

I use wdio example for jasmine. I made two changes. I defined expectationResultHandler function for taking screenshot and failed test from example where I changed title. I got result only 0 passing and random I got right result 0 passing, 1 failing.

I think there is problem with this line of code:
browser.saveScreenshot(('assertionError_' + title + '.png'));

The jasmine runner allows running of selected specs by changing it -> fit. However, fit's with generator functions don't correctly run the generator. It seems that "fit" was left out of the INTERFACES in lib/adapter.js

Hi, I am trying to use selenium grid with jasmine, is there any way to do it?
Thanks.

The problem

When a waitFor command or waitUntil times out, it throws an exception like below.. However, after this error is encountered, the runner fails to end the session where the error was encountered. Leaving many dead browsers open, when running a full suite of integration tests.

/Developer/myProject/node_modules/wdio-jasmine-framework/node_modules/wdio-sync/node_modules/fibers/future.js:313
                        throw(ex);
                        ^
Error: element (something) still not existing after 5000ms

Environment

• WebdriverIO version: 4.0.9
• Node.js version: 4.2.4
• wdio test runner
• synchronous & asynchronous modes
• gulp-webdriver
• selenium-standalone 2.5.3 (occurs in both FF driver & Chromedriver 2.22)

Link to Selenium/WebdriverIO logs

Selenium Logs
Wdio Logs

Code To Reproduce Issue

Example Spec

@christian-bromann i don't think this is happening with each framework, so i'm posting it here as a jasmine specific bug. Let me know if you think this is more a wdio bug and i can close this and open it in that project

Version 4.2.3 of sinon was just published.

This version is covered by your current version range and after updating it in your project the build failed.

sinon is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

First at all, thank @christian-bromann and the team so much for bringing us an awsome JS framework.

I just started using webdriverio, I used wdio as a test runner and Jasmine framework. Sorry if it's a dummy question if I missed any info from documentation (Developer Guide).

I saw that to capture a failure, we need to add:

But in my wdio.conf.js. I didn't set that but I'm still getting screeenshot on failure. That's cool but I don't now why.

Thanks!

We are considering switching from sync: true to sync: false in wdio.conf and using the new ES 7 async/await syntax instead of fibers, but we cannot get jasmine to work with sync: false.

We have found fibers to sometimes cause confusion to devs not yet familiar with what fibers are actually doing and how they work. They also require adding commands to browser for async actions like setting up drymock that has nothing to do with browser.

My understanding is that if a test returned a promise that it would wait. This appears to be natively supported by mocha, but this does not seem to be natively supported by jasmine. In other projects we are using jasmine-pit to add support for promises (and the async/await syntax).

The wdio-sync project assumes the specFn takes no arguments here and thus swallows the done function object provided by jasmine and required by plugins such as jasmine-pit.

Additionally wdio-jasmine-framework only supports test methods named 'it' or 'fit' here so a plugin like jasmine-pit would not work anyway.

I created a test project webdriverio-async-jasmine-tests with some very simple specs that don't behave as expected.

I created two async style tests with sync:false configured in wdio.conf. One of which expects done argument which is not provided and the other which returns a promise.

Interestingly, wdio does appear to wait for the promise in beforeAll, but it does not wait for promise returned from it. I'm not entirely certain why promises work in beforeAll but not in it as they seem to use mostly the same code. It would be awsome if wdio-jasmine-framework would resolve promises returned from the it block because then the async/await syntax should just work.

Is it feasible to have promises returned by it block auto resolved by wdio? or does this require changes to jasmine itself?
If jasmine changes would be needed how about supporting the pit method to resolve promises?

webdriverio-async-jasmine-tests

Version 2.9.0 of jasmine was just published.

This version is covered by your current version range and after updating it in your project the build failed.

jasmine is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

Version 2.19.0 of np was just published.

This version is covered by your current version range and after updating it in your project the build failed.

np is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 2.99.0 of jasmine was just published.

This version is covered by your current version range and after updating it in your project the build failed.

jasmine is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Your Greenkeeper Bot 🌴

I'm not sure if this is the correct place to request this feature, but it would be great to be able suppress stack traces when tests fail, similar to the --noStack option provided by jasmine-node.

Version 2.9.0 of eslint-plugin-import was just published.

This version is covered by your current version range and after updating it in your project the build failed.

eslint-plugin-import is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

...and pass on the stopSpecOnExpectationFailure config option.

Related issue: webdriverio/webdriverio#1654 (comment)

Version 2.4.0 of eslint-plugin-import just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As eslint-plugin-import is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

From the discussion at webdriverio/webdriverio#1928

Jasmine has implemented the concept of stopping after the first failure: https://github.com/jasmine/jasmine-npm/blob/80e6b334d424202893519e6900568c65a663db40/lib/jasmine.js#L147-L149

However, it simply causes expectations to throw exceptions. Based on the framework tests, those throws are intentionally ignored.

I have been unable to figure out how to properly handle this. If I can get some guidance, I'll PR the changes.

Version 4.17.0 of eslint was just published.

This version is covered by your current version range and after updating it in your project the build failed.

eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Writing a test with waitForVisible gives a command error instead of being picked up by jasmine. This leads to the test not being reported as failed in the generated junit xml file.

Test with:

describe('test waitForVisible', function() {

    it('should not crash on waitForVisible', function() {
        browser
            .url('http://the-internet.herokuapp.com/shifting_content/menu?mode=random')
            .waitForVisible('#content7', 1000, false, function(err, res){
                console.log("Error: " + err + ", response " + res);
            })
        ;
        expect(true).toBe(true);
    });

});

Version 5.0.1 of mocha was just published.

This version is covered by your current version range and after updating it in your project the build failed.

mocha is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

I opened issue #16 , but it was closed. The resolution for issue #16 provides a synchronous example. My use case is while using async/await functionality and with the sync flag to false.

☝️ Greenkeeper’s updated Terms of Service will come into effect on April 6th, 2018.

Version 4.4.0 of sinon was just published.

This version is covered by your current version range and after updating it in your project the build failed.

sinon is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

When running WDIO tests in SauceLabs only the names of the it functions are appearing. Other than this the test is listed as Jasmine TopLevel Suite. The WDIO documentation for this functionality uses Cucumber as the example and it clearly displays the name of both the feature and scenario.

Version 4.7.2 of eslint just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As eslint is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

Hi there,

I am building a test suite using this adapter, and can't get my jasmine tests to retry when expectations fail. Throwing an error explicitly works though (as explained in #16 by @christian-bromann ):

package.json

{
  "dependencies": {
    "jasmine": "2.5.3",
    "selenium-standalone": "6.1.0",
    "wdio-jasmine-framework": "0.3.0",
    "wdio-selenium-standalone-service": "0.0.8",
    "webdriverio": "4.6.2"
  }
}

wdio.config.js

exports.config = {
  specs: ['./tests.js'],
  services: ['selenium-standalone'],
  capabilities: [{
    browserName: 'chrome',
  }],
  framework: 'jasmine',
};

tests.js

describe('failing tests', () => {
  it('b', () => {
    console.log('run me b');
    throw new Error('fail');
  }, 2);

  it('a', () => {
    console.log('run me a');
    expect(true).toBe(false);
  }, 2);
});

console output

run me b
run me b
run me b
run me a

0 passing (3.00s)
2 failing

I tried throwing errors in the expectationResultHandler instead, but that doesn't help either. Any help would be greatly appreciated! 🙂

Version 4.16.0 of eslint was just published.

This version is covered by your current version range and after updating it in your project the build failed.

eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

☝️ Greenkeeper’s updated Terms of Service will come into effect on April 6th, 2018.

Version 4.4.1 of sinon was just published.

This version is covered by your current version range and after updating it in your project the build failed.

sinon is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 2.20.0 of np was just published.

This version is covered by your current version range and after updating it in your project the build failed.

np is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 8.2.1 of babel-eslint was just published.

This version is covered by your current version range and after updating it in your project the build failed.

babel-eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 4.18.0 of eslint was just published.

This version is covered by your current version range and after updating it in your project the build failed.

eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Version 2.7.0 of jasmine just got published.

This version is covered by your current version range and after updating it in your project the build failed.

jasmine is a direct dependency of this project this is very likely breaking your project right now. If other packages depend on you it’s very likely also breaking them.
I recommend you give this issue a very high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

In theory you should be able to call brower.debug() to manually inspect the browser state continue when you are ready by pressing enter at the console.
http://webdriver.io/api/utility/debug.html

I get that async tests shouldn't wait forever, but it would be nice if there was a way to disable this automatic timeout when debugging.

We enjoy using jasmine with WebdriverIO. We prefer to use async/await over the synchronous code with node-fibers and therefore have set sync: false in our config. With the change to node 7 we had issues with node-fibers binding not being compatible. I looked through some issues and found that others had problems with fibers as well.

Would it be possible to avoid requiring node-fibers (or wdio-sync) if synchronous mode is disabled anyways? I understand that it needs to be in the dependencies, but I imagine it wouldn't need to be required.

I am happy to provide a PR if you think this is a good idea. Any pointers to code are appreciated. Thanks!