Checking app code agains thresholds about ember-cli-code-coverage HOT 13 CLOSED

@adamjmcgrath with the merge of #169 which said that version works with nyc, can we officially recommend nyc as the method for checking thresholds in the README? Perhaps even offer a config to add nyc and check automatically, failing the Ember build if the threshold is not met?

from ember-cli-code-coverage.

Yep it’s possible to fail the build on Travis with nyc now that #169 is merged, I do it with this config:

.nycrc

{
  "all": true,
  "check-coverage": true,
  "per-file": true,
  "temp-directory": "coverage",
  "branches": 60,
  "functions": 60,
  "lines": 60,
  "statements": 60,
  "watermarks": {
    "branches": [70, 80],
    "functions": [70, 80],
    "lines": [70, 80],
    "statements": [70, 80]
  }
}

in package.json:

"scripts": {
  "test": "COVERAGE=true ember test",
  "check-coverage": "./node_modules/.bin/nyc check-coverage",
}

in .travis.yml

script:
   - npm test
   - npm run check-coverage

from ember-cli-code-coverage.

@jonathansamines sorry for the delay in responding, but could you please elaborate 😄

from ember-cli-code-coverage.

@rwwagner90 I mean, something like instanbul check coverage functionality, where if your current coverage doesn't comply with the expected coverage your build breaks :)

from ember-cli-code-coverage.

@jonathansamines I think we can configure istanbul more directly now, so this might be possible with the latest versions. Do you have any interest in trying to figure it out? I generally just enforce coverage thresholds with Code Climate, Coveralls, etc.

from ember-cli-code-coverage.

Not checked which fields of.istanbul.yml would work. You could also do nyc check-coverage --lines 95 --functions 95 --branches 95 in your posttest script.

from ember-cli-code-coverage.

Not checked which fields of .istanbul.yml would work.

Have checked this, the check .istanbul.yml fields don't work - in the long run we should probably fix this, but it wont work in the short run.

You could also do nyc check-coverage --lines 95 --functions 95 --branches 95 in your posttest script.

Have tested this, also doesn't work because nyc expects the coverage report to be in a different place/format

The solution that I've tested and works fine is using the istanbul@1 CLI

$ npm install --save-dev istanbul
$ COVERAGE=true ember test
$ istanbul check-coverage --statements=100 --functions=100 --branches=100 --lines=100 coverage/coverage-final.json

You probably need to make sure you use the json reporter so that you get a coverage-final.json file in your coverage folder

from ember-cli-code-coverage.

I haven't had an opportunity to try it yet - perhaps @charlesdemers can confirm?

from ember-cli-code-coverage.

@adamjmcgrath @charlesdemers would one of you like to document this in the README? I also think it could be nice to add nyc as a dep to this addon, and just build this functionality in. I would like the Ember build to fail, when the coverage is not sufficient, if we can do that. That way, testing locally and on travis are the same.

from ember-cli-code-coverage.

I think leaving it to the user to add in checking coverage is sufficient. They can change their npm test command to both run tests and check coverage, if they want that. I do think documenting that nyc utils can now be used with this addon is something we should do.

from ember-cli-code-coverage.

Sounds good. @charlesdemers would you mind submitting a PR to document how you use nyc?

from ember-cli-code-coverage.

Sure! I can do it sometime this week :)

from ember-cli-code-coverage.

This probably shouldn't be closed until the PR is either merged or closed.

from ember-cli-code-coverage.