Jest have this feature http://facebook.github.io/jest/docs/configuration.html#coveragethreshold-object and it would be nice to have a similar CLI arg to fail on CI :)

Hi there,

I would love to use this tool with codecov.io but I think I will need a lcov report, can you tell me what needs to be done than I will try to have a look into it 👍

First of all, this looks like a great tool. I still find that, for example, when I call
const result: MyType = await response.json() on an isomorhic-fetch I get that part of code uncovered, whatever I do. The other place is a default (action: empty) guard in a react-redux reducer which is also reported as an uncovered code.

I want to try to get a 100% coverage, if that is possible at all, and thus would like to mark "unsafe" places somehow, the first idea is to analyze the generated json file, extract "uncovered_locs" together with a source code for those chunks and get rid of those pieces of code which I explicitly specify. The other option would be to somehow annotate those places directly, although no ideas how, may be with a special comment like
const result: MyType = /*UnsafeStart*/ response.json() /*UnsafeEnd*/

Getting Error while generating Flow Coverage Report: Error: stdout maxBuffer exceeded Error: stdout maxBuffer exceeded for some reason.

node: v6.2.0
command: node_modules/.bin/flow-coverage-report -i 'src/**/*.js' -t html -t json -t text

Currently react and react-dom specified as dependencies in package.json.
If user use next React version, e.g. React v16, there will have some unexpected dependencies in node_modules.
Here to propose move them as peerDependencies.

Let's say I have pragma defined at the top of the file:

// @flow @jsx h

Running a script below adds annotation no-flow which is not correct.

flow-coverage-report -f ./node_modules/.bin/flow -i src/**/*.js

Are you able to fix it?

Hello. I'm trying to use this package, but am running into an odd problem when attempting to run a report over a directory of files.

The command I'm using is:

yarn flow-coverage-report -- -i 'app/**/*.js'

This results in:

ERROR: The include glob needs to be quoted. error Command failed with exit code 1.

The odd thing is, I can run this command successfully on a directory that doesn't exist (say I swap app with xyz. I can also run the report on a single js file.

I get the same result running the package directly (outside of yarn). Thanks!

When I run the following command:

flow-coverage-report -i 'src/**/*.js' -i 'src/**/*.jsx' -t html -t json -t text

I get this error for any file with a // @flow strict header:

Error while generating Flow Coverage Report: Error: Unexpected missing flow annotation on src/.../foo.js Error: Unexpected missing flow annotation on src/.../foo.js

When I remove 'strict' from that header comment, the errors moves on to another file with it. This is syntax officially supported by flow (https://flow.org/en/docs/strict/) so expected it to work with this tool.

Hi,

I get the following warnings when running flow-coverage-report from my package.json:

WARN: "includeGlob" config file property has been renamed to "globIncludePatterns"
WARN: "excludeGlob" config file property has been renamed to "globExcludePatterns"
WARN: "type" config file property has been renamed to "reportTypes"

Is there any reason these changes haven't been made in the README? If not I can submit a PR.

when i use flow-coverage-report through my package.json, none of my .flowconfig settings get picked up. this results in a lot of false negatives (basically, every single imported 3rd party module gets marked as not typed)

my settings look like this:

It would be really useful for integrating with other tools if flow-coverage-report had a programmatic api.

import flowCoverageReport from 'flow-coverage-report'

flowCoverageReport(options).then(({percent}) => console.log(`${percent}% covered`))

Because this project uses pinned dependency versions, it's very likely that consumers will end up with multiple versions of various dependencies like React. React doesn't like to have multiple versions, and in some cases this is causing me problems.

I think it would be better to use semver and specify only the minimum required versions, and allow proper dependency resolution. You can check in a package.lock or yarn.lock file to ensure that consistent versions are used during development, if you like. But I don't think it's a good idea to pin dependencies as you have in your package.json.

This is a feature request and pool for ideas.

I can cannot achieve 100% coverage for the reasons explained here, but I want to make sure as the codebase grows that the amount of untyped code will not grow.

One way to limit this is to enforce coverage on a per-file basis. Theory being e.g. 95% on each file is better then 99% overall.

There would also need to be an additional option to specify a coverage map which would describe override thresholds for files that need it (for example, files that use exceptions from catch.

Thoughts?

npm audit on a package that uses flow-coverage-report reports a moderate severity vulnerability in flow-coverage-report > badge-up > svgo > js-yaml > argparse > underscore.string

When incrementally adding flow coverage to an existing project it is recommended to add the @flow annotation to one file, run flow to see a list of errors and proceed to fix those errors before starting on the next file. See https://flowtype.org/docs/existing.html. However it's not reasonable to expect each file will achieve 100% coverage after it has been touched because not all of the dependencies will have coverage either.

Tp support this workflow it would be helpful if flow-coverage-report would indicate not just what percent of a file is covered by flow, but which files have /* @flow */ or /* @flow weak */ annotations. This would make it easier to ensure that all statements are covered, but also that errors/warnings are emitted by flow when they should be.

Hi,
I was trying to implement your amazing package, but I've run into a bit of problem internally with the license.
It looks like we don't support the Mozilla Public License, and that's turning out to be a real blocker for me :-(
I think I would have more luck using your package internally if the license were one of these:

• Apache 2.0 License
• BSD "2-clause" or "Simplified" License
• BSD "3-clause" License
• MIT License
• Code Project Open License 1.02
• Microsoft Public or "Permissive" License
• PostgreSQL License
• Scala License
• zlib/libpng License
• ISC
• WTFPL (slightly offensive)
• Public Domain Dedication
• The Unlicense
• Creative Commons Attribution License (but NOT other Creative Commons licenses such as ShareAlike and Noncommercial)

Any help or consideration you could offer would be appreciated! I'd love to have the ability to use your package here at LinkedIn!

• John

#1

Sorting the columns of a report with ~5500 files/table rows takes ~40 seconds for the first sort, then ~6seconds for subsequent ones.

First sort (by uncovered lines, descending):

Subsequent Sort (uncovered lines, asc):

Overhead from jQuery is the likely culprit. Google shows there are many other drop-in options that may or may not work just as well. I've no suggestions. I'll try to test a few out and see about a PR

It would be nice to be able to get the coverage in decimal points (e.g. 52.34%) and likewise fix the threshold to a custom precision (e.g. --threshold 52.34).

The use case I have in mind is bigger code bases where 100 lines of code being covered or not would currently result in the same coverage percentage, so there is no fine way to control the threshold.

We use Jenkins for CI. For now, we are linking to the html version of flow coverage report, which is awesome. But we want to use cobertura to have better historical graph and simple statistics in task view.

Is there any plan to output xml in cobertura format? Or anyone knows if there's already some scripts available to convert the json result into cobertura format?

Uses:

.CodeMirror {
  height: 300px
}

300px very small, feel free get higher please. We need to grow here. 👆

I started to add flow annotations to my project by adding // @flow to certain files.
I would like to make sure that those files are passing a certain flow coverage threshold.

At the moment, it looks like the files included in the coverage report are based on -i (and -x).
This causing the covered percent to be very low.

Is it possible to report coverage only on files marked with // @flow?

This is useful for precommit checks and CI integration.

The generated index.html list of files can get really long for large projects. It would be helpful to split the output by source folder and summarize the coverage info for everything below.

ie:

index.html

comments_index.html

Hi!
First of all, thanks for the great lib :) It really helps us tracking+finding how to improve our coverage!

I wanted to suggest a feature, it would be great to be able to see the decimals of the coverage in the final summary. For instance:

(instead of just 70%)

It would allow to have a more micro visibility on the impact of the recent changes the developer made.

What do you think about it? I would be happy to help ;)

Hello! Awesome project! I'm not sure how you can work using Flow without this.

I've been trying to upload the coverage reports to http://codecov.io/ so we can get some additional reporting on our coverage over time. The @codecov team has been really helpful but one thing is missing from the generated report.

There is uncovered_locs but no inverse for covered_locs, only the sum covered_count which doesn't make is possible for them to create the line-by-line report. The flow report only shows lines not covered and therefore Codecov will never know when lines are so it is always resulting in 0% coverage.

Can covered_locs be added to the output?

I searched but couldn't find any previous mention of covered_locs in issues or code. I could try to take a look if I knew where to get started. I'm happy to sit down and pair on this as well.

Example

Oh yeah, here's an example coverage file from our project just in case its useful:
https://codecov.s3.amazonaws.com/v4/raw/2017-01-11/AAA12261C5E4BE20272185677850291B/a1ff5a7fc828aa85277cb09a5e534eb4a699960c/d74f9f70-e1c8-469d-8f37-6cf4e1afe8eb.txt

You have to scroll down to:

# path=/tmp/circle-junit.qhJLwiZ/flow-coverage/flow-coverage.json

Thank you!

😃

Ideally I'd have a 100% flow coverage in my projects. However, this is usually not realistic, there are almost always parts of the code base that either can't be covered at all, or are extremely difficult to cover properly. As a result, we generally can't set the threshold to 100% for CI builds, we need to lower it to a somewhat arbitrary lower value. This has as the unintended side-effect that the build will now allow a non-zero percentage of unintended uncovered code as well.

It would therefore be very useful if we could somehow specify that certain parts of the code should not count towards coverage percentage, e.g. by using comments. For example:

/* flow-coverage ignore-next-line */
const y : any; 

const z : any; // flow-coverage-ignore-line

/* flow-coverage ignore-start */
const a : any; 
const b : any;
/* flow-coverage ignore-end */

The tool could still warn about missing coverage, but it just wouldn't count towards to threshold.

With something like this we would be able to set the threshold as 100% and explicitly opt-out of coverage where desired.

Would something like this be possible and desirable for this project?

I just noticed that flow-coverage-report isn't escaping special chars, and it causes the check to fail.

Without the escaping we're basically asking flow for the ast of some other file:

"error":{"killed":false,"code":10,"signal":null,"cmd":"flow ast /path/to/file-with--a-slash.js"},"stdout":"","stderr":"File not found: /path/to/file-with--a-slash.js\n"}

Should be file-with-\-a-slash.js

I added some fixtures to flow-annotation-check and a little method to help out, here's how it looks:

It looks like master is 50+ commits ahead of the latest release (0.3.0). Any chance we can get a new release?

Version 10 of Node.js (code name Dubnium) has been released! 🎊

To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:

• Added the new Node.js version to your .travis.yml

If you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.

Your Greenkeeper Bot 🌴

I've added flow-coverage-report to my project. I'm using [email protected], and it's not configured to ignore node_modules. (I believe this is best practice.)

Flow reports this:

node_modules/flow-coverage-report/src/lib/flow.js:87
 87:   // $FLOW_FIXME: code is there, but flow doesn't seem to know about it.
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Error suppressing comment. Unused suppression

I guess an earlier version of flow barfed at the next line. It isn't now, so flow is now differently barfing.

Removing line 87 stops the error, which I guess is the correct fix.

 ~/src/frontend  master ?  flow-coverage-report -f node_modules/.bin/flow -i 'src/**/*.js' -x '**/*-test.js' -x '**/index.js' -t html                                                                 ✓  5575  17:05:17

(node:26397) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Missing percent in coverageData
(node:26397) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: Missing percent in coverageData

No other output, although it does write a report to file.

If a file with no type at all, flow-coverage-report returns this result line:

src/url.js                          │   NaN % │     7 │       0 │         7

I first thought it was a division by 0, but then realised that's the other way around.

Maybe unrelated, but when I then try to generate an html report, I get:

(node:54292) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Missing percent in coverageData
(node:54292) DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

For reference, here is the content of the src/url.js file:

// @flow

const url: any = window.URL || window.webkitURL || window.mozURL

export default url

Not working properly on Windows.

I've tested it on multiple different Windows and Linux computers. It works perfectly on Linux, but on Windows, here's what the output looks like:

┌──────────┬─────────┬───────┬─────────┬───────────┐
│ filename │ percent │ total │ covered │ uncovered │
└──────────┴─────────┴───────┴─────────┴───────────┘
┌─────────────────────────┬───────────────────────────────────────────────────────────┐
│ included glob patterns: │ 'core/src/**/*.js'                                        │
│ excluded glob patterns: │ '**/node_modules/**'                                      │
│              threshold: │ 1                                                         │
│       concurrent files: │ 20                                                        │
│           generated at: │ Wed Oct 18 2017 14:46:14 GMT+0200 (Romance Daylight Time) │
│           flow version: │ 0.52.0                                                    │
│      flow check passed: │ no ( >= 50 errors)                                        │
└─────────────────────────┴───────────────────────────────────────────────────────────┘
┌───────────┬─────────┬───────┬─────────┬───────────┐
│ project   │ percent │ total │ covered │ uncovered │
│ workspace │   100 % │     0 │       0 │         0 │
└───────────┴─────────┴───────┴─────────┴───────────┘
View generated HTML Report at file://C:\Jenkins\jobs\ActiveUI-nightly\workspace/dist-coverage/flow/index.html

The above output was reproduced on two different very different Windows computers, so this is likely not an environment-related issue, but rather an issue in this project.

I'm running flow-coverage-report against a moderately large codebase. It's failing get coverage information for some files, seemingly at random. I'm not sure what's different about these files from the others.

An example:

ERROR Collecting coverage data from scripts/uglify.js { Error: Command failed: flow coverage --json scripts/uglify.js
    at ChildProcess.exithandler (child_process.js:297:12)
    at ChildProcess.emit (events.js:197:13)
    at maybeClose (internal/child_process.js:984:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:265:5)
  killed: true,
  code: null,
  signal: 'SIGTERM',
  cmd: 'flow coverage --json scripts/uglify.js' }

Interestingly, running flow coverage --json scripts/uglify.js manually works completely fine. The number of files it fails on also seems to vary depending on what the -c flag has been set to. If it's left on the default of 1, it fails on about a third to half of the files, and setting it to 4 causes it to fail on all of the files in the entire codebase.

I'm starting a new project from scratch and want to have things type checked with Flow as best as I can from the start. To that end, I'm using all=true under the [options] section of my .flowconfig. with all of my code in {project dir}/src, I run the coverage tool as:

yarn run flow-coverage-report -i src/**/*.js

All of the files come back marked red/failing due to no flow annotation regardless of coverage %, which makes it more annoying to identify which files actually need attention. With the all=true option enabled, no annotation should not be a reason for failing.

When I install this module locally to the project as a devDependency, running flow-coverage-report with my custom flags yields:

Error: Cannot find module 'react/lib/ReactDOMServer'

However, if I install this module globally and run the same command, it works. Ideas? My app uses...

    "react": "^15.4.2",

I'm using the tool for the first time, and I'm finding something weird. I'm typing the variables that I require, but the tool still tells me that the line is not fully covered because I need to type also the required part. But I'm not sure how to type it. Maybe is this s bug? I'm not really sure. Here it is a screenshot of my output:

When using flow in a project where flow-coverage-report is installed to node_modules, I get the following errors:

Error: node_modules/flow-coverage-report/src/lib/report-html.js:70
                v------------------------
 70:     return Promise.reject(new Error(
 71:       'filename is not relative to the projectDir: ' +
 72:         [opt.projectDir, opt.filename].join(' - ')
 73:     ));
         -^ type parameter `T` of call of method `reject`. Missing annotation

Error: node_modules/flow-coverage-report/src/lib/report-html.js:181
                v-----------------------------------------
181:     return waitForReportContent.then(function (res) {
182:       var reportFilePath = res.reportFilePath;
183:       var reportFileContent = res.reportFileContent;
...:
188:     });
         -^ type parameter `U` of call of method `then`. Missing annotation

Error: node_modules/flow-coverage-report/src/lib/report-html.js:191
191:   return Promise.reject(new Error('Unknown report type: ' + opt.type));
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ type parameter `T` of call of method `reject`. Missing annotation

I've excluded flow-coverage-report via my .flowconfig for now, but it would be nice to have this fixed upstream.

The HTML report doesn't get passed custom threshold values.

I just installed the module globally. When I run:
flow-coverage-report report -i "src/**/*.js"
straight from the examples (which happens to match my dir structure) I get:
Configuration error: ERROR: No glob has been specified.

Am I doing something stupid here?

It would be nice if the HTML report came with a badge that I can put on top of my README.md that links to the HTML report.

flow-coverage-report starts a Flow server in:

but never stops it. The server process thus continues running unnoticed in the background.

When using services like CircleCI, running flow-coverage-report without running flow stop manually afterwards will fail the build. Indeed, CircleCI will consider that the Flow process has been running for too long, will terminate it, and mark the build as failed.

Would you please consider making flow-coverage-report stop the Flow server when it started it itself?

Thanks.

I ran the following command:

./node_modules/.bin/flow-coverage-report \\
-f ./node_modules/.bin/flow \\
-i 'packages/wonder-blocks-*/**/*.js' \\
-x 'packages/wonder-blocks-*/dist/*.js' -x '**/generated-snapshot.test.js' \\
-o "./flow-coverage"

and it prints out the coverage but doesn't store the output anywhere. The -o seems redundant since "./flow-coverage" is the default so I tried with and without that parameter with the same results.

Hi,
I stumbled across some difficulties when trying to invoke flow-coverage-report from npm script.

I'm using execa for this but it was reproduced with child_process too.

 await execa.shell('yarn flow-coverage-report --project-dir /rootDir --output-dir /rootDir/coverage -t json --threshold 1 --include-glob /rootDir/src/**/*.js --exclude-glob /rootDir/src/legacy/**/*.js')

What I get is

message: Command failed: <command>
    Error while generating Flow Coverage Report: [object Object] undefined␊
    error Command failed with exit code 255.␊

Invoking command from command line seems to work pretty well.
Issue is gone after removing --project-dir, though

Hey there,

I would really like to get such a cool coverage report for my project. Unfortunately the latest version says that node_modules/.bin/flow status --json fails. I can verify that, this command needs a file as a target. My call is flow-coverage-report -i './src/**/*.js' -x './node_modules' -f node_modules/.bin/flow -t html -t json -t text. Am I misusing this library?

Currently it's possible to set a coverage threshold. It would be nice if there was another option that made it required for the Flow check to pass.

I should've opened a bug before. But track this in this repo too:
3rd-Eden/find-package-json#5

codemirror supports jsx you should just add it to the list of dropdown options.
maybe you assume all jsx files have jsx extentions which is not correct all the time.
I have jsx inside my normal js files

It'd be really useful to be able to specify a threshold on uncovered line count, i.e. a maximum number of uncovered lines.

Why? In a codebase which continuously grows, if you fix the number of lines uncovered then you (loosely) enforce that all new code should be covered.

It's a neat trick which provides an automatic coverage ratchet, so long as the codebase keeps growing.