dvdoug / behat-code-coverage Goto Github PK

Hi,
first of all, very thanks for this package !

I am using Behat for testing a Symfony API Platform REST API that I developed. As of now I have 3 tests suites involving ~200 scenarios. Whatever the tests I am running I always get the same results which are:

Code Coverage Report:     
  2022-02-05 11:10:25     
                          
 Summary:                 
  Classes:  0.00% (0/8)   
  Methods: 27.27% (15/55) 
  Lines:   11.89% (27/227)

App\Doctrine\DocumentAnnexListener
  Methods:  50.00% ( 2/ 4)   Lines:  18.18% (  4/ 22)
App\Doctrine\DocumentReceivedListener
  Methods:  50.00% ( 2/ 4)   Lines:  12.00% (  3/ 25)
App\Doctrine\DocumentStatusListener
  Methods:  50.00% ( 2/ 4)   Lines:  11.43% (  4/ 35)
App\Doctrine\HashPasswordListener
  Methods:  40.00% ( 2/ 5)   Lines:  16.67% (  4/ 24)
App\Entity\User
  Methods:  14.29% ( 4/28)   Lines:  13.95% (  6/ 43)
App\Repository\ContactRepository
  Methods:  33.33% ( 1/ 3)   Lines:   6.45% (  2/ 31)
App\Repository\DocumentHistoryRepository
  Methods:  50.00% ( 1/ 2)   Lines:  18.18% (  2/ 11)
App\Repository\UserRepository
  Methods:  20.00% ( 1/ 5)   Lines:   5.56% (  2/ 36)

Any idea about this behaviour ?

Indeed I have more than 10 tested Doctrine 'entites' located in the App\Entity but the only one that is in the report is my User class that is used for login on the API ...

The application has one simple feature file:

Feature: Foo
  Szenario: Bar
    Given I am on the homepage

This succeeds and runs all the code in the file "src/index.php". But the following Behat configuration does not measure the application coverage:

    DVDoug\Behat\CodeCoverage\Extension:
      filter:
        include:
          directories:
            'src': ~
      reports:
        html:
          target: "tmp/behat/coverage"

It is missing the connection between Behat and the actual code that runs.
Is there a solution for that?

If not, then please consider adding a "bootstrap.php"-like file that could be included in the actual application. Such file could measure the actual coverage for each time the application is run during the test.

Hello 👋

Thank you for this amazing library. I Started using it at work for GitLab coberturra coverage reports. Recently I noticed this Path coverage, and I thought I'd dig in a little more as even with high branch and line coverage, our Path coverage is quite low.

Big Picture, the coders on my team should stop writing long, winding methods. I'm with you; but I'm not sure how practical it is to say "You should have known and not been doing this in the past".

Just found the setting branchAndPathCoverage. I like Branch coverage, but would like to disable path coverage. In the worst case scenario I will turn off both branch and path coverage to see if it improves the coverage reports we generate; as other teams are using pcov, which isn't xdebug and so should not be generating path coverage anyhow. But will I lose branch coverage? Is there a possibility to have each of the two independently configurable?

Thanks for your time reading.

Hello 👋,

First of all thank you very much for your hard work! I recently (re)started dabbling with Behat and was lucky enough to find this extension.

I would like to propose a feature request: optionally allow to fail tests below a certain threshold, similar to Infection with their --min-msi and --min-covered-msi options (see their docs).

Is this something worth considering? If so I think I can give it a shot as PR the feature.

Thank you for your time!

Related: #1

Was debugging a bit as the throw error with message No code coverage driver is available is a bit non-descriptive when you've never thought about it ;)

Suggestion:

File: src/Driver/Selector.php
Line: 43

Change:

$driver = new Xdebug3Driver($filter);

to:

try {
    $driver = new Xdebug3Driver($filter);
} catch (Xdebug3NotEnabledException) {
    throw new NoCodeCoverageDriverAvailableException('Xdebug 3 was found enabled, but "mode" does not include "coverage", did you forget to enable this?');
}

Subsequent required update to NoCodeCoverageDriverAvailableException :

final class NoCodeCoverageDriverAvailableException extends RuntimeException implements Exception
{
    private const DEFAULT_MESSAGE = 'No code coverage driver available';

    public function __construct(string $message = null)
    {
        parent::__construct($message ?? self::DEFAULT_MESSAGE);
    }
}

Suggested changes would make finding why it's not working (initially) easier. (Admittedly: I did do a "doh, that makes sense" after I found it :p )

Suggested changes would also allow the same for other coverage drivers to have similar messages with minimal effort (try/catch blocks).

If you would like some help doing such changes, let me know and I'll see if I can spare some time outside of work.

I've just found your package and fell in love immediately. But there's one more thing I'd like to see: on my local machine, I run Behat tests before commiting. Sometimes, I have to debug them using XDebug. With the extension enabled, this fails, as I usually don't run XDebug in coverage mode, and the driver throws an exception for this in https://github.com/sebastianbergmann/php-code-coverage/blob/0b4ce85a86040303d93beddf4f31f2ab07775e9a/src/Driver/Xdebug3Driver.php#L65.

You properly catch errors based on SebastianBergmann\CodeCoverage\NoCodeCoverageDriverAvailableException (when absolutely no driver is available), but I think you could extend the catch block with SebastianBergmann\CodeCoverage\Driver\Xdebug3NotEnabledException. This could help to always enable the extension and run whenever possible. In case of any error, display an information like "No code coverage driver is available" or "Code coverage driver not configured properly", but keep on running the testsuite

The project I'm working on has the following shape (I simplified it):

├── projects
│   ├── api
│   ├── backend
│   ├── cdn
│   ├── common-lib
│   ├── frontend
│   └── worker
├── tests
│   ├── behat.yml
│   ├── features

How can I tell this extension that the code is in ../projects/*/src ?

Note: I tried the following configuration, but the coverage is always 0% (but files are good)

        DVDoug\Behat\CodeCoverage\Extension:
            filter:
                include:
                    directories:
                        '../projects/*/src/': ~

Thanks