Integration With SonarQube? about minicover HOT 21 CLOSED

btw there also seems to be another option but seems to need a lot of work.

from minicover.

I'll give it a shot.

from minicover.

I think from what is reported here:

if we want integration with SonarQube, we should be able to produce reports compatible with the ones produced from the following tools:

• Visual Studio Code Coverage
• dotCover
• OpenCover

NCover seems to be kind of not-up-to-date ?

(unless of course we opt for the generic data format supported by SQ.)

from minicover.

This seems to be working! Nice job! thanks @havocbcn

just a question in case anyone knows: is there a way to grep the total percentage from some file? (I need it in case I want to keep my gitlab coverage badge) or some other way of extracting cmd-wise the percentage? was not able to grep it in opencovercoverage.xml neither from coverage.json

from minicover.

@pkaramol
Thanks for all details. Everything looks good, except that that's the exact version that has the bug I mentioned above.
I merged this #32 at 26th 19:01, and the version you're using was generated minutes later.
Change 2.0.0-ci-20180226190451 to ci-20180301063918

from minicover.

yep. that fixed it! thx for the support.

from minicover.

Hey, currently it doesn't support SonarQube.

@jonjomckay added support to generate NCover xml like reports. If SonarQube supports it, maybe it's possible to integrate.

from minicover.

Would love to have this feature for MiniCover.

According to SonarQube documentation - only NCover 3 is supported. @lucaslorentz the new support for NCover xml like reports - which version of NCover did @jonjomckay use?

from minicover.

Changing MiniCover to generate a file compatible with SonarQube shouldn't be hard.
But does SonarQube support to simply upload that file?

I tried to follow the instructions in https://docs.sonarqube.org/pages/viewpage.action?pageId=6389770 , but it seems that SonarQube doesn't support .NET Core for it's analysis, and there is no way to just upload the coverage report.

Do you have any suggestions?

from minicover.

According to their documentation, .NET Core is supported if you have at least:

• SonarQube v6.7+
• SonarC# v6.7+
• Sonar Scanner MSBUILD 4+

Doc: https://docs.sonarqube.org/display/SCAN/Scanning+on+Linux+or+macOS

I have been using mono to run Sonar Scanner MSBUILD in practice in a ConcourseCi pipeline on a debian linux container and everything works. The issue with .NET Core is Coverage Reports on linux are near non-existent.

My theory is that we may find success using the /d:sonar.cs.ncover3.reportsPaths="%CD%\coverage.nccov" option for the Sonar Scanner MSBUILD - to point at MiniCover 'NCover3 like' reports.

from minicover.

Here is the task snippet used in my CI/CD pipeline + the addition of ncover3.reportPaths.

#!/bin/bash
mono $SONARMSBUILDPATH/SonarQube.Scanner.MSBuild.exe begin \
    /d:sonar.host.url="$SONAR_URL" \
    /d:sonar.login="$SONAR_KEY" \
    /d:sonar.cs.ncover3.reportsPaths="%CD%\coverage.nccov" \
    /key:"$PROJECTNAME" \
    /name:"$PROJECTNAME"

dotnet restore
dotnet build
dotnet test

mono $SONARMSBUILDPATH/SonarQube.Scanner.MSBuild.exe end /d:sonar.login="$SONAR_KEY"

from minicover.

fyi I have tried @mcblair 's approach.

the two xml s do not seem to be compatible. perhaps could work with some xslt transormation because they do not seem to be too far apart. Here is what seems to be the ncover's format.

from minicover.

@lucaslorentz is there an easy way to parse (sed/grep) the total percentage of the cover from either the html or the xml reports?

from minicover.

@pkaramol My approach should work just fine if minicover outputs compatible NCover 3 xml reports.

It appears that some changes are needed to align current NCover reports with NCover 3. @lucaslorentz Is minicover open for contributions? I wouldn't have a problem working on this if needed.

@lucaslorentz If that is a direction you would prefer not to go in, we could consider a new approach based off of: https://docs.sonarqube.org/display/SONAR/Generic+Test+Data

from minicover.

@pkaramol @mcblair

@lucaslorentz is there an easy way to parse (sed/grep) the total percentage of the cover from either the html or the xml reports?
I think the easier would be to add or change minicover report implementation.

Sure, it is open for contributions, I do review PRs and merge them as son as I can.

The current NCover XML report is at:
https://github.com/lucaslorentz/minicover/blob/master/src/MiniCover/Reports/XmlReport.cs

It seems that we need to add more data to the current ncover xml. But we need to keep it compatible with ReportGenerator, as mentioned on the PR that created that report: #7

from minicover.

@pkaramol I definitely agree that supporting dotCover and OpenCover should be of interest, especially since they clearly have no intention of supporting anything past NCover 3.

My priority though, is to get coverage into SonarQube ASAP. I'm currently working on the work required for the xml reports to have parity with NCover 3 reports.

from minicover.

@pkaramol @mcblair
Please, check if the changes merged on #30 helps you to integrate it with SonarQube

from minicover.

@pkaramol
The best way to extract total coverage today is from console report.
Check this issue: #28

from minicover.

I am seeing a huge discrepancy between the report generated by minicover itself and the report generated by opencovercoverage.

For example I am seeing around ~22% reported to SonarQube when using the /d:sonar.cs.opencover.reportsPaths="opencovercoverage.xml" flag (this is the correct value however) and about 65% by minicover's html report.
65% (the erroneous value) is the one I am also seeing when asking for printing the report in ASCII in my job's trace: dotnet minicover report --workdir ../ --threshold 20

Can anyone else reproduce this?

update: this happens most likely because (for some reason), by default now the integrated reporting mechanism in minicover does not ignore test/**/*type of patterns ...

from minicover.

Are you specifying --sources that doesn't include test source files?

We had a bug related to source files filtering, it was fixed though: 111097f

Can you try updating to latest version: MiniCover 2.0.0-ci-20180301063918

Update:
You can also use --exclude-sources to remove some source files.

from minicover.

Here is the sequence of commands in my ci job: (all running in a docker container equipped with ms build scanner)

PROJECT_VERSION=<somehow greping version>
cp -r /tools . # copying the tools folder - I have already installed minicover
mono /msbuild/SonarQube.Scanner.MSBuild.exe begin /d:sonar.login=<my-sonar-login>  /d:sonar.host.url=<my-sonar-version> /v:$PROJECT_VERSION /k:<my-project-key> /d:sonar.cs.opencover.reportsPaths="opencovercoverage.xml" /d:sonar.coverage.exclusions=test/**
dotnet restore
dotnet build
cd tools
dotnet minicover instrument --workdir ../ --assemblies test/**/bin/**/*.dll --sources src/**/*.cs
dotnet minicover reset
cd ..
for project in test/**/*.csproj; do dotnet test --no-build $project; done
cd tools
dotnet minicover uninstrument --workdir ../
dotnet minicover opencoverreport --workdir ../ --threshold 20
dotnet minicover report --workdir ../ --threshold 20
cd ..
mono /msbuild/SonarQube.Scanner.MSBuild.exe end /d:sonar.login=<my-sonar-login>

Here is the All files line from dotnet minicover report --workdir ../ --threshold 20:

+--------------------------------------------------------------+-------+---------------+------------+
| All files                                                    | 16493 |     10793     | 65.439890% |
+--------------------------------------------------------------+-------+---------------+------------+
SonarQube Scanner for MSBuild 4.0.2

the above value is wrong because includes files as follows:

| test/TestsDir/File1cs                                      |    6   |     6         | 100.000000% |
| test/TestsDir/Functonality1Tests.cs                        |  379  |      379      | 100.000000% |
... (and so on...)

Here is what is reported to my sonarqube server: 21.6% (correct!)

Here is my tools.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <DotNetCliToolReference Include="MiniCover" Version="2.0.0-ci-20180226190451" />
  </ItemGroup>
</Project>

from minicover.