yonderblue / sharpcover Goto Github PK

View Code? Open in Web Editor NEW

32.0 4.0 17.0 758 KB

C# code coverage tool with Linux (Mono) and Windows (.NET 4.0) support.

License: MIT License

C# 96.40% Shell 1.30% Batchfile 2.30%

sharpcover's Introduction

#SharpCover

C# code coverage tool with Linux (Mono) and Windows (.NET 4.0) support.

##Features

CIL instruction coverage
Namespace, class, method, line and instruction inclusions/exclusions
Inclusions/Exclusions specifications are outside of code.
Cross platform Linux/Windows by way of Cecil
Easy integration into builds (target user program is invoked seperately)

##Usage

After building run SharpCover.exe instrument json where json is a string or file with contents that reflects the following format, most options are optional:

{
    "assemblies": ["../anAssembly.dll", "/someplace/anotherAssembly.dll"],
    "typeInclude": ".*SomePartOfAQualifiedTypeName.*",
    "typeExclude": ".*obviouslyARegex.*",
    "methodInclude": ".*SomePartOfAQualifiedMethodName.*",
    "methodExclude": ".*obviouslyARegex.*",
    "methodBodyExcludes": [
        {
            "method": "System.Void Type::Method()",
            "offsets": [4, 8],
            "lines": ["line content", "++i;"]
        }
    ]
}

The exit code will be zero on instrument success.

Excercise the assemblies you listed in the config.
Afterwards run SharpCover.exe check in the same directory you ran instrument. The results will be in coverageResults.txt, with missed instructions prefixed with MISS !. The exit code will be zero for success, and total coverage percentage is printed.

###Notes Full method names for methodBodyExcludes can be found in the output, as well as offsets.

The methodBodyExcludes by lines are line content matches ignoring leading/trailing whitespace. This keeps coverage exclusions outside the code while not relying on offsets which can easily change if new code is added to the method. For excluding instructions by line that have no source, the last instruction to have a sequence point is used as that instructions "line".

Remember to rebuild your assemblies before you instrument again !

It is highly recommended to use the includes/excludes to achieve a zero exit from check, otherwise you are cheating yourself !

##Tool Build

Make sure you are in the repository root.

###Linux

Make sure Mono which comes with xbuild is installed.

xbuild Gaillard.SharpCover/Program.csproj

###Windows

Make sure .NET SDK which comes with MSBuild is installed.

C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe Gaillard.SharpCover\Program.csproj

Navigate to the Gaillard.SharpCover/bin/Debug directory where the SharpCover.exe executable can be used.

##Contact

Developers may be contacted at:

Questions / Feedback / Feature requests are welcome !!

##Project Build

Make sure you are in the repository root. Make sure nunit-console is installed.

###Linux

Make sure Mono which comes with xbuild is installed.

sh build.sh

###Windows

Make sure .NET SDK which comes with MSBuild is installed.

build.bat

#####Notes

Some paths might need changing depending on your environment.

##Enhancements

A standard output format that can be used with available visualizers would be very useful.

A more complete test suite.

Contributions welcome !

sharpcover's People

Contributors

Stargazers

Watchers

Forkers

jcline jdeering galaxiaguy aallfredo maciejczechowski izrik nextmunich liulinc keilalex mcintyre94 jschmid vmuriart ewhitmore leomehlig joxno vikashkaruna

sharpcover's Issues

Installable package

I'm looking at using SharpCover to add support to my coveralls.io publisher, and need a way to install it on my appveyor and travis-ci builds. Is it possible to get this as a nuget package that I can install in my projects?

Output as XML or JSON

Hey! I'm looking for a way to output the coverage as json or xml which contains mapping of line coverage numbers and their hit value. Is there such an output?

The current output format, from what I can tell, does not contain enough details to map coverage reports back to source code.

Looking to make this example repository work: https://github.com/codecov/example-csharp-sharpcover

coreclr

Hi,
I'm try to test some assemblies generated with coreclr, using dnu build. They have a pdb rather than an mdb. Is there any way to get support for this?

Thanks,
Chris

No license

Would be nice if there were a license for this to legitimize our using it. :)

Custom results output

From the readme:

A standard output format that can be used with available visualizers would be very useful.

Right now I have a janky local modification we're using to integrate with http://phabricator.org/, but it would be nice to refactor this into something less hacky.

I was thinking something like this would work:

public enum CoverageType {
  Hit,
  Miss
}

public struct Result {
  public int startLine;
  public int endLine;
  public string path;
  public CoverageType coverage;
}

public interface IResults {
  void Write(IList<Result> results);
}

Then users can add their own IResults and it would be simple to add a configuration option to coverage.json.

If this seems reasonable to you then I'd be happy to write it and put up a pull request?

Instrumenting and running tests with multiple assemblies causes tests to fail

I've been refactoring our unit tests in order to separate them into multiple assemblies (which makes more sense for our architecture).

NUnit supports running tests on multiple assemblies at once by simply providing the path to multiple dlls. We currently have two such test assemblies. Both of them reference one shared library and one of them references a different library. There are three library dlls in total, an example:

project/
  coverage.json
  library1/
  library2/
  tests1/
    bin/
      library1.dll
      tests1.dll
  tests2/
    bin
      library1.dll
      library2.dll
      test2s.dll

We want to have complete coverage results for library1.dll, so coverage.json looks like:

{
    "assemblies": [
        "tests1/bin/library1.dll, 
        "tests2/bin/library1.dll", 
        "tests2/bin/library2.dll", 
    ],
    "typeExclude": ".*System.*",
    "methodExclude": ".*System.*",
}

We then run things like so, from project/

mono --debug /path/to/SharpCover.exe instrument coverage.json
mono --debug /path/to/nunit-console.exe -labels -nologo tests1/bin/tests1.dll tests2/bin/tests2.dll
mono --debug /path/to/SharpCover.exe check

We get this failure when running the tests for what appears to be every test in tests2.dll:

Errors and Failures:
1) SetUp Error : ________GlobalTestSetup
   SetUp : System.TypeInitializationException : An exception was thrown by the type initializer for Hyena.Log
  ----> System.IO.IOException : Sharing violation on path /home/jcline/src/mobile/coverageHits.txt
  at ________GlobalTestSetup.Setup () [0x00000] in <filename unknown>:0 
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00000] in <filename unknown>:0 
--IOException
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x00000] in <filename unknown>:0 
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share) [0x00000] in <filename unknown>:0 
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at System.IO.File.Open (System.String path, FileMode mode) [0x00000] in <filename unknown>:0 
  at Gaillard.SharpCover.Counter.Count (System.String path, Int32 index) [0x00000] in <filename unknown>:0

I don't currently have any great suggestions for fixing this and wanted to see what your opinions are?

Fails on macOS

When trying to run on macOS, I get the following error:

$ mono ../SharpCover/Gaillard.SharpCover/bin/Debug/SharpCover.exe instrument ./coverageConfig.json
Mono.Cecil.AssemblyResolutionException: Failed to resolve assembly: 'System.Runtime, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
  at Mono.Cecil.BaseAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name, Mono.Cecil.ReaderParameters parameters) <0x31f6e70 + 0x002bb> in <filename unknown>:0 
  at Mono.Cecil.BaseAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name) <0x31f6e30 + 0x00038> in <filename unknown>:0 
  at Mono.Cecil.DefaultAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name) <0x31f6d08 + 0x00087> in <filename unknown>:0 
  at Mono.Cecil.MetadataResolver.Resolve (Mono.Cecil.TypeReference type) <0x31f6a48 + 0x000b2> in <filename unknown>:0 
  at Mono.Cecil.ModuleDefinition.Resolve (Mono.Cecil.TypeReference type) <0x31f6710 + 0x00027> in <filename unknown>:0 
  at Mono.Cecil.TypeReference.Resolve () <0x31f66a0 + 0x0003b> in <filename unknown>:0 
  at Mono.Cecil.Mixin.CheckedResolve (Mono.Cecil.TypeReference self) <0x31c85b0 + 0x00011> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.GetConstantType (Mono.Cecil.TypeReference constant_type, System.Object constant) <0x31d7748 + 0x00077> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.GetConstantType (Mono.Cecil.TypeReference constant_type, System.Object constant) <0x31d7748 + 0x00127> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddConstant (IConstantProvider owner, Mono.Cecil.TypeReference type) <0x31d75f8 + 0x0002b> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddParameter (UInt16 sequence, Mono.Cecil.ParameterDefinition parameter, Mono.Cecil.ParamTable table) <0x31d5620 + 0x00117> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddParameters (Mono.Cecil.MethodDefinition method) <0x31d5450 + 0x000ef> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddMethod (Mono.Cecil.MethodDefinition method) <0x31d3008 + 0x00163> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddMethods (Mono.Cecil.TypeDefinition type) <0x31d2f98 + 0x0003f> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddType (Mono.Cecil.TypeDefinition type) <0x31cf890 + 0x0023f> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.AddTypeDefs () <0x31cf818 + 0x00047> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.BuildTypes () <0x31cec58 + 0x0002f> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.BuildModule () <0x31cbc48 + 0x00123> in <filename unknown>:0 
  at Mono.Cecil.MetadataBuilder.BuildMetadata () <0x31cbc08 + 0x00013> in <filename unknown>:0 
  at Mono.Cecil.ModuleWriter.<BuildMetadata>b__0 (Mono.Cecil.MetadataBuilder builder, Mono.Cecil.MetadataReader _) <0x31cbbe8 + 0x00013> in <filename unknown>:0 
  at Mono.Cecil.ModuleDefinition.Read[TItem,TRet] (Mono.Cecil.TItem item, System.Func`3 read) <0x7c81d8 + 0x00035> in <filename unknown>:0 
  at Mono.Cecil.ModuleWriter.BuildMetadata (Mono.Cecil.ModuleDefinition module, Mono.Cecil.MetadataBuilder metadata) <0x31cbb18 + 0x000ab> in <filename unknown>:0 
  at Mono.Cecil.ModuleWriter.WriteModuleTo (Mono.Cecil.ModuleDefinition module, System.IO.Stream stream, Mono.Cecil.WriterParameters parameters) <0x2635bd0 + 0x001e3> in <filename unknown>:0 
  at Mono.Cecil.ModuleDefinition.Write (System.IO.Stream stream, Mono.Cecil.WriterParameters parameters) <0x2635b38 + 0x00063> in <filename unknown>:0 
  at Mono.Cecil.ModuleDefinition.Write (System.String fileName, Mono.Cecil.WriterParameters parameters) <0x2635aa0 + 0x0004f> in <filename unknown>:0 
  at Mono.Cecil.AssemblyDefinition.Write (System.String fileName, Mono.Cecil.WriterParameters parameters) <0x2635a68 + 0x00023> in <filename unknown>:0 
  at Gaillard.SharpCover.Program.Instrument (System.String assemblyPath, Gaillard.SharpCover.InstrumentConfig config, System.IO.TextWriter writer, System.Int32& instrumentIndex) <0x589e98 + 0x0013f> in <filename unknown>:0 
  at Gaillard.SharpCover.Program.Main (System.String[] args) <0x57dee0 + 0x00157> in <filename unknown>:0

Exception when running with no arguments

Cloned 08e888a.

Ran xbuild Gaillard.SharpCover/Program.csproj
cd Gaillard.SharpCover/bin/Debug/
$ mono SharpCover.exe
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at Gaillard.SharpCover.Program.Main (System.String[] args) <0x40032d60 + 0x00059> in :0

$ mono --debug SharpCover.exe
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at Gaillard.SharpCover.Program.Main (System.String[] args) [0x00002] in ./git/SharpCover/Gaillard.SharpCover/Program.cs:257

yonderblue / sharpcover Goto Github PK

sharpcover's Introduction

sharpcover's People

Contributors

Stargazers

Watchers

Forkers

sharpcover's Issues

Installable package

Output as XML or JSON

coreclr

No license

Custom results output

Instrumenting and running tests with multiple assemblies causes tests to fail

Fails on macOS

Exception when running with no arguments

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent