macro187 / testrunner Goto Github PK

View Code? Open in Web Editor NEW

20.0 4.0 11.0 381 KB

A console MSTest runner

License: MIT License

C# 100.00%

test-runner console mstest dotnet dotnet-core mono windows linux

testrunner's People

Contributors

Stargazers

Watchers

Forkers

m3talen jdeltorp lukaszchm owenniblock ming86 yk35 urs-meili-clx panhsu hosho alexandersurbashev basvo

testrunner's Issues

Add TestRunBeginEvent, TestRunEndEvent, and TestRunResults

For use by output format event handlers.

Multiple output formats support

Refactor TestRunner to support different output formats, with an initial "default" format producing the current format.

Fold `Results` into `Events`

Is there any reason for the Results classes to exist separately from their respective End events, or can they just be folded into them?

TestContext support

Hi,

I don't have the TestContext available in my Tests (e.g. in a [AssemblyInitialize] method) if I run the test with the testrunner.exe.

Is it possible to get the TestContext in another way or do you know how to fix the problem?

Many thanks!

Multiprocess not working on .NET Core

This is probably because TestRunner needs to be taught how to re-invoke itself using the dotnet command on .NET Core.

=================================
testrunner v1.0.0
Copyright (c) 2012-2018
Ron MacNeil, Rickenberg, JDeltorp
=================================

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

.config file redirection doesn't work on Mono

Originally thought this was impossible due to...
mono/mono#10584

...but there may be hope...
https://stackoverflow.com/a/39394998/2167919

Pure reflection test discovery

Don't depend on test framework assemblies (UnitTestFramework.dll, nunit.dll, etc.)

.trx output file format support

Requires #3 #20 #21

Add --method option

To run only particular test method(s).

TestContext property getter(s) not working on .NET Core

For some reason, .NET Core will not accept the dynamically generated TestContext overrides. .NET Framework seems to accept them.

=============================================================================================
Assembly: testrunner.Tests.MSTest/bin/Debug/netcoreapp2.0/publish/testrunner.Tests.MSTest.dll
=============================================================================================

Test Assembly:
C:\p\testrunner\testrunner.Tests.MSTest\bin\Debug\netcoreapp2.0\publish\testrunner.Tests.MSTest.dll

[AssemblyInitialize] AssemblyInitialize()

An internal error occurred in testrunner.dll:
Method 'get_Properties' in type 'TestContextProxy' from assembly 'TestContextProxyAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.
Type: System.TypeLoadException
Source: System.Private.CoreLib
StackTrace:
  at System.Reflection.Emit.TypeBuilder.TermCreateClass(RuntimeModule module, Int32 tk, ObjectHandleOnStack type)
  at System.Reflection.Emit.TypeBuilder.CreateTypeNoLock()
  at System.Reflection.Emit.TypeBuilder.CreateTypeInfo()
  at TestRunner.Domain.TestContextProxy.GetTypeFromTypeBuilder(TypeBuilder typeBuilder)
    in C:\p\testrunner\testrunner\Domain\TestContextProxy.cs:line 189
  at TestRunner.Domain.TestContextProxy.BuildProxy()
    in C:\p\testrunner\testrunner\Domain\TestContextProxy.cs:line 59
  at TestRunner.Domain.TestContextProxy.get_Proxy()
    in C:\p\testrunner\testrunner\Domain\TestContextProxy.cs:line 26
  at TestRunner.Program.Program.RunMethod(MethodInfo method, Object instance, Boolean takesTestContext, Type expectedException, Boolean expectedExceptionAllowDerived, String prefix)
    in C:\p\testrunner\testrunner\Program\Program.cs:line 509
  at TestRunner.Program.Program.RunTestAssembly(String assemblyPath)
    in C:\p\testrunner\testrunner\Program\Program.cs:line 198
  at TestRunner.Program.Program.Main2(String[] args)
    in C:\p\testrunner\testrunner\Program\Program.cs:line 59

Add --class option

To run only particular test class(es).

Original request:

Could you add some arguments to the command line to limit the run to any/some testclass or any/some testmethod

ex:

testrunner.exe C:\Path\To\TestAssembly.dll C:\Path\To\AnotherTestAssembly.dll -c "xxxns.yyyyns.MyTestClass1,xxxns.zzzns.MyTestClass2" -m "xxxns.zzzns.MyTestClass3.TestingMethod1,......"

Research whether it's possible for assembly redirects in switched config files to work

If not, can it be improved?

NUnit test support

Research correct behaviour on exceptions in initialize and cleanup methods

Find out what MS runner behaviour is, add tests for it, and adjust TestRunner's behaviour if necessary.

TestRunDirectory support

Figure out how this normally works in MSTest, then implement it and surface appropriate information in TestContext.

cc @eczimmermann

ResultAccumulatingEventHandler

For building event handlers that need to remember all the results from a test run.

Remove assembly reference(s) from test suites to TestRunner

All tests currently fail with the following exception, likely due to how per-test-assembly child processes are being run. Does not occur on Windows.

Could not resolve type with token 01000008
  Type: System.TypeLoadException
  StackTrace:
    at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
    at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038]
      in <f712f98eb8e445c8918edaf595bbe465>:0

xUnit test support

Mono automated builds and tests

Support running multiple test assemblies in a single TestRunner invocation

Hi,

do you plan to support testing multiple assemblies at once (either by wildcard or multiple parameters)?

Thanks.

Move non-essential logic out of runners into event handlers

Runners are still doing too much, e.g. results tabulation, "success" decision logic, and maybe more.

Reify method, test, class, and assembly result information as types in a new Results namespace.

Teach *End events to carry results using those types.

Shift logic that aggregates finer-grained event(s) into *End results into a new ResultBuilderEventHandler or similar.

???

Auto-discover tests in solution

If no test DLL's specified on command line, auto-discover them in some reasonable way

Not compatible with System.Reactive.Interfaces, Version=3.1.1

Test Runner gives exception when used with System.Reactive.Interfaces, Version=3.1.1. The exception stack trace gives information that it is looking for 'System.Reactive.Interfaces, Version=3.0.0.0".