Git Product home page Git Product logo

meissa's Introduction

Build status Join the chat at https://gitter.im/meissarunner/Lobby Overview ChangeLog download

Meissa Description

MEISSA is a distributed tests runner. It is built using the latest technologies such as .NET Core, ASPNET.Core MVC and more.

It is completely cross-platform able to run tests on Windows, Linux and OSX.

It is designed to be programming language agnostic which means that it can run tests written on different languages.

Right now it can run .NET Framework and .NET Core tests, but it is written in a way that quickly we can add support for Java and other languages and runners.

Features

MEISSA Features

Parallel on Agents

The tests execution tremendously speeds up from the distribution on multiple machines. However, it can be further improved by executing the tests in parallel on the tests agents. It can parallelize the tests even in a single container.

Cross-Platform

Runs tests on Windows, OSX, and Linux.

Smart Tests Balancing

Smartly balances the tests across remote machines based on previous tests execution times which can significantly improve the overall tests runs’ time.

Failing Tests Retry

Retries the failing tests multiple times to see whether there are real problems in the system under tests or some random environment issue occurred during the run. In the end, the tool produces single tests results file. You can specify a threshold %. If more than the specified number of tests are failing, they won't be retried.

Custom Parameters

Supports passing custom data to your tests through the runner. Like unique CI build number or some specific folder. You will be able to access the data from your tests through environmental variables.

Built-in Extensibility

Executes custom logic without modifying the source code. Offers plug-in API.

Test Framework and Programming Language Agnostic

Runs tests written on different test frameworks no matter the language- C#, Java, etc. Offers plug-in API.

Single CLI

Single common command-line interface. No complex installations or configurations.

Safe Test Run Abortion

Keeps the clean state of agents and fast test run abortion.

How Do We Use It?

  1. Start MEISSA server
meissaserver

To coordinate all agents and runners MEISSA has its own server with its own DB.

By the way in order to run MEISSA you don’t even need to install .NET Core since or other dependencies it is completely portable. You just download it as zip, unzip it and that is. Everything you need, runners, agents, servers, DBs is there.

  1. Start MEISSA test agent
meissa agent --tag="APIAgent" --server="http://IPServerMachine:89"

Usually, you have one test agent per machine. So, more machines you have, faster your tests will be executed.

  1. Start MEISSA tests runner
meissa runner --results="pathToResults\result.trx" 
--tag="APIAgent" --testTechnology="MSTest" 
--library="pathToBuildedFiles\SampleTestProj.dll" --server="http://IPServerMachine:89"

Usually, you start the runner from CI job. The typical workflow will be. Download the tests source code. Build it. Execute tests with MEISSA. Publish the results produced by MEISSA.

Advanced Parameters

--nativeRunnerArguments="--verbosity="detailed""

This argument will pass the value to the native tests runner. Here we tell the .NET Core tests runner to produce detailed log.

--retriedResults="pathToResults\retriedResult.trx" --retries=3 --threshold=90

With this one we tell MEISSA to retry the failed tests three times if less than 5% of all tests have failed.

--filter="test.FullName != \"TestName\" AND !test.Categories.Contains(\"CI\")"

MEISSA has built-in complex test filter parser and we can write complex queries to filter the tests.

--customArguments="BuildNumber=42" 

Sometimes it is useful to pass data to tests from the runner. For example, you may need to pass the current build number so that you can create a folder in your tests. If you use this argument each agent will create an environmental variable with the name BuildNumber and it will assign the value 42. After that, it is an easy job to get the value from your tests.

--timeBased

Instructs MEISSA to balance the tests not based on the count but rather than on the previous execution times of the tests. To use it you need to execute all of your tests at least one time. This is quite useful because some of your UI tests may execute for 1 minute or more but most of them for 30 seconds or less. As I previously mentioned, the whole tests execution time is equal to the slowest sub-run. This feature can sometimes drastically decrease the execution time.

--parallelRun

Instructs MEISSA to execute in parallel the tests on each agent. You can even specify how many processes to be spawn. This is most useful for unit, API or headless UI tests.

--sameMachineByClass

The tests from a single class will be executed on the same machine.

Getting Started

Open CLI and execute

dotnet tool install --global meissaserver

This will install the MEISSA server as a global tool.

dotnet tool install --global meissa

This will install the MEISSA agent + runner as a global tool.

To update to the latest version execute:

dotnet tool update --global meissaserver
dotnet tool update --global meissa

Note: The new update feature is written in a way to preserve the data important for distributing your tests- like previous test execution times.

Get Involved

  • Report bugs
  • Suggest new features
  • Help us write more test runners
  • Add more unit tests
  • Optimize the performance
  • If you have any questions or suggestions you can post them to our forum.

meissa's People

Contributors

angelovstanton avatar gitter-badger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

meissa's Issues

Meissa Getting Started Documentation

Regarding the install instructions using .NET Core templates and the package name:

Execute dotnet new -i Meissa.Windows (this will install the template)/
Execute dotnet new Meissa.Windows (this will extract all tool’s files to the folder)

The instructions should be updated as the nuget package name is Meissa.Windows64.

Thanks,
Howard

Exception Error

Environment: VS 2017, xUnit, netcore2.1.

Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0

Thinking that I'm missing the assembly from my project, I NuGet version 15, but the process fails with a "downgrade" message...rolling back etc...(The current version of Microsoft.VisualStudio.TestPlatform.ObjectModel is 16.2.0.0)

So what am I missing or is this an issue with Meissa?

Thanks,

Created by Bellatrix Ltd.
http://bellatrix.solutions
Test Execution started....
System.AggregateException: One or more errors occurred. (Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.) ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
at System.ModuleHandle.ResolveMethod(RuntimeModule module, Int32 methodToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount)
at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, CustomAttributeRecord caRecord)
at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeType target)
at Meissa.Plugins.MSTest.NativeTestsRunnerTestCasesPluginService.ExtractAllTestCasesFromTestLibrary(String testLibraryPath)
at Meissa.Core.Services.TestExecutionService.ExecuteAsync(TestRunSettings testRunSettings) in D:\SourceCode\Meissa\Meissa.Core.Services\TestExecutionService.cs:line 100
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Meissa.Program.ExecuteTestRun(RunnerModeOptions runnerModeOptions) in D:\SourceCode\Meissa\Meissa\Program.cs:line 223
---> (Inner Exception #0) System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at System.ModuleHandle.ResolveMethod(RuntimeModule module, Int32 methodToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount)
at System.ModuleHandle.ResolveMethodHandleInternalCore(RuntimeModule module, Int32 methodToken, IntPtr[] typeInstantiationContext, Int32 typeInstCount, IntPtr[] methodInstantiationContext, Int32 methodInstCount)
at System.ModuleHandle.ResolveMethodHandleInternal(RuntimeModule module, Int32 methodToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
at System.Reflection.CustomAttributeData..ctor(RuntimeModule scope, CustomAttributeRecord caRecord)
at System.Reflection.CustomAttributeData.GetCustomAttributes(RuntimeModule module, Int32 tkTarget)
at System.Reflection.CustomAttributeData.GetCustomAttributesInternal(RuntimeType target)
at Meissa.Plugins.MSTest.NativeTestsRunnerTestCasesPluginService.ExtractAllTestCasesFromTestLibrary(String testLibraryPath)
at Meissa.Core.Services.TestExecutionService.ExecuteAsync(TestRunSettings testRunSettings) in D:\SourceCode\Meissa\Meissa.Core.Services\TestExecutionService.cs:line 100

Database Error updating Meissa.Model.TestsRunsContext

I'm running a set of chrome tests (around 600) in parallel across 3 workers and get the following error on the Meissa Server:

fail: Microsoft.EntityFrameworkCore.Update[10000]
An exception occurred in the database while saving changes for context type 'Meissa.Model.TestsRunsContext'.
Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: 'database is locked'.
at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
at Microsoft.Data.Sqlite.SqliteConnectionExtensions.ExecuteNonQuery(SqliteConnection connection, String commandText)
at Microsoft.Data.Sqlite.SqliteTransaction..ctor(SqliteConnection connection, IsolationLevel isolationLevel)
at Microsoft.Data.Sqlite.SqliteConnection.BeginTransaction(IsolationLevel isolationLevel)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransactionWithNoPreconditions(IsolationLevel isolationLevel)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransactionAsync(IsolationLevel isolationLevel, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransactionAsync(CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(DbContext _, ValueTuple2 parameters, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IReadOnlyList1 entriesToSave, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: 'database is locked'.
at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
at Microsoft.Data.Sqlite.SqliteConnectionExtensions.ExecuteNonQuery(SqliteConnection connection, String commandText)
at Microsoft.Data.Sqlite.SqliteTransaction..ctor(SqliteConnection connection, IsolationLevel isolationLevel)
at Microsoft.Data.Sqlite.SqliteConnection.BeginTransaction(IsolationLevel isolationLevel)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransactionWithNoPreconditions(IsolationLevel isolationLevel)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransactionAsync(IsolationLevel isolationLevel, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.BeginTransactionAsync(CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(DbContext _, ValueTuple2 parameters, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IReadOnlyList1 entriesToSave, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)

Error running tests in parallel ... Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware

I'm running a set of chrome tests (around 600) in parallel across 3 workers and get the following error on the Meissa Server:

fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Reading the request body timed out due to data arriving too slowly. See MinRequestBodyDataRate.
at Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException.Throw(RequestRejectionReason reason)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1MessageBody.PumpAsync()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody.ReadAsync(Memory1 buffer, CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory1 buffer, CancellationToken cancellationToken)
at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
at Microsoft.AspNetCore.WebUtilities.StreamHelperExtensions.DrainAsync(Stream stream, ArrayPool1 bytePool, Nullable1 limit, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.JsonInputFormatter.ReadRequestBodyAsync(InputFormatterContext context, Encoding encoding)
at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(ModelBindingContext bindingContext)
at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(ActionContext actionContext, IModelBinder modelBinder, IValueProvider valueProvider, ParameterDescriptor parameter, ModelMetadata metadata, Object value)
at Microsoft.AspNetCore.Mvc.Internal.ControllerBinderDelegateProvider.<>c__DisplayClass0_0.<g__Bind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.InvokeInnerFilterAsync()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeNextResourceFilter()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow(ResourceExecutedContext context)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeFilterPipelineAsync()
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.InvokeAsync()
at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)

Nuget package missing

Hi Anton,
I was attempting to create a plugin needed for xUnit and per the instructions to create a new plugin:

Steps to Write New Plugin
Create a new C# .NET Core/.NET Standard library project
Install Meissa.Plugins.Contracts NuGet Package

I go an error message:

NU1101: Unable to find package Meissa.Core.Model. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages, nuget.org
Nuget package missing error

Thanks,
Howard

Tests not running across machines

I have 3 machines, one server, one worker and one runner. I get the following error on the Worker and runner...

Test agent with tag APIAgent starts tests execution on machine WORKER.
NUnit Console Runner 3.7.0
Copyright (c) 2017 Charlie Poole, Rob Prouse
Runtime Environment
OS Version: Microsoft Windows NT 10.0.14393.0
CLR Version: 4.0.30319.42000
Test Files
C:\Users\XXX\AppData\Local\Temp\7\8fb689fc-f5f4-4318-a9a6-9140c7a56716\TestAutomation.Dashboard.Tests.dll
C:\Tests\packages
Errors, Failures and Warnings

  1. Invalid : C:\Users\XXX\AppData\Local\Temp\7\8fb689fc-f5f4-4318-a9a6-9140c7a56716\TestAutomation.Dashboard.Tests.dll
    File not found: C:\Users\XXX\AppData\Local\Temp\7\8fb689fc-f5f4-4318-a9a6-9140c7a56716\TestAutomation.Dashboard.Tests.dll
  2. Invalid : C:\Tests\packages
    File not found: C:\Tests\packages
    Test Run Summary
    Overall result: Failed
    Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
    Start time: 2018-10-09 11:15:40Z
    End time: 2018-10-09 11:15:41Z
    Duration: 0.284 seconds
    Results (nunit3) saved as C:\Users\XXX\AppData\Local\Temp\7\tmpCC63.tmp
    TEST AGENT RUN COMPLETED

Custom NUnit Test tags not supported.

Hello,

In Meissa project "Meissa.Plugins.NUnit" there is a method, which extract tests cases from library named "ExtractAllTestCasesFromTestLibrary". It extracts tests, which -> x.GetType().FullName.Equals(NunitTestAttributeName). In my tests project I have custom Test Tag named [CustomTest] and [CustomTestCase(string arg1, int arg2... etc)]. When i start runner, it extracts testsCases from library, and in my case it returns 0 cases. Temp file with TestsLists is empty. When i changed for a moment Tags from [CustomTest] to standard [Test] it works good.
Another question is related with [TestCase] tag. Is it supported in extracting cases from the library? Because there is a constant field with namespace only for Test tag, not for TestCase tag.

Support for Docker

We can add public Docker images for the agent and server. Also, give guidelines how people to use them.

[Question] Is there a posibility, to run tests on multiple RDP connections.

Hello,

In Meissa runner main goal is to run tests on a single machine - multiple tests in parallel, multiple machines with a single test or both in one. In my case, I want to run multiple test agents on one machine but each on a separate RDP connection.
Is it much effort to change it in code? I made some research and see some problems with it, but I am not sure how much it "costs" to change. My application is opened by multiple users, which connect to one PC by RDP connection, so I want to make tests with a similar situation.

Infinite recursion during resource lookup within System.Private.CoreLib

Hi.

I have tried to start our API test on multiple machines.
Setup: 1 test agent, 1 test server (different machnines) (Windows)

Test start command:

PS C:\Users\kudrin.CORP\Downloads\Meissa> .\Meissa.exe runner --resultsFilePath="e:\Temp\MeissaResults\result.trx" --outputFilesLocation="E:\Repo\BCKAPI\mdm.webapi.mdmonli ne.integrationtest\Mdm.WebApi.MdmOnline.IntegrationTest.Tests\bin\Debug\" --agentTag="MeissaAgent" --testTechnology="NUnit" --testLibraryPath="E:\Repo\BCKAPI\mdm.webapi.md monline.integrationtest\Mdm.WebApi.MdmOnline.IntegrationTest.Tests\bin\Debug\Mdm.WebApi.MdmOnline.IntegrationTest.Tests.dll" --runInParallel --nativeRunnerArguments="--wor kers=10" --testServerUrl="http://msk-ws27018:5000"

During the startup followed error happend:

Created by Bellatrix Ltd. http://bellatrix.solutions Test Execution started.... FROM TestRunnerPluginService ExecutePreTestRunLogic $$$$$$$$$$$$$$$ Assert Failure Expression: [Recursive resource lookup bug] Description: Infinite recursion during resource lookup within System.Private.CoreLib. This may be a bug in System.Private.CoreLib, or potentially in certain extensibility points such as assembly resolve events or CultureInfo names. Resource name: ArgumentNull_Generic Stack Trace: at System.SR.InternalGetResourceString(String key) at System.SR.GetResourceString(String resourceKey, String defaultString) at System.ArgumentNullException..ctor(String paramName) at System.Runtime.Loader.AssemblyLoadContext.GetLoadContext(Assembly assembly) at System.Reflection.Assembly.LoadFromResolveHandler(Object sender, ResolveEventArgs args) at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName) at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks, IntPtr ptrLoadContextBinder) at System.Reflection.RuntimeAssembly.InternalGetSatelliteAssembly(String name, CultureInfo culture, Version version, Boolean throwOnFileNotFound, StackCrawlMark& stackM ark) at System.Resources.ManifestBasedResourceGroveler.GetSatelliteAssembly(CultureInfo lookForCulture, StackCrawlMark& stackMark) at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(CultureInfo culture, Dictionary2 localResourceSets, Boolean tryParents, Boolean createIfNotExist
s, StackCrawlMark& stackMark)
at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo requestedCulture, Boolean createIfNotExists, Boolean tryParents, StackCrawlMark& stackMark)
at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo culture, Boolean createIfNotExists, Boolean tryParents)
at System.Resources.ResourceManager.GetString(String name, CultureInfo culture)
at System.SR.InternalGetResourceString(String key)
at System.SR.GetResourceString(String resourceKey, String defaultString)
at System.ArgumentNullException..ctor(String paramName)
at System.Runtime.Loader.AssemblyLoadContext.GetLoadContext(Assembly assembly)
at System.Reflection.Assembly.LoadFromResolveHandler(Object sender, ResolveEventArgs args)
at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark,
IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks, IntPtr ptrLoadContextBinder)
at System.Reflection.RuntimeAssembly.InternalGetSatelliteAssembly(String name, CultureInfo culture, Version version, Boolean throwOnFileNotFound, StackCrawlMark& stackM
ark)
at System.Resources.ManifestBasedResourceGroveler.GetSatelliteAssembly(CultureInfo lookForCulture, StackCrawlMark& stackMark)
at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(CultureInfo culture, Dictionary2 localResourceSets, Boolean tryParents, Boolean createIfNotExist s, StackCrawlMark& stackMark) at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo requestedCulture, Boolean createIfNotExists, Boolean tryParents, StackCrawlMark& stackMark) at System.Resources.ResourceManager.InternalGetResourceSet(CultureInfo culture, Boolean createIfNotExists, Boolean tryParents) at System.Resources.ResourceManager.GetString(String name, CultureInfo culture) at System.SR.InternalGetResourceString(String key) at System.SR.GetResourceString(String resourceKey, String defaultString) at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(AssemblyName assemblyName) at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName) at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) at System.Reflection.Assembly.GetTypes() at Meissa.Plugins.NUnit.NativeTestsRunnerTestCasesPluginService.ExtractAllTestCasesFromTestLibrary(String testLibraryPath) at Meissa.Core.Services.TestExecutionService.<ExecuteAsync>d__19.MoveNext() in D:\SourceCode\AutomateThePlanet\Meissa\Meissa.Core.Services\TestExecutionService.cs:line 89 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask) at System.Threading.Tasks.Task.RunContinuations(Object continuationObject) at System.Threading.Tasks.Task1.TrySetResult(TResult result)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetExistingTaskResult(TResult result)
at Meissa.Core.Services.TestRunProvider.d__7.MoveNext() in D:\SourceCode\AutomateThePlanet\Meissa\Meissa.Core.Services\TestRunProvider.cs:line 72

at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask)
at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
at System.Threading.Tasks.Task1.TrySetResult(TResult result) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetExistingTaskResult(TResult result)
at Meissa.API.Client.RestClientRepository1.<CreateAsync>d__12.MoveNext() in D:\SourceCode\AutomateThePlanet\Meissa\Meissa.API.Client\DataSetupWebApi.cs:line 51 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask) at System.Threading.Tasks.Task.RunContinuations(Object continuationObject) at System.Threading.Tasks.Task1.TrySetResult(TResult result)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetExistingTaskResult(TResult result) at Meissa.API.Client.HttpClientExtensions.<SendAsyncWithRetry>d__0.MoveNext() in D:\SourceCode\AutomateThePlanet\Meissa\Meissa.API.Client\HttpClientExtensions.cs:line 5 6 at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask) at System.Threading.Tasks.Task.RunContinuations(Object continuationObject) at System.Threading.Tasks.Task1.TrySetResult(TResult result)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetExistingTaskResult(TResult result) at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext() at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask) at System.Threading.Tasks.Task.RunContinuations(Object continuationObject) at System.Threading.Tasks.Task1.TrySetResult(TResult result)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetExistingTaskResult(TResult result) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetResult(Task1 completedTask) at System.Net.Http.HttpContent.<LoadIntoBufferAsyncCore>d__48.MoveNext() at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(Action action, Boolean allowInlining, Task& currentTask) at System.Threading.Tasks.Task.RunContinuations(Object continuationObject) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot) at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(Task task, Boolean taskWasPreviouslyQueued) at System.Threading.Tasks.TaskScheduler.TryRunInline(Task task, Boolean taskWasPreviouslyQueued) at System.Threading.Tasks.TaskContinuation.InlineIfPossibleOrElseQueue(Task task, Boolean needsProtection) at System.Threading.Tasks.Task.RunContinuations(Object continuationObject) at System.Threading.Tasks.Task1.TrySetResult(TResult result)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetExistingTaskResult(TResult result) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1.SetResult(Task`1 completedTask)
at System.Net.Http.WinHttpResponseStream.d__18.MoveNext()
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
at System.Threading.ThreadPoolWorkQueue.Dispatch()
PS C:\Users\kudrin.CORP\Downloads\Meissa>

`

And it seems that 'nunitConsoleExePath' in 'meissaSettings.json' file wasn't described in documentation

`
{
"testAgent": {
.....

    "nunitConsoleExePath": "C:\\NUnit\\nunit-console\\nunit3-console.exe"

....
},
`

Question about parameter --timeBased

I have a question for the --timeBased parameter.
If I use this parameter with the --parallelRun parameter, does it sort the tests by runtime?

Does it run the long running tests in parallel first and after them the tests, which are faster?

Support for Java- TestNG and JUnit

Since the creation of the tool, there were tons of requests to create support for Java projects.

Since I am not a Java developer, I will need some help with the Java specifics. I will code everything related to the Meissa plugins.
Before I can define what help I will need, let's me first give a short explanation of how Meissa run tests written in different technologies.

The tool is architectured in a way to use plugins (DLLs placed in the Plugins folder), we utilise the MEF framework for the job. We have two C# interfaces that the plugin project should implement. One for extracting the tests from the library files (DLLs, jars, js) and one more for executing the tests from CMD, parsing the test results files, merge the results, extract pass/failed tests (we use that for the retry logic).

So, as far as I know, there are two major test frameworks for Java- TestNG and JUnit. I want to support both. Are there any other major ones that I didn't mention?

I guess we will need two separate plugins for TestNG and JUnit.
What will I need for each technology?

  1. Test project with few classes and tests inside them
  2. Explanation what I will need to install on my machine to run tests
  3. Example how to run the tests from the command line
  4. Explanation how to filter tests from CMD (we need that to execute only the failed tests for the retry and also to execute a subset of tests on each test agent)
  5. Explanation how to specify the name and location of the test result files
  6. Not sure how but maybe we will need separate Java console app that extracts- test names, test classes and save them to a file (later we can read this file in C# and do the rest from there)

TestsFilter arguments not working on NUnit Tests

Hello,

In my tests i wanted to use argument --testsFilter to pick tests from 2 categories eg.:
--testsFilter="test.Categories.Contains(\"Cat1\") AND test.Categories.Contains(\"Cat2\")".

When I use an argument like this, my testSuite is not filtered to tests with these two categories. It starts to run all tests from an assembly. I must use Test Selection Language from NUnit passed in --nativeRunnerArguments argument:
--where cat==Cat1&&cat==Cat2
But here is a small problem too, because I need to write it like above, without whitespaces and quotation marks. In the example, on the NUnit page, there is an example like below:
nunit3-console mytest.dll --where "cat == Urgent || Priority == High"

If I use whitespaces without quotation marks it starts filtering to bad tests.

Meissa test runner - cannot start execution

Hi,

I get the following error while trying to execute the test cases with Meissa:
System.AggregateException: One or more errors occurred. (Unable to cast object of type 'System.Collections.ObjectModel.ReadOnlyCollection1[System.String]' to type 'System.Collections.Generic.List1[System.String]'.) ---> System.InvalidCastException: Unable to cast object of type 'System.Collections.ObjectModel.ReadOnlyCollection1[System.String]' to type 'System.Collections.Generic.List1[System.String]'.

Do you have any clue what can be the cause of this?
Thanks!

Support for Nodejs Jasmine Tests

We are working on UI automation using Protractor ( https://www.protractortest.org ). We use Jasmine ( https://jasmine.github.io/setup/nodejs.html ) as our test runner.

We could make use of Meissa for test execution if you could add support. I met you in Selenium Conference 2018 and discussed the same with you.

Below is example of simple protractor test using Jasmine.
https://github.com/angular/protractor/tree/master/example

Please let me know if you need more info.

Thanks,
Raghav

Meissa Runner Aborts Runs if Tests Are Retried

2018-03-12T10:54:28.4677043Z Test agent with tag Agent1 finished tests execution on machine Machine1.
2018-03-12T10:54:28.4677043Z Test agent with tag Agent1 starts waiting for new jobs on machine Machine1.
2018-03-12T10:55:18.7664325Z UPDATE Test Run on machine Machine1 3/12/2018 10:55:18 AM
2018-03-12T10:55:25.9193674Z Abort Test Agent Run because test runner lost connection with test agent on machine: Machine1 3/12/2018 10:53:23 AM 3/12/2018 10:53:25 AM current status InProgress
2018-03-12T10:56:18.8316025Z ### All Test Agent Runs Finished! ###

Cannot find file or reference "System.Configuration.ConfigurationManager"

Hello,

I dowloaded latest release version of Meissa and trying to run tests with it. I started srevre and test agent, but when i try to run meissa.exe runner i get exception

System.IO.FileNotFoundException: Could not load file or assembly 'System.Configuration.ConfigurationManager, Version=0.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.

What can i do with this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.