Git Product home page Git Product logo

agent-net-gauge's Introduction

Join Slack chat! stackoverflow GitHub contributors Docker Pulls License Build with Love lambdaTest a27c44

Repositories structure

ReportPortal high level architecture

reportportal high level architecture

ReportPortal repositories structure

Report Portal organized into multiple repositories.

ReportPortal server side consists of the following services:

  • service-authorization Authorization Service. In charge of access tokens distribution
  • service-api API Service. Application Backend
  • service-ui UI Service. Application Frontend
  • service-index Index Service. Info and health checks per service.
  • service-analyzer Analyzer Service. Finds most relevant test fail problem.
  • gateway Traefik Gateway Service. Main entry point to application. Port used by gateway should be opened and accessible from outside network.
  • rabbitmq Load balancer for client requests. Bus for messages between servers.
  • minio Attachments storage.

Available plugins developed by ReportPortal team:

Client side adapters related repositories:

  • client-* - API integrations. Http clients, which process HTTP request sending.
  • agent-* - Frameworks integration. Custom reporters/listeners, which monitor test events and trigger event sending via client-*
  • logger-* - Logging integration. Logger appenders, which help to collect logs, bind it with test-case item via agent-* and send to server via client-*

Other repositories stored according to next rules

  • service-* - micro-services which are a part of Application
  • commons-* - common libraries, models, etc., used by micro-services

Installation steps

Simple setup with Docker

It's the best way for demo purposes and small teams. The database is already in the docker-compose.

  1. Install Docker (Engine, Compose)

  2. Download Example of docker-compose descriptor to any folder

  3. Deploy ReportPortal using docker compose plugin within the same folder

docker compose -p reportportal up

To start ReportPortal in daemon mode, add '-d' argument:

docker compose -p reportportal up -d
  1. Open in your browser IP address of deployed environment at port 8080
http://IP_ADDRESS:8080
  1. Use next login\pass for access:
  • default\1q2w3e
  • superadmin\erebus.

Please change admin password for security.

Mentioned compose file deploy all available Bug Tracking System integrations, which not always needed, but use resources.

Production-ready set and Custom deployment with Docker

For production usage, we recommend:

  • choose only required Bug Tracking System integration service. Exclude the rest

To customize deployment and make it production-ready please follow customization steps and details

Integration. How to get log data in

You should add Client Side code inside your test automation. It consists of:

  • client-* - API integrations. Http clients, which process HTTP request sending. E.g. for Java (client-java-*)
  • agent-* - Frameworks integration. Custom reporters/listeners, which monitor test events and trigger event sending via client-*
  • logger-* - Logging integration. Logger appenders, which helps to collect logs, bind it with test-case via agent-* and send to server via client-*

Integration steps and documentation

Contribution

There are many different ways to contribute to Report Portal's development, just find the one that best fits with your skills. Examples of contributions we would love to receive include:

  • Code patches
  • Documentation improvements
  • Translations
  • Bug reports
  • Patch reviews
  • UI enhancements

Big features are also welcome but if you want to see your contributions included in Report Portal codebase we strongly recommend you start by initiating a chat through our Team in Slack.

Contribution details

Documentation

Community / Support

License

Report Portal is Apache 2.0.

agent-net-gauge's People

Contributors

maksim-shila avatar nvborisenko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

agent-net-gauge's Issues

BadHttpRequestException: Request body too large

Error occurred when running test in the pipeline:

fail: Grpc.AspNetCore.Server.ServerCallHandler[6]
Error when executing service method 'NotifySuiteResult'.
Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelBadHttpRequestException.Throw(RequestRejectionReason reason)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.MessageBody.AddAndCheckObservedBytes(Int64 observedBytes)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestPipeReader.AdvanceTo(SequencePosition consumed, SequencePosition examined)
at Grpc.AspNetCore.Server.Internal.PipeExtensions.ReadSingleMessageAsync[T](PipeReader input, HttpContextServerCallContext serverCallContext, Func`2 deserializer)
at Grpc.AspNetCore.Server.Internal.CallHandlers.UnaryServerCallHandler`3.HandleCallAsyncCore(HttpContext httpContext, HttpContextServerCallContext serverCallContext)
at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.<HandleCallAsync>g__AwaitHandleCall|8_0(HttpContextServerCallContext serverCallContext, Method`2 method, Task handleCall)
Unable to connect to plugin ReportPortal 1.11.0. rpc error: code = Unknown desc = Exception was thrown by handle
Specifications:	24 executed	24 passed	0 failed	0 skipped
Scenarios:	177 executed	177 passed	0 failed	0 skipped

Screenshots are not attached to steps

screenshots to step has no effect on report portal, only screenshot on failure is attached

From the sources, it looks like ScreenshotFiles field isn't handled. If i got everything right, it should be something similar to this block of code(which handles FailureScreenshotFile field):

try
{
var failureScreenshotFile = request.StepResult.ProtoItem.Step.StepExecutionResult.ExecutionResult.FailureScreenshotFile;
if (!string.IsNullOrEmpty(failureScreenshotFile))
{
stepReporter.Log(new CreateLogItemRequest
{
Time = DateTime.UtcNow,
Level = LogLevel.Error,
Text = "Screenshot",
Attach = new LogItemAttach
{
Name = "screenshot",
MimeType = Shared.MimeTypes.MimeTypeMap.GetMimeType(Path.GetExtension(failureScreenshotFile)),
Data = File.ReadAllBytes(Path.Combine(_gaugeScreenshotsDir, failureScreenshotFile))
}
});
}
}
catch (Exception exp)
{
TraceLogger.Error($"Couldn't parse failure step screenshot. {exp}");
}

@nvborisenko FYI
#33

NullReferenceException when running data-driven specs in parallel

Problem:

Running data-driven specs in parallel is causing NullReferenceException to be thrown on gauge versions:

Gauge version: 1.1.1 Commit Hash: 6e0d83f Plugins html-report (4.0.10) java (0.7.9) reportportal (1.7.0) screenshot (0.0.1) spectacle (0.1.4) xml-report (0.2.3)

Tests are executed on command line using maven mvn clean test -P<profile> -Denv=<env>

The following exception is thrown:

Finishing to send results to Report Portal... Unexpected errors: System.NullReferenceException: Object reference not set to an instance of an object. at ReportPortal.GaugePlugin.Results.Sender.Sync() at ReportPortal.GaugePlugin.ReportMessagesHandler.Kill(KillProcessRequest request, ServerCallContext context)

Example Spec:

Spec A

tags: abc

URL
URLA
URLB
URLC
  • Context step set values
    a b c
    A B C

Scenario A

  • Step 1
  • Step 2
  • Step 3 be "Created"
  • Step 4 <URL>
  • Step 5 assert
    ABC
    123
    345
  • Step 6 Assert
    key value
    a test1
    b test2

Agent doesn't mark skipped tests in Report Portal

We found that: https://github.com/reportportal/agent-net-gauge/blob/develop/src/ReportPortal.GaugePlugin/Results/Sender.Steps.cs#L120. Look like agent has functionality for skip steps, but it doesn't work.

As I see here: https://github.com/getgauge/gauge-proto/blob/master/doc/gauge-proto-doc.md#gauge.messages.ProtoSpecResult we could have possibility get info about skip state of scenario.

This functionality could be added into agent by supporters?

Dead?

Can we expect any issues to be fixed here or is development stopped?

Add enable flag

Will it be possible to add the RP_ENABLE env property to allow control of when logging occurs to reportportal for test runs?

Thanks

Gauge concept steps are not properly shown in reports portal UI- It should be ideally shown as a collapsible item

Gauge has a functionality of concepts: so in a concept file you can have multiple gauge steps, and when you call a concept header (Also a part of concept file) as a step in spec file:
More details are mentioned here:

In Gauge HTML report : Concepts are shown as collapsible item, where in by default everything is collapsed and only concept header is shown : you can expand the header to show the internal steps concept is using

But when same HTML reports are uploaded to report portal, all internal steps of concepts are shown by defualt, which makes reports a bit lengthy : It would be great if you could provide support of concepts as well. We and probably many others uses concepts functionality of gauge quite extensively.

Also thank you for creating such a cool Project, and supporting so many different types of plugins.

Attaching a sample gauge HTML report of a spec using concepts, as a reference:
html-report.zip

Please let me know if you need any more information regarding this topic.

The SSL connection could not be established

From slack:

Any idea how to bypass insecure TLS certificates(self signed) errors, I am using plugin https://github.com/reportportal/agent-net-gauge
any configuration to ignore tls-verify??

Unexpected errors: System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
-------------------------------------------------------------------------
$ gauge --version
Gauge version: 1.1.1
Commit Hash: 6e0d83f
Plugins
-------
go (0.1.4)
html-report (4.0.12)
reportportal (1.9.0)
screenshot (0.0.1)
xml-report (0.2.3)
-------------------------------------------------------------------
reportportal version: v5

Unexpected errors - The operation was canceled

To give some background, gauge tests were run on jenkins which is hosted in the cloud. Reportportal is hosted on a VM on a different network. It may be that I need to speak to DevOps team to check that this VM can be accessed from jenkins in the cloud (it should be as I did ask them to allow it) but I would have expected a time out message or something similar instead of an unexpected error. Though it may be a different issue. See stack trace below:

Finishing to send results to Report Portal...
Unexpected errors: System.Threading.Tasks.TaskCanceledException: The operation was canceled.
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean ********Http2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at ReportPortal.Client.Resources.ServiceBaseResource.SendAsJsonAsync[TResponse,TRequest](HttpMethod httpMethod, String uri, TRequest request) at ReportPortal.Client.Resources.ServiceBaseResource.PostAsJsonAsync[TResponse,TRequest](String uri, TRequest request) at ReportPortal.Client.Resources.ServiceLaunchResource.StartAsync(StartLaunchRequest request) at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action1 beforeNextAttempt) at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action`1 beforeNextAttempt)
at ReportPortal.Shared.Reporter.LaunchReporter.<>c__DisplayClass20_0.<b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at ReportPortal.Shared.Reporter.LaunchReporter.Sync()
at ReportPortal.GaugePlugin.Results.Sender.Sync()
at ReportPortal.GaugePlugin.ReportMessagesHandler.Kill(KillProcessRequest request, ServerCallContext context)

Tags on specification level are not applied to scenario level

Preconditions:

  1. Gauge runner is used.
  2. There is a specification with two test scenarios: specification has its own tags and each test scenario also has its tags.

gauge_specification_with_tests

Actual Result:
In Report Portal in each test scenario there is only its own tag present (e.g. android-case-id:5555) without tags from specification (owner: SomeTeam, tag:android).

specification_in_rp

tests_tags_actual

Expected Result:
In Report Portal in each test scenario there are all tags from specification + tags from test scenario.
So tags for one test should be next: android-case-id:5555, owner: SomeTeam, tag: android.

test_ tags_expected

item_details_expected_result

Jenkins error

Local windows reporting works fine, however; I receive and error when reporting on linux jenkins and nothing is reported.

My config:
rp_uri = http://mdzhlbmac05.standard.xyz.net:8080/api/v1
rp_project = demo
rp_uuid = e13cd8e6-9823-49c7-aff8-f8333d536bf7

Error:

๏ฟฝ[0mFailFast:๏ฟฝ[0m
๏ฟฝ[0mCouldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.๏ฟฝ[0m
๏ฟฝ[0m
๏ฟฝ[0m at System.Environment.FailFast(System.String)๏ฟฝ[0m
๏ฟฝ[0m at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()๏ฟฝ[0m
๏ฟฝ[0m at System.Globalization.GlobalizationMode..cctor()๏ฟฝ[0m
๏ฟฝ[0m at System.Globalization.CultureData.CreateCultureWithInvariantData()๏ฟฝ[0m
๏ฟฝ[0m at System.Globalization.CultureData.get_Invariant()๏ฟฝ[0m
๏ฟฝ[0m at System.Globalization.CultureInfo..cctor()๏ฟฝ[0m
๏ฟฝ[0m at System.StringComparer..cctor()๏ฟฝ[0m
๏ฟฝ[0m at System.AppDomain.InitializeCompatibilityFlags()๏ฟฝ[0m
๏ฟฝ[0m at System.AppDomain.Setup(System.Object)๏ฟฝ[0m
๏ฟฝ[0mError starting plugin ReportPortal 1.1.0. Failed to connect to plugin. Timed out connecting to 127.0.0.1:34297

Finish launch is not allowed. Launch '<launch_id>' already finished with status 'PASSED'

@nvborisenko Getting the following error after execution (auto-generated).

  1. Not sure how the test was marked as Passed/Finished before the final finish call (test was not timed-out).
  2. Is it possible we can add an error handling here?

There were no skipped cases, but a few set of failed cases were present.

Finishing to send results to Report Portal...
Unexpected error: ReportPortal.Client.ReportPortalException: Response status code does not indicate success: NotAcceptable (406) PUT <url>/api/v1/<project>/launch/<launch_id>/finish
---> System.Net.Http.HttpRequestException: Response message: {"errorCode":40023,"message":"Finish launch is not allowed. Launch '<launch_id>' already finished with status 'PASSED'"}
--- End of inner exception stack trace ---
at ReportPortal.Client.Resources.ServiceBaseResource.CheckSuccessStatusCode(HttpResponseMessage response, Stream stream)
at ReportPortal.Client.Resources.ServiceBaseResource.SendHttpRequestAsync[TResponse](HttpMethod httpMethod, String uri, HttpContent httpContent)
at ReportPortal.Shared.Internal.Delegating.BaseRequestExecuter.ExecuteAsync[T](Func`1 func, Action`1 beforeNextAttemptCallback, IStatisticsCounter statisticsCounter)
at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func`1 func, Action`1 beforeNextAttempt, IStatisticsCounter statisticsCounter)
at ReportPortal.Shared.Reporter.LaunchReporter.<>c__DisplayClass30_0.<<Finish>b__0>d.MoveNext()
--- End of stack trace from previous location ---
at ReportPortal.Shared.Reporter.LaunchReporter.Sync()
at ReportPortal.GaugePlugin.Results.Sender.Sync()
at ReportPortal.GaugePlugin.Services.ReportMessagesHandler.Kill(KillProcessRequest request, ServerCallContext context)

Unable to install the reportportal-1.1.0-preview.1 version on mac

Hi Team,

We are unable to install the latest preview version on mac , where installation fails with below error

Error ----------------------------------

Panicing : runtime error: invalid memory address or nil pointer dereference
goroutine 1 [running]:
runtime/debug.Stack(0xc0000bb940, 0x1665cc0, 0x1ca09a0)
/usr/local/Cellar/go/1.11/libexec/src/runtime/debug/stack.go:24 +0xa7
main.recoverPanic()
src/github.com/getgauge/gauge/gauge.go:36 +0x57
panic(0x1665cc0, 0x1ca09a0)
/usr/local/Cellar/go/1.11/libexec/src/runtime/panic.go:513 +0x1b9
github.com/getgauge/gauge/plugin/install.InstallPluginFromZipFile(0x7ffeefbffb6a, 0x47, 0x7ffeefbffb56, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/stechops/Documents/go-agent-17.4.0/pipelines/Gauge-Build/src/github.com/getgauge/gauge/plugin/install/install.go:159 +0x830
github.com/getgauge/gauge/cmd.glob..func10(0x1caa5c0, 0xc000162fc0, 0x1, 0x3)
/Users/stechops/Documents/go-agent-17.4.0/pipelines/Gauge-Build/src/github.com/getgauge/gauge/cmd/install.go:43 +0xa6
github.com/getgauge/gauge/vendor/github.com/spf13/cobra.(*Command).execute(0x1caa5c0, 0xc000162f30, 0x3, 0x3, 0x1caa5c0, 0xc000162f30)
/Users/stechops/Documents/go-agent-17.4.0/pipelines/Gauge-Build/src/github.com/getgauge/gauge/vendor/github.com/spf13/cobra/command.go:651 +0x24a
github.com/getgauge/gauge/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x1ca9900, 0xf, 0xc000170180, 0xc0001204b0)
/Users/stechops/Documents/go-agent-17.4.0/pipelines/Gauge-Build/src/github.com/getgauge/gauge/vendor/github.com/spf13/cobra/command.go:726 +0x2cc
github.com/getgauge/gauge/vendor/github.com/spf13/cobra.(*Command).Execute(0x1ca9900, 0x0, 0xc00017bf78)
/Users/stechops/Documents/go-agent-17.4.0/pipelines/Gauge-Build/src/github.com/getgauge/gauge/vendor/github.com/spf13/cobra/command.go:685 +0x2b
github.com/getgauge/gauge/cmd.Parse(0x0, 0x17638c8)
/Users/stechops/Documents/go-agent-17.4.0/pipelines/Gauge-Build/src/github.com/getgauge/gauge/cmd/cmd.go:158 +0x3d
main.main()
src/github.com/getgauge/gauge/gauge.go:29 +0x3e

Get Support ----------------------------
Docs: https://docs.gauge.org
Bugs: https://github.com/getgauge/gauge/issues
Chat: https://gitter.im/getgauge/chat

Your Environment Information -----------
darwin, 1.0.3, ff6c0c3
html-report (4.0.8), java (0.7.2), screenshot (0.0.1), xml-report (0.2.1)

Unable to log failing tests

The plugin is not able send to reportportal when a scenario fails.

Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
at ReportPortal.Gauge.Program.Main(String[] args) in C:\projects\agent-net-gauge\ReportPortal.Gauge\Program.cs:line 131
Successfully generated html-report to => C:\Daten\git\delimac-test\e2e\reports\html-report\index.html
Specifications: 1 executed 0 passed 1 failed 0 skipped
Scenarios: 2 executed 1 passed 1 failed 0 skipped

Test Scenario is marked as STEP in reportal portal data base

Preconditions:

Gauge runner is used.
There is a specification with two test scenarios.

gauge_specification_with_tests

Actual result:
"Scenario Heading 1" is recorded into database reportportal.test_item table with Type 'STEP'.

test

db

Expected result:
"Scenario Heading 1" is recorded into database reportportal.test_item table with Type 'TEST'.
As it is not a step, but name of test scenario.

Error : signal: segmentation fault with reportportal plugin when running tests using gauge-java-docker

Description:

When running tests using gauge-java-docker (https://github.com/sitture/docker-gauge-java), reportportal fails to start. The following error is outputted:

Error occurred while waiting for plugin process to finish.
Error : signal: segmentation fault
Error starting plugin ReportPortal 1.9.0. timed out connecting to reportportal

As suggested, libc6-compat is installed (https://github.com/sitture/docker-gauge-java/blob/master/base/Dockerfile#L23) but this has not resolved the issue above.

Steps to recreate:

  1. Use sample gauge_maven project: https://github.com/imindersingh/gauge-maven
  2. Update reportportal config in env properties
  3. Run tests using:
docker run --rm -it -v $(pwd):/usr/src/app -v ~/.m2:/root/.m2 -e RP_ENABLED=true -e ReportPortal_TraceLevel=Verbose sitture/docker-gauge-java:latest mvn clean test

Gauge versions from the docker image:

Gauge version: 1.1.4
Commit Hash: c982558

Plugins
-------
html-report (4.0.12)
java (0.7.13)
reportportal (1.9.0)
screenshot (0.0.1)
xml-report (0.2.3)

ReportPortal Shows all Specs as Passed, even though the spec has failed.

Hi Team,

I am facing issues with the gauge integration with Report Portal, where the results uploaded to Gauge shows incorrect results. The ReportPortal shows all the specs as passed, even though the TCs failed. Please find below the details of Gauge version that i have in my environment. Any help will be highly appreciated

Gauge version: 1.0.5

Plugins
html-report (4.0.8) java (0.7.2) reportportal (1.0.1) screenshot (0.0.1) xml-report (0.2.1)

Unable to finish test rerun

When running gauge tests with rp_launch_rerun=true the following error appears in the logs.

Main execution works fine. The issue is reproducible for reruns only.

Unexpected errors: System.AggregateException: Cannot finish launch due finishing of child items failed. (One or more errors occurred. (Cannot finish test item due finishing of child items failed. (One or more errors occurred. (Cannot finish test item due starting item failed.)))) (One or more errors occurred. (Cannot finish test item due finishing of child items failed. (One or more errors occurred. (Cannot finish test item due starting item failed.))))

[09:39:03 ](...) ---> System.AggregateException: One or more errors occurred. (Cannot finish test item due finishing of child items failed. (One or more errors occurred. (Cannot finish test item due starting item failed.)))

[09:39:03 ](..  ---> System.AggregateException: Cannot finish test item due finishing of child items failed. (One or more errors occurred. (Cannot finish test item due starting item failed.))

[09:39:03 ](...) ---> System.AggregateException: One or more errors occurred. (Cannot finish test item due starting item failed.)

[09:39:03 ](...) ---> System.Exception: Cannot finish test item due starting item failed.

[09:39:03 ](...)  ---> System.AggregateException: One or more errors occurred. (Response status code does not indicate success: InternalServerError (500) POST <RP URL>/item/1ed85a89-1798-4422-8e98-ac46b7b04066

[09:39:03 ](...)  ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST <RP URL>/1ed85a89-1798-4422-8e98-ac46b7b04066

09:39:03   ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [org.hibernate.exception.LockAcquisitionException: could not extract ResultSet]"}

[09:39:03 ](...)  --- End of inner exception stack trace ---

Dynamic attributes

Hello,

Do you see a possibility to add support for dynamic reportportal attributes? I could make a pull request. I am wondering where u think the dynamic attributes could be stored to be accessible from the reporter?

Jose

Don't rely on Gauge.CSharp.Core package

This package is for internal usage and is not guaranteed to be compatible with rpc messages.

It would better to get proto messages directly and generate client at build-time.

Use gRPC

It also allows to report results in real-time.

Launch hangs when rerunning failed tests

Discovered that launches are hanging and showing in separate launches when retrying failed tests. It may be an issue with the fact that the tests are running in parallel. These are being run via gauge docker container on jenkins and when investigating this on my local machine I've discovered the following.

Command used to run tests:
mvn clean install -Ptest -DinParallel=true || mvn gauge:execute -Dflags=--failed

Seeing the following stack trace a few times in the log after enabled verbose logging:

ReportMessagesHandler Error: 0 : 13:39:27.8348090 : 1-ReportPortal.GaugePlugin : System.Collections.Generic.KeyNotFoundException: The given key '{"FileName":"/Users/eee04/Workspace/api-tests-java/specs/Tests/Test.spec","Name":"Test1","ScenarioName":"Scenario1","StepName":"Store tokens"}' was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at ReportPortal.GaugePlugin.Results.Sender.FinishStep(StepExecutionEndingRequest request)
at ReportPortal.GaugePlugin.ReportMessagesHandler.NotifyStepExecutionEnding(StepExecutionEndingRequest request, ServerCallContext context)

And this one a few times forwards the end of the log:

ReportMessagesHandler Error: 0 : 13:39:28.2988000 : 1-ReportPortal.GaugePlugin : System.InsufficientExecutionStackException: Some of child test item(s) are not scheduled to finish yet.
at ReportPortal.Shared.Reporter.TestReporter.Finish(FinishTestItemRequest request)
at ReportPortal.GaugePlugin.Results.Sender.FinishScenario(ScenarioExecutionEndingRequest request)
at ReportPortal.GaugePlugin.ReportMessagesHandler.NotifyScenarioExecutionEnding(ScenarioExecutionEndingRequest request, ServerCallContext context)

Whilst running locally a single launch is being shown but then I have seen the following in console output:

panic: runtime error: index out of range [1] with length 1

goroutine 521 [running]:
github.com/getgauge/gauge/execution.(*stepExecutor).executeStep(0xc002bd0bf0, 0xc000c67e00, 0xc0023362d0, 0x0)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/stepExecutor.go:43 +0x9ff
github.com/getgauge/gauge/execution.(*scenarioExecutor).executeStep(0xc000bd0930, 0xc000c67e00, 0xc0003d4b60, 0xc00049ed80, 0xc000d20000)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/scenarioExecutor.go:176 +0x174
github.com/getgauge/gauge/execution.(*scenarioExecutor).executeSteps(0xc000bd0930, 0xc00034e0c0, 0x4, 0x4, 0xc00049ee20, 0x4, 0x4, 0xc00049ed80)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/scenarioExecutor.go:154 +0xa6
github.com/getgauge/gauge/execution.(*scenarioExecutor).execute(0xc000bd0930, 0x18417a0, 0xc00149f4a0, 0x1857f80, 0xc00049ed80)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/scenarioExecutor.go:86 +0x94e
github.com/getgauge/gauge/execution.(*specExecutor).executeScenario(0xc0005254a0, 0xc00149f4a0, 0xc0025e00a0, 0x0, 0x0)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/specExecutor.go:324 +0x26d
github.com/getgauge/gauge/execution.(*specExecutor).executeScenarios(0xc0005254a0, 0xc000209100, 0x9, 0x10, 0x10, 0x8, 0x8, 0x0, 0x0)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/specExecutor.go:281 +0x83
github.com/getgauge/gauge/execution.(*specExecutor).executeTableRelatedScenarios(0xc0005254a0, 0xc000209100, 0x9, 0x10, 0x0, 0x0)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/specExecutor.go:148 +0x82
github.com/getgauge/gauge/execution.(*specExecutor).executeSpec(0xc0005254a0, 0xc0022f02c0, 0xc00315aee0)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/specExecutor.go:168 +0x190
github.com/getgauge/gauge/execution.(*specExecutor).execute(0xc0005254a0, 0x1010101, 0xc000d22d70)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/specExecutor.go:129 +0x929
github.com/getgauge/gauge/execution.(*simpleExecution).executeSpecs(0xc0011da700, 0xc0003fb680, 0xc00037e0e0, 0x0, 0x0)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/simpleExecution.go:130 +0x193
github.com/getgauge/gauge/execution.(*simpleExecution).execute(0xc0011da700)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/simpleExecution.go:92 +0x1a4
github.com/getgauge/gauge/execution.(*parallelExecution).startSpecsExecutionWithRunner(0xc00073c280, 0xc0003fb680, 0xc0001b4c00, 0x1858100, 0xc000d22d70, 0x2)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/parallelExecution.go:273 +0xa1
github.com/getgauge/gauge/execution.(*parallelExecution).startStream(0xc00073c280, 0xc0003fb680, 0xc0001b4c00, 0x2)
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/parallelExecution.go:251 +0x15c
created by github.com/getgauge/gauge/execution.(*parallelExecution).executeLazily
/private/tmp/gauge-20200319-67576-151qpxe/gauge-1.0.8/src/github.com/getgauge/gauge/execution/parallelExecution.go:162 +0x81

Getting this error: Object reference not set to an instance of an object

Getting this error randomly during many runs. Hope you will please have a check @nvborisenko

Finishing to send results to Report Portal...
Unexpected error: System.NullReferenceException: Object reference not set to an instance of an object.
at ReportPortal.GaugePlugin.Results.Sender.Sync()
at ReportPortal.GaugePlugin.Services.ReportMessagesHandler.Kill(KillProcessRequest request, ServerCallContext context)
fail: Grpc.AspNetCore.Server.ServerCallHandler[6]
Error when executing service method 'Kill'.
System.NullReferenceException: Object reference not set to an instance of an object.
at ReportPortal.GaugePlugin.Services.ReportMessagesHandler.Kill(KillProcessRequest request, ServerCallContext context)
at Grpc.Shared.Server.UnaryServerMethodInvoker`3.Invoke(HttpContext , ServerCallContext , TRequest )
--- End of stack trace from previous location ---
at Grpc.AspNetCore.Server.Internal.CallHandlers.UnaryServerCallHandler`3.HandleCallAsyncCore(HttpContext , HttpContextServerCallContext )
at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.<HandleCallAsync>g__AwaitHandleCall|8_0(HttpContextServerCallContext , Method`2 , Task )

executing scenario resulting with error

cant manage to execute a scenario and receive result at the reportportal web UI
tried several configuration(i used the 'uuid' from the 'profile' of the user i used in the 'reportportal' portal):

rp_uri = https://rp.epam.com/api/v1
rp_project = superadmin_personal
rp_uuid = 2f954b1f-7af3-437a-9bc9-aac3bf501a39

result:
Unexpected response status code. Request URI: https://rp.epam.com/api/v1/superadmin_personal/launch
Response Body: {"error":"invalid_token","error_description":"2f954b1f-7af3-437a-9bc9-aac3bf501a39"} ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).

rp_uri = http://localhost:8080
rp_uuid = 2f954b1f-7af3-437a-9bc9-aac3bf501a39
rp_launch = superadmin_TEST_EXAMPLE
rp_project = superadmin_personal

resulting with:
... System.Xml.XmlException: Encountered unexpected character '<'.

Feature request: Add launch metadata to Gauge ExecutionContext

Why?
We want to add the link to the failed test in the Report Portal to the corresponding test in TestRail run.

How?
We use a Gauge hook and grab the metadata from Gauge ExecutionContext and then pass it to TestRail.

So it would be great to have the ability to get the link to the executed test (or get the metadata that will allow us to build the link) from Gauge ExecutionContext in methods marked by com.thoughtworks.gauge.AfterScenario annotation.

error when executing a run

hi

when running a spec im receiving the following error:

Error starting plugin ReportPortal 1.0.0. fork/exec ./ReportPortal.Gauge.dll: permission denied

and no results appearing in the dashboard

Logging of Steps

Hi,

First, thanks allot for this plugin, its very useful.

This is a feature request. According to your documentation "A typical Launch structure comprises the following elements: Suite > Test > Step > Log."

Currently this plugin does not log steps. Is this by design? If so, then I would like to request that the gauge steps are included in the reporting.

Key-value launch attributes aren't trimmed before sending to the Report Portal

Gauge Report Portal Plugin Version: 1.8.0
Report Portal: API Service: 5.2.2; Index Service: 5.0.7; Authorization Service: 5.2.1; Service UI: 5.2.2;

STR:

  1. Set rp_launch_attributes = tag1; tag2; platform:x64; tag3
  2. Run Gauge tests to initiate launch in The Report Portal
  3. Check the item_attribute table in The Report Portal Database (select * from item_attribute where key = ' platform')

Expected result:
The launch attributes key is trimmed.

Actual result:
The launch attributes key isn't trimmed and contains the leading space. It breaks The Report Portal launches filtering using the launch attributes key. The expected launch doesn't appear in the filtering results.

Test retried via Gauge --max-retries-count flag are reported as different tests in Report Portal

Tests executed with retries using Gauge --max-retries-count flag (gauge run --max-retries-count=<number of retries>) are shown in Report Portal as separate tests, not several retries of the same test.

Gauge --max-retries-count flag description here.

Expected all test retried via --max-retries-count mechanism to be reported as retries of the same test. The same way it works for Gauge reruns executed with --failed flag.

To reproduce: create a spec with failing test and run it via gauge run --max-retries-count=3

Unexpected Error following retry failed tests - launch hanging

Not sure if this is still related to the previous issue (#28), but I thought it would be fixed. However noticed a couple of launches hanging using the latest plugin version and in the jenkins logs got the following output. This happened following the retry of failed of some failed tests (mvn clean install -Ptest -DinParallel=true || mvn gauge:execute -Dflags=--failed) - I did test this scenario but probably not enough times.

[int] Finishing to send results to Report Portal...
[int] Unexpected errors: System.AggregateException: Cannot finish launch due finishing of child items failed. (One or more errors occurred. (Cannot finish test item due starting item failed.)) (One or more errors occurred. (Cannot finish test item due starting item failed.)) (One or more errors occurred. (Cannot finish test item due starting item failed.)) (One or more errors occurred. (Cannot finish test item due starting item failed.)) (One or more errors occurred. (Cannot finish test item due starting item failed.))
[int] ---> System.AggregateException: One or more errors occurred. (Cannot finish test item due starting item failed.)
[int] ---> System.Exception: Cannot finish test item due starting item failed.
[int] ---> System.AggregateException: One or more errors occurred. (Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item)
[int] ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item
[int] ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [query did not return a unique result: 2]"}
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Client.Resources.ServiceBaseResource.CheckSuccessStatusCode(HttpResponseMessage response, Stream stream)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.SendAsJsonAsync[TResponse,TRequest](HttpMethod httpMethod, String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.PostAsJsonAsync[TResponse,TRequest](String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceTestItemResource.StartAsync(StartTestItemRequest request)
[int] at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action1 beforeNextAttempt)
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass22_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass27_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Shared.Reporter.LaunchReporter.<>c__DisplayClass25_0.<b__0>d.MoveNext()
[int] --- End of stack trace from previous location where exception was thrown ---
[int] at ReportPortal.Shared.Reporter.LaunchReporter.Sync()
[int] at ReportPortal.GaugePlugin.Results.Sender.Sync()
[int] at Re
[int] portPortal.GaugePlugin.ReportMessagesHandler.Kill(KillProcessRequest request, ServerCallContext context)
[int] ---> (Inner Exception #1) System.AggregateException: One or more errors occurred. (Cannot finish test item due starting item failed.)
[int] ---> System.Exception: Cannot finish test item due starting item failed.
[int] ---> System.AggregateException: One or more errors occurred. (Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item)
[int] ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item
[int] ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [query did not return a unique result: 2]"}
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Client.Resources.ServiceBaseResource.CheckSuccessStatusCode(HttpResponseMessage response, Stream stream)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.SendAsJsonAsync[TResponse,TRequest](HttpMethod httpMethod, String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.PostAsJsonAsync[TResponse,TRequest](String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceTestItemResource.StartAsync(StartTestItemRequest request)
[int] at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action1 beforeNextAttempt)
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass22_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass27_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---<---
[int] ---> (Inner Exception #2) System.AggregateException: One or more errors occurred. (Cannot finish test item due starting item failed.)
[int] ---> System.Exception: Cannot finish test item due starting item failed.
[int] ---> System.AggregateException: One or more errors occurred. (Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item)
[int] ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item
[int] ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [query did not return a unique result: 3]"}
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Client.Resources.ServiceBaseResource.CheckSuccessStatusCode(HttpResponseMessage response, Stream stream)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.SendAsJsonAsync[TResponse,TRequest](HttpMethod httpMethod, String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.PostAsJsonAsync[TResponse,TRequest](String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceTestItemResource.StartAsync(StartTestItemRequest request)
[int] at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action1 beforeNextAttempt)
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass22_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass27_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---<---
[int] ---> (Inner Exception #3) System.AggregateException: One or more errors occurred. (Cannot finish test item due starting item failed.)
[int] ---> System.Exception: Cannot finish test item due starting item failed.
[int] ---> System.AggregateException: One or more errors occurred. (Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item)
[int] ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item
[int] ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [query did not return a unique result: 3]"}
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Client.Resources.ServiceBaseResource.CheckSuccessStatusCode(HttpResponseMessage response, Stream stream)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.SendAsJsonAsync[TResponse,TRequest](HttpMethod httpMethod, String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.PostAsJsonAsync[TResponse,TRequest](String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceTestItemResource.StartAsync(StartTestItemRequest request)
[int] at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action1 beforeNextAttempt)
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass22_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass27_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---<---
[int] ---> (Inner Exception #4) System.AggregateException: One or more errors occurred. (Cannot finish test item due starting item failed.)
[int] ---> System.Exception: Cannot finish test item due starting item failed.
[int] ---> System.AggregateException: One or more errors occurred. (Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item)
Post stage
[int] ---> ReportPortal.Client.ReportPortalException: Response status code does not indicate success: InternalServerError (500) POST http://1.1.1.1:8080/api/v1/test/item
[int] ---> System.Net.Http.HttpRequestException: Response message: {"errorCode":5000,"message":"Unclassified error [query did not return a unique result: 3]"}
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Client.Resources.ServiceBaseResource.CheckSuccessStatusCode(HttpResponseMessage response, Stream stream)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.SendAsJsonAsync[TResponse,TRequest](HttpMethod httpMethod, String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceBaseResource.PostAsJsonAsync[TResponse,TRequest](String uri, TRequest request)
[int] at ReportPortal.Client.Resources.ServiceTestItemResource.StartAsync(StartTestItemRequest request)
[int] at ReportPortal.Shared.Internal.Delegating.ExponentialRetryRequestExecuter.ExecuteAsync[T](Func1 func, Action1 beforeNextAttempt)
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass22_0.<b__0>d.MoveNext()
[int] --- End of inner exception stack trace ---
[int] --- End of inner exception stack trace ---
[int] at ReportPortal.Shared.Reporter.TestReporter.<>c__DisplayClass27_0.<b__0>d.MoveNext()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.