damianszczepanik / cucumber-sandwich Goto Github PK

I have a report where I am attaching text/plan to both the after hooks and also in step definitions for cucumberJs. A sample of the json generated in the report is as follows
[ { "description": " Test", "keyword": "Feature", "name": "Test 1", "line": 2, "id": "test-1", "tags": [ { "name": "@test1", "line": 1 } ], "uri": "test/feature/test1.feature", "elements": [ { "id": "test-1-feature-sceanrio", "keyword": "Scenario", "line": 26, "name": "test 1 feature scenario", "tags": [ { "name": "@test1", "line": 1 }, { "name": "@testSceanrio", "line": 23 }, { "name": "@tag2", "line": 23 } ], "type": "scenario", "steps": [ { "keyword": "Before", "hidden": true, "match": { "location": "test/support/Hooks.js:89" }, "result": { "status": "passed", "duration": 1000000 } }, { "keyword": "Before", "hidden": true, "match": { "location": "test/support/Hooks.js:105" }, "result": { "status": "passed", "duration": 3000000 } }, { "arguments": [], "keyword": "Given ", "line": 8, "name": "I perofrm a test", "match": { "location": "test/stepDefinitions/login/Login.js:44" }, "result": { "status": "passed", "duration": 1410000000 } }, { "arguments": [ { "rows": [ { "cells": [ "data1", "data2", "data3", "data4" ] }, { "cells": [ "data1", "data2", "data3", "" ] } ] } ], "keyword": "And ", "line": 9, "name": "I have a test creaed", "match": { "location": "test/stepDefinitions/common/CommonDataPrep.js:51" }, "result": { "status": "passed", "duration": 16178000000 }, "embeddings": [ { "data": "Test this", "mime_type": "text/plain" }, { "data": "TestName: Test A TestId: 123", "mime_type": "text/plain" } ] }, { "arguments": [ { "rows": [ { "cells": [ "data1", "data2", "data3", "data4", "data5", "data6", "data7", "data8", "data9" ] }, { "cells": [ "data1", "data2", "data3", "data4", "data5", "data6", "data7", "data8", "data9" ] } ] } ], "keyword": "And ", "line": 12, "name": "I finish test", "match": { "location": "test/stepDefinitions/common/CommonDataPrep.js:72" }, "result": { "status": "failed", "duration": 60010000000, "error_message": "Error: function timed out, ensure the promise resolves within 60000 milliseconds\n at Timeout._time.default.setTimeout [as _onTimeout] (/Users/raymondkelly/WebstormProjects/quotepad-service/node_modules/cucumber/lib/user_code_runner.js:81:20)\n at ontimeout (timers.js:498:11)\n at tryOnTimeout (timers.js:323:5)\n at Timer.listOnTimeout (timers.js:290:5)" } }, { "arguments": [ { "rows": [ { "cells": [ "data1", "data2", "data3", "data4", "data5" ] }, { "cells": [ "data1", "data2", "data3", "data4", "data5" ] }, { "cells": [ "data1", "data2", "data3", "data4", "data5" ] }, { "cells": [ "data1", "data2", "data3", "data4", "data5" ] } ] } ], "keyword": "When ", "line": 15, "name": "I did the test", "match": { "location": "test/stepDefinitions/quoting/ContributionStrategy.js:40" }, "result": { "status": "skipped" } }, { "arguments": [ { "rows": [ { "cells": [ "data1" ] }, { "cells": [ "data1" ] } ] } ], "keyword": "Then ", "line": 20, "name": "everything is complete", "match": { "location": "test/stepDefinitions/quoting/QuoteCensus.js:105" }, "result": { "status": "skipped" } }, { "keyword": "After", "hidden": true, "match": { "location": "test/support/Hooks.js:122" }, "result": { "status": "passed", "duration": 1000000 }, "embeddings": [ { "data": "Test this after", "mime_type": "text/plain" } ] } ] } ] } ]

I have reviewed the code and tests in https://github.com/damianszczepanik/cucumber-reporting and the json appears to be in the correct format however when I generate the report the attachment in the report shows as M�-���

I have tested this with cucumber-reporting-4.7.0 and cucumber-reporting-4.6.0 and have the same issue. Attached below is an image of the html

We've

noticed that when a scenario has a step that returns 'pending' that the cucumber reports show the scenario as 'failed'. Is there a way to change that behavior? We use 'pending' to indicate that a certain dependency is not configured during a particular test run, and this is an expected condition

Reference:damianszczepanik/cucumber-reporting#636

I feel there is a need for listing all the failed scenarios within a feature, when we click on failed tests within features.

It would be great to wrap this into the jenkins cucumber reports so you could see the progress of your CI builds.

net.masterthought.cucumber.ValidationException: File 'target/cucumber-reports/cucumber.json' is not proper Cucumber report! at net.masterthought.cucumber.ReportParser.parseForFeature(ReportParser.java:99) at net.masterthought.cucumber.ReportParser.parseJsonFiles(ReportParser.java:71) at net.masterthought.cucumber.ReportBuilder.generateReports(ReportBuilder.java:88)

I am currently using 4.7.0
<cucumber.version>4.3.1</cucumber.version>

Below is the plugin setup in runner class
plugin = {"json:target/cucumber-reports/cucumber.json", "pretty", "html:target/cucumber-reports/cucumber-pretty.html",

Hi. I'm looking for this exact thing but for JavaScript cucumber.js

Any plans to support that? :)

In my feature files i use specification by example or Scenario Outline. When i try to generate the report from the json i get warnings about missing results. the report gets generated but all steps are listed as missing

Hi,

I have been using the tool for a while and it is fantastic! Thanks.

I have added some asserts into my after hooks
If one of these asserts fails the report doesn't mark the scenario as failed .
However, these are marked as fails in the console when I run the tests through the jUnit runner.
They are also highlighted when in the html cucumber report.
This may be a bit of a grey area should the scenario be marked as failed in this situation?

Thanks.

I wanted to remove additional task of starting the monitor / listener for my destination report directory. Instead i made the function generateReport() as public and calling it in @after hook of cucumber so that html report gets generated. However this generates just plain reports.

Not sure if its a bug. But i dont want to use seperate window, i am trying to integrate it with my test pack.

Hi, when i run the jar, it just says there is no main manifest in jar (i used 3.8.0)

Hi,
This is a good stand alone cucumber reporting tool that mimics the Jenkins plug in one.
Would it be possible to have this updated to draw/use the same charts?
i.e In the "Feature Overview" page, the Jenkins Plugin displays 3D pie and ring charts and also has hover feature that shows the number of passing/failing/skipped steps and scenarios.
Many thanks,
dh14162

Hi Damian,

I recently upgraded to the latest version stand-alone cucumber report (2.8.0) and now the report does not display any test scenario data tables from my feature file. Is this a known issue/bug?

Using the following dependency in my pom.xml, it pulls down the cucumber-sandwich-2.4.1.jar.

<dependency>
      <groupId>net.masterthought</groupId>
      <artifactId>cucumber-sandwich</artifactId>
      <version>RELEASE</version>
    </dependency>

When I then try to use it

java -jar ~/.m2/repository/net/masterthought/cucumber-sandwich/2.4.1/cucumber-sandwich-2.4.1.jar -n -f {myJSONfolder} -o {myOutputFolder}

I get this error:
no main manifest attribute, in ~/.m2/repository/net/masterthought/cucumber-sandwich/2.4.1/cucumber-sandwich-2.4.1.jar

I saw that you had a similar issue and suggested the latest release, but I can see in your Manifest that you do NOT have the Main-Class defined.

> cat MANIFEST.MF
Manifest-Version: 1.0
Built-By: Damian
Build-Jdk: 1.7.0_79
Created-By: Apache Maven 3.3.3
Archiver-Version: Plexus Archiver

Hi Good Day Damian,
I have reported this observation on the Google Cukes forum where the Junit assert statement errors are not fully reported on cucumber reports.

Link: https://groups.google.com/forum/#!topic/cukes/SaiUmv7Nd8k

Current Environment: Cukes 1.2.5 and Junit4.12 along with JenkinsPlugin3.0

Kindly let me know if this can be resolved.

Details:
I tried to generate the Cucumber-Sandwich report from https://github.com/damianszczepanik/cucumber-sandwich

I still see the Issue with latest jar's and with the Cucumber JVM PluginVersion 3.0
https://wiki.jenkins-ci.org/display/JENKINS/Cucumber+Reports+Plugin

Given LotusApplication is Installed on connected Android Device015ms
Error message
java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(
Assert.java:144)
at StepDefinition.Test_Steps.LotusAppInstalled(Test_Steps.java:79)
at ✽.Given LotusApplication is Installed on connected Android Device(Feature/Test.feature:8)

Actual Source:

java.lang.AssertionError: expected: but was:
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at StepDefinition.Test_Steps.LotusAppInstalled(Test_Steps.java:79)
at ✽.Given LotusApplication is Installed on connected Android Device(Feature/Test.feature:8) 

Error: Unable to access jarfile cucumber-sandwich.jar while accessing command
"java -jar cucumber-sandwich.jar -f CucumberTestReport.json -o " from jar folder itself.

Hi,
What could be the reason, that some steps results are missing in cucumber scenarios report?
Actually in my case it affects Background steps

Thanks in advance for your help

To correctly handle UTF-8 in cucumber.json cucumber-sandwich.jar has to be called with -Dfile.encoding=UTF-8 despite the fact that JSON defaults to using UTF-8 and that the created HTML advertises text/html; charset=UTF-8 (this is on Windows 7 with a default encoding of Windows-1252).

It seems the manifest Main-Class entry is missing in the latest release

Hi This is my json. I am cucumber sandwich jar to generate report.My html report is not showing image when step failed..

I have added a image in step definition.

Droid_Automation_report.txt

Hi there, I love cucumber sandwiches 😄 Glad you are looking after the sandwich bar so well now that @kingsleyh is not working at the counter.

Here is my problem: I am running protractor at the moment with cucumber and have multi-capabilities configured in such a way that a json file gets produced for each capability (they would contain identical content). That's great and is working fine. However when I run Cucumber Sandwich I see the two files get detected but only a single set of artifacts get produced. How do I get a report per capability please?

Starting Cucumber Sandwich.....
Running once only as -n flag supplied.....
Found json reports: 2
/Users/nick/dev/git/client/web-ui/target/protractor/cucumber/json/os_x_10_10_safari_8_results.json
/Users/nick/dev/git/client/web-ui/target/protractor/cucumber/json/windows_7_chrome_47_results.json
About to generate Cucumber Report into: /Users/nick/dev/git/client/web-ui/target/protractor/cucumber/json/cucumber-html-reports
Finished generating Cucumber Report into: /Users/nick/dev/git/client/web-ui/target/protractor/cucumber/json/cucumber-html-reports

Oh, btw, I'm using the args: -f target/protractor/cucumber -o target/protractor/cucumber/json -n

It would be nice to have a toggle button in the menu to toggle on/off collapsible scenario steps so that you only say the scenario and the steps were hidden / visible by clicking on the button.

Clicking on the scenario to also toggle making the steps visible or not would also be great.

It would make it easier to scroll quickly to the failed scenarios

Below code,
embed("data:image/png;base64,#{encoded_img}",'image/png', "ERROR SCREENSHOT:#{time}:#{@feature_name}feature_at#{name_of_scenario}")

embeds screenshot in html report successfully but when i try to embed in json report, it fails.
is there a different approach to embed screenshot in cucumber sandwich report ?

I need to integrate cucumber-sandwich to my maven project and execute it from the framework after my test run , as i see from github it doesnt have the mechanism to run , not details on how to
integrate it with the POM file and execute it. Can you please guide , i am new to this ...

While trying run the generate report command as mentioned in the documentation ,we are getting below error not sure what iam missing here?

Saw the jar with 5.3.1 with dependencies created and renamed to cucumber-sandwich.jar and tried to run the jar file to test but getting below error all the time.

cucumber-sandwich thopukr$ java -jar cucumber-sandwich.jar
Error: Unable to access jarfile cucumber-sandwich.jar

Hey there,

So i was trying to use this plugin to generate the html reports, i ran the jar file ...But when i ran my cucumber scripts to genrate json file ...nothing was generated ...can u please help ?

Log.info is not showing logs under Step Defination files, is there Reported that i can use ?
i am using Latest version of Cucumber-Sandwich.

I tried to generate report in cmd: mvn clean verify. Then I got these error:

T E S T S

Running mgl.example.RunTest
@FIRST
Feature: Test drag-drop and scroll options

Scenario: Drag-drop and scroll must be OK # MoveAndScroll.feature:3
Given I move from the root role dialog to choosing picture # AddNewDevice.i_move_from_the_root_role_dialog_to_choosing_picture()
When I Choose picture and click continue # AddNewDevice.i_Choose_picture_and_click_continue()
Then I can drag-drop device and scroll options # AddNewDevice.i_can_drag_drop_device_and_scroll_options()

1 Scenarios (1 passed)
3 Steps (3 passed)
2m12.865s

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 133.918 sec - in mgl.example.RunTest

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ Example ---
[INFO] Building jar: /Users/enclaveit/Documents/workspace/Example/target/Example-0.0.1-SNAPSHOT.jar
[INFO]
[INFO] --- maven-cucumber-reporting:0.0.6:generate (execution) @ Example ---
About to generate
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:24 min
[INFO] Finished at: 2014-06-17T17:07:09+07:00
[INFO] Final Memory: 16M/200M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.masterthought:maven-cucumber-reporting:0.0.6:generate (execution) on project Example: Error Found: BUILD FAILED - Check Report For Details -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.masterthought:maven-cucumber-reporting:0.0.6:generate (execution) on project Example: Error Found:
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error Found:
at net.masterthought.cucumber.CucumberReportGeneratorMojo.execute(CucumberReportGeneratorMojo.java:86)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 19 more
Caused by: org.apache.maven.plugin.MojoExecutionException: BUILD FAILED - Check Report For Details
at net.masterthought.cucumber.CucumberReportGeneratorMojo.execute(CucumberReportGeneratorMojo.java:82)
... 21 more
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Can somebody help me? Thank you for your help!

Hi,

There seems to be an issue when there are duplicate screenshots that exist on the same page. Only the first instance of the screenshot on the page will actually display/maximize.

As an example I have:

Scenario 1

Given some stuff happens
Then I should be able to view this screenshot
Screenshot 1

Scenario 2

Given some stuff happens
Then something else might happen

Scenario 3

Given some stuff happens
Then I should be able to view this screenshot
Screenshot 1

In this case suppose the two screenshots are exactly the same. When I click on the screenshot link in Scenario 3 it will instead expand the screenshot in Scenario 1. This seems to be an ongoing issue and has been prevalent in the past few versions I have used.

This issue is exacerbated in the latest version given that scenario text is collapsed/hidden by default. In previous versions the page would shift vertically, hinting that something else had been expanded further up on the screen. Now that the reports are collapsed there is no hint that something else has expanded and even if you knew to look above you would not know which report to look in given that the screenshot is still hidden within a collapsed scenario.