prma85 / cypress-teamcity-reporter Goto Github PK

Describe the bug

When we are using the new Rule syntax in a feature file, runner.on('test end') breaks with the described behaviour.

To Reproduce

Steps to reproduce the behavior:

1. Create a simple scenario that has a Rule keyword around it

Rule: I see the temporary messages grouped based on their status

    @dev @test
    Scenario Outline: Hide messages section when empty
      Given I do not have a message
      When I go to the "/message" path
      Then I do not see the "<message group>" temporary messages section

3. Run the scenario
4. See the reporter error

Expected behavior

• Ignore any Rule syntax from Gherkin 6 and up, and simply jump to the next step.
• Make it safe with the ? operator to let the runner step to the next scenario/block

Actual behavior

runner.on('test end', function (test)..)

test.parent.parent.file. is undefined, hence calling replace on it breaks the runner.

Screenshots

Running:  TemporaryMessagesEdit/TemporaryMessagesEdit.feature                             (1 of 3)
##teamcity[testSuiteStarted name='Temporary messages - Edit existing temporary messages' flowId='69656']
##teamcity[testSuiteStarted name='I want to have a UI in Phoebe to edit existing temporary messages' flowId='69656']
##teamcity[testStarted name='Step back to Temporary Messages Overview from the messages details | ' captureStandardOutput='true' flowId='69656']
##teamcity[testFinished name='Step back to Temporary Messages Overview from the messages details | ' duration='1318' flowId='69656']
Cannot read properties of null (reading 'replace')
TypeError: Cannot read properties of null (reading 'replace')
    at L.<anonymous> (/../node_modules/cypress-teamcity-reporter/lib/teamcity.js:136:115)
    at L.emit (node:events:525:35)
    at A.emit (<embedded>:4479:18259)
    at Object.onMocha (<embedded>:4526:27943)
    at p.<anonymous> (<embedded>:4479:47429)
    at p.emit (node:events:513:28)
    at <embedded>:4479:26630
Cypress exited with code 1

Environment

• Device: [Desktop]
• OS: [MacOS Sonoma 14.1.2]

Additional context

Reference for Gherkin 6 Rule

Current behavior

For projects that do not yet use / cannot use the Cypress Dashboard, the current way to save test evidence TC is to register screenshots and videos as Build Artifacts. This makes them discoverable from the Build page, but associating them with failing tests requires mentally mapping the filenames to the failing tests, which is a bit of a hassle.

Describe the solution you'd like

TC has built-in support for associating metadata with tests, including (since 2018) screenshots. It would be nice if we could hook into the screenshot event and emit the testMetadata type='image' service message. A tangential and equally missing feature is artifacts in the first place: there should be an option to turn on "Automatically publish test evidence as artifacts" so that the user does not have to reinvent the proper Ant-style search pattern each time.

Describe alternatives you've considered

Doing this manually after-the-fact (i.e. after Cypress finished, find each media file and echo something) is near impossible because test by file names (suite, test, retries...) lose all case-information and has _-replaced chars.

Additional context

Since videos are recorded one-per-suite, I do not think there is any way to make these easily included - apart from registering one and the same video for each test (could be better than nothing). There is no suite-level metadata in TC.

Describe the solution you'd like

Now since Cypress version 5.x.x when test retries is more of a first-class feature, it would make sense to report retries per test as custom test metadata, and the total number of retries as a build statistic.

🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.

I recently updated one of my projects to use Cypress 5.2.0 and started seeing tests with failed assertions showing up as green in TeamCity. I have not yet had an opportunity to make a minimal reproducible example, so I'm just throwing this out here: has anyone else noticed any similar issue?

Expected: 5 tests, 4 passing, 1 failing as per TC Build Log excerpt below:

  Running:  reports.spec.js                                                                 (3 of 3)
    Reports area 1s
    (Results)
    ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
    │ Tests:        5                                                                                │
    │ Passing:      4                                                                                │
    │ Failing:      1                                                                                │
    │ Pending:      0                                                                                │
    │ Skipped:      0                                                                                │
    │ Screenshots:  3                                                                                │
    │ Video:        true                                                                             │
    │ Duration:     18 seconds                                                                       │
    │ Spec Ran:     reports.spec.js                                                                  │
...

Actual: 5 tests, 5 passing, 0 failing as per TC Tests tab screenshot below:

Is your feature request related to a problem? Please describe

I think it would be nice to add to the README what the differences / advantages of using this module is, as compared to the @cypress-io/mocha-teamcity-reporter one that comes bundled with Cypress, which itself seems to be a few years' worth of bugfixes behind its upstream version.

Is your feature request related to a problem? Please describe

Describe the solution you'd like

Describe alternatives you've considered

Additional context