you54f / cypress-multi-reporters Goto Github PK

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

We are using cypress in conjunction with ReportPortal which is an open source test reporting and aggregation solution. We are using agent-js-cypress plugin to push the test results and logs to ReportPortal, and cypress-multi-reporters to allow generating multiple log outputs at the same time (we need agent-js-cypress for ReportPortal and mocha, mochawesome for other systems).

Expected behavior

Expected is that regardless of the reporter used, complete output generated by cypress-terminal-reporter is shown in the output (either console or output file).

Actual behavior

When using cypress-multi-reporters with cypress-terminal-report, the majority of the logs are omitted. I think all the cons:log (K): calls are not shown in the output file or in the console; only the cy:command and cy:xhr calls are shown.

Information about the Issue

The same issue is logged with cypress-terminal-report project as well: archfz/cypress-terminal-report#28

Steps to reproduce the behavior

When using the below configuration, most parts of the logs are missing from the report.

Cypress.json:
"reporter": "cypress-multi-reporters", "reporterOptions": { "configFile": "cypress/reporter-config.prod.json" }

reporter-config.prod.json:
{ "reporterEnabled": "mochawesome,agent-js-cypress", "mochawesomeReporterOptions": { "reportDir": "cypress/reports/individual-reports", "overwrite": false, "html": false, "json": true }, "reporterOptions": { "endpoint": "https://reportportal.xxxx/api/v1", "token": "xxxx", "launch": "xxx", "project": "CanaryTests", "description": "Synthetic monitoring tests for the main user journey including perf measurements", "isLaunchMergeRequired": true, "debug": false, "attributes": [ { "value": "PROD" } ] } }

When using the below, all logs are present:
"reporter": "agent-js-cypress", "reporterOptions": { "endpoint": "https://reportportal.xxxx/api/v1", "token": "xxx", "launch": "xxx", "project": "CanaryTests", "description": "Synthetic monitoring tests for the main user journey including perf measurements", "isLaunchMergeRequired": true, "debug": false, "attributes": [ { "value": "localDEV" } ] }

Expected behavior

cypress-multi-reporters can be found

Actual behavior

Could not load reporter by name: cypress-multi-reporters
--
  |  
  | We searched for the reporter in these paths:
  |  
  | - /tribe-brand-cypress/cypress-multi-reporters
  | - /tribe-brand-cypress/node_modules/cypress-multi-reporters
  |  
  | The error we received was:
  |  
  | Cannot find module '/tribe-brand-cypress/node_modules/cypress-multi-reporters'

Information about the Issue

I'm trying to setup our automation CI/CD pipeline (Buildkite) to generate reports and integrate it to slack (cypress-slack-reporters).

Here's the cypress.json setup:

{
  "baseUrl": "http://172.16.0.6",
  "env": {
    "platform_host": "http://172.16.0.5",
    "brand_app_host": "http://172.16.0.6"
  },
  "pageLoadTimeout": 150000,
  "defaultCommandTimeout": 20000,
  "viewportWidth": 1200,
  "viewportHeight": 1000,
  "video": false,
  "chromeWebSecurity": false,

  "reporter": "cypress-multi-reporters",
  "reporterOptions": {
    "reporterEnabled": "mochawesome",
    "mochawesomeReporterOptions": {
        "reportDir": "cypress/reports/mocha",
        "quiet": true,
        "overwrite": false,
        "html": false,
        "json": true
    }
  }
}

Here's package.json with the currently installed modules

{
  "name": "tribe-brand-cypress",
  "version": "1.0.0",
  "description": "QA cypress tests for new brand app",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "cy:run": "cypress run --browser chrome",
    "clean:reports": "rm -R -f cypress/reports && mkdir cypress/reports && mkdir cypress/reports/mochareports",
    "pre-test": "npm run clean:reports",
    "combine-reports": "mochawesome-merge cypress/reports/mocha/*.json > cypress/reports/mochareports/report.json",
    "generate-report": "marge cypress/reports/mochareports/report.json -f report -o cypress/reports/mochareports",
    "post-test": "npm run combine-reports && npm run generate-report",
    "slack-reporter": "npx cypress-slack-reporter --report-dir=cypress/reports/mochareports --ci-provider=custom --custom-url=cypress/reports/mochareports/report.html"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/TribeInfluence/tribe-brand-cypress.git"
  },
  "keywords": [
    "cypress"
  ],
  "author": "oscar",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/TribeInfluence/tribe-brand-cypress/issues"
  },
  "homepage": "https://github.com/TribeInfluence/tribe-brand-cypress#readme",
  "devDependencies": {
    "cypress": "^3.8.3",
    "cypress-multi-reporters": "^1.5.0",
    "cypress-slack-reporter": "^1.2.1",
    "gmail-tester": "^1.1.0",
    "mocha": "^9.0.3",
    "mochawesome": "^6.2.2",
    "mochawesome-merge": "^4.2.0",
    "mochawesome-report-generator": "^5.2.0"
  },
  "dependencies": {
    "xhr": "^2.5.0"
  }
}

Steps to reproduce the behavior

The dockerfile used in the automation CI/CD pipeline also includes the installation of the packages needed for the report:

Based on the logs, cypress-multi-reporters is installed successfully so I'm not sure where my config went wrong

The issue cannot be reproduced when ran locally.

Expected behavior

The mocha-junit-reporter reporter has a nice feature to allow dynamic substitution of a portion of the file name (if "[hash]" is found within mochaFile, it will be auto-substituted with a unique hash). Cypress docs also provide an example of its use: https://docs.cypress.io/guides/tooling/reporters.html#Merging-reports-across-spec-files

In the context of multi-reporters, Cypress has an example of this also (again with mocha-junit-reporter): https://docs.cypress.io/guides/tooling/reporters.html#Multiple-reporters.

I would like to see a more generic mechanism that:

1. Works across different reporters, even if those reporters didn't have built-in support for such within-file-name substitutions. For example, I am using https://github.com/mocha-community/json-file-reporter/ to build JSON files with Mocha's built in simple JSON reporter (no need for mocha-junit-reporter or mochawesome), but this reporter does not itself allow substitutions within file names.
2. Does not limit substitutions within such (file name) properties to hashes, but allows a reporter option to pass in a literal string. For example, in https://github.com/indexeddbshim/indexeddbshim , I am using this approach with a fork of mocha-multi-reporters to add a test group type to the resulting file names, so that the multiple generated file names can be more human readable, e.g., building file-mock.json, file-fake.json, etc., depending on the test type (having human readable names can still be useful to get at even if one is typically building separate files with the intent to eventually merge them). Even if one can pass in the whole file name reporter option such that cypress-multi-reporters can pick it up, this would be less than ideal in that it can be nice to store the longer base path in the config file, and just specify the test file group in the command line call. So in indexeddbshim, I only need my CLI call to be mocha --reporterOptions configFile=mocha-multi-reporters.json,mmr-output=-mock which gets applied to the MMR config file (JSON below) whereby my MMR fork substitutes {num} (which I probably should have named {id} or possibly repurposed "[hash]" from the junit reporter) with the mmr-output CLI argument (i.e., here "mock") to dynamically substitute the output value (if implementing, we might want a config option to indicate which reporters and which of their properties should get dynamic substitutions); there is no need to respecify test-support/results/file... on the command line:

{
  "mochajsJsonFileReporterReporterOptions": {
     "output": "test-support/results/file{num}.json"
  }
}

(Perhaps a config could also be set to auto-set a GUID if that were desired by other projects.)

Actual behavior

I am otherwise not able to build multiple Mocha result files, unless using the junit reporter or avoiding taking advantage of multiple reporters at the same time.

While I could file this as an issue to https://github.com/mocha-community/json-file-reporter/ and hope they implement, I would think this may be a common enough need, especially when having a complex enough set-up to want multiple reporters, to make a generic mechanism.

Thanks!

Expected behavior

I can put my config files into cypress/config folder and have separate files for each environment but then this plugin doesn't work

Actual behavior

Getting an error saying:
Cannot set property 'mochaFile' of undefined
TypeError: Cannot set property 'mochaFile' of undefined

Information about the Issue

Steps to reproduce the behavior

1. Create config folder in cypress and configure plugin to use config files based on env variable
2. Run tests with cypress-multi-reporters as a reporter and see above error is thrown

Expected behavior

Having users install mocha in their package.json is undesirable and causes problems; see also: cypress-io/cypress#3537

Having cypress installed is sufficient, because cypress installs the version of mocha it wants.

Expected behavior

Finds cypress-multi-reporters in any package.json/node_modules anywhere higher in the enclosing directories. Even if a package.json or node_modules is found, keep searching upwards in enclosing directories.

Actual behavior

Could not load reporter by name: cypress-multi-reporters

We searched for the reporter in these paths:

Information about the Issue

In monorepos (such as those made by @nrwl/nx that I am using) the tendency is to place shared dependencies in a top-level package.json so they don't have to be duplicated all over the place. cypress-multi-reporters can't find them there, because I have a cypress folder for each app in the monorepo, now each of those needs its own node_modules with redundant copies of cypress-multi-reporters to install and maintain for each.

Steps to reproduce the behavior

1. Build an Nx-angular monorepo with cypress e2e
2. install cypress-multi-reporters in the top level package.json
3. Try to get reports working

And while I'm here allow me to strongly second the feature request of including config in the cypress.json rather than forcing CLI use. Thanks for this handy tool!

If reporterEnabled of config file is an array, use it as is, without calling split(',').

{
  "reporterEnabled": "spec,json"
}

{
  "reporterEnabled": [
    "spec",
    "json"
  ]
}

Wasted a bit of time trying to get this to read my reporter config json file, which kept returning unexpected ' '. Found that adding this trim() fixed the issue.
I switched to using a .js config file, but this might keep someone else from a moments frustration...
#170

Expected behavior

reporter search should respect NODE_MODULE

Actual behavior

only search ${PWD}/cypress-multi-reporters and ${PWD}/node_modules/cypress-multi-reporters

Information about the Issue

I have global installed cypress and cypress-multi-reporters. by setting NODE_MODULE, cypress can load other modules but cannot load multi reporter

Just a heads up, I will be migrating this repository over to https://github.com/YOU54F/cypress-plugins to live alongside my other cypress plugins.

The name of the package won't change, but I am going to delete the repo rather than archive it, and it won't redirect, as the cypress-plugins repo exists (it was renamed, from cypress-slack-reporter)

We are running cypress on multiple localized version of the website.

It would be useful to be able to use plugins/index.js to set options for the reports.

For example with :

plugins/index.js:

module.exports = (on, config) => {
    
  // some logic to det `locale`

  config.mochawesomeReporterOptions = {
    reportDir: `.reports/${locale}`,
  }

  config.mochaJunitReporterReporterOptions = {
    mochaFile: `./junit-${locale}.xml`,
  }
  
   return config;
}

This may be simpler to implement than #101:

In order to dynamically configure the reporters instead of having a static JSON file it would be great to be able to use full Javascript. Think .babelrc vs .babelrc.js. In this way the path of the reporter output could be dynamically read from an environment variable.

should .reports be ./reports in the README?

Expected behavior

Hi,

I would like to be able to have reporter behaviors run asynchronously while still being able to exit when complete.

I've added this to the end of the MultiReporters constructor (within the execute block) and it has been working fine:

        return Promise.all(this._reporters).then(function () {
          process.exit();
        });

Actual behavior

My reporters (e.g, to build mocha badges) otherwise either cannot work asynchronously or do not know when to exit (unless they coordinate with one another).

Would you be open to a PR to add this behavior?

Thanks!

Dear cypress-multi-reporters maintainers,
Thank you for your contribution to the open-source community.

This issue was automatically created to inform you a new version (1.6.1) of cypress-multi-reporters was published without a matching tag in this repo.

As part of our efforts to fight software supply chain attacks, we would like to verify this release is known and intended, and not a result of an unauthorized activity.

If you find this behavior legitimate, kindly close and ignore this issue. Read more

Expected behavior

I can utilize cypress-multi-reporters to employ both @reportportal/agent-js-cypress and mochawesome in my project.

Actual behavior

I am presented with the following error when executing my tests:

Fail to start launch Error: connect ECONNREFUSED 127.0.0.1:80
    at [Path to Project]/node_modules/@reportportal/client-javascript/lib/rest.js:31:23
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Information about the Issue

I believe that cypress-multi-reporters may be pointing to the wrong child folder of the @reportportal module. The error implies it's pointed to @reportportal/client-javascript when it should be looking at @reportportal/agent-js-cypress.

My cypress.json file includes:

    "reporter": "cypress-multi-reporters",
    "reporterOptions": {
        "configFile": "reporter.json"
    },
    "supportFile": "cypress/support/index.js"

My reporter.json file:

{
    "reporterEnabled": "@reportportal/agent-js-cypress, mochawesome",
    "agentJsCypressReporterOptions": {
        "endpoint": "http://localhost:8080/api/v1",
        "token": "35c84358-5bb2-419f-9bd7-3cafa424db74",
        "launch": "Cypress Test Run 2020.09.21",
        "project": "Pricing",
        "description": "Test run from [Project Name]",
        "isLaunchMergeRequired": true
    },
    "mochawesomeReporterOptions": {
        "reportDir": "cypress/reports/mocha",
        "quite": true,
        "overwrite": true,
        "html": false,
        "json": true
    }
}

Steps to reproduce the behavior

1. Implement cypress-multi-reporters with @reportportal/agent-js-cypress and mochawesome as two reporters
2. Run Cypress tests

This may be related to #43.

Here is my cypress.json:

"reporter": "cypress-multi-reporters",
"reporterOptions": {
  "reporterEnabled": "spec, @mochajs/json-file-reporter",
  "mochajsJsonFileReporteReporterOptions": {
    "output": "/project/cypress/report.json"
  }
}

This error is thrown before tests start running:

TypeError: Cannot set property 'output' of undefined
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:344:37
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:4905:15
    at baseForOwn (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:2990:24)
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:4874:18
    at Function.forEach (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:9342:14)
    at Object.setPluginResolvedOn (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:338:16)
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:342:26
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:4905:15
    at baseForOwn (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:2990:24)
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:4874:18
    at Function.forEach (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:9342:14)
    at Object.setPluginResolvedOn (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:338:16)
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:342:26
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:4905:15
    at baseForOwn (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:2990:24)
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:4874:18
    at Function.forEach (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/lodash/lodash.js:9342:14)
    at Object.setPluginResolvedOn (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:338:16)
    at Object.updateWithPluginValues (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/config.js:383:14)
    at /caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/lib/project.js:126:38
    at tryCatcher (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/caches/Cypress/4.5.0/Cypress.app/Contents/Resources/app/packages/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (internal/timers.js:439:21)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency sinon to v15

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update dependency chai to v4.3.7
• chore(deps): update commitlint monorepo to v17 (major) (@commitlint/cli, @commitlint/config-conventional)
• chore(deps): update dependency eslint to v8
• chore(deps): update dependency husky to v8
• chore(deps): update dependency mocha to v10

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Expected behavior

The reporterOptions set in report-config.json file should be considered

Actual behavior

The options set in repor-config are not being considered

Information about the Issue

I'm trying to use mochawesome as well as mocha-junit-reporters for generating both html as well as xml reports for our tests. I specified cypress-multi-reporters as the reporter in cypress.json and in the commandline, I'm passing the report-config file.
I get both the reports but they're being created in the default locations and html:false is also not being considered.
Please take a look at both the files.

cypress.json

{
    "baseUrl": "http://localhost:4200",
    "testFiles": "**/*.test.js",
    "chromeWebSecurity": true,
    "requestTimeout": 90000,
    "video": false,
    "reporter": "../../../node_modules/cypress-multi-reporters",
    "firefoxGcInterval": null
}

chromeReportconfig

{
  "reporterEnabled": "../../../node_modules/mochawesome, ../../../node_modules/mocha-junit-reporters",
  "reporterOptions": {
    "mochawesomeReporterOptions": {
      "reportDir": "results/results_chrome/",
      "quiet": true,
      "overwrite": false,
      "html": false,
      "json": true
    },
    "mochaJunitReporterReporterOptions": {
      "mochaFile": "results/junits/test_results[hash].xml",
      "toConsole": false
    }
  }
}

finally starting the tests with this
cypress run --headless --browser chrome --reporter-options configFile=chromeReportconfig.json --project ./cypress/integration --spec ./cypress/integration/cypress/integration/home/**/*.test.js

Steps to reproduce the behavior

1. Run any test with above config.

Appreciate your time and help here. Thanks

Is there some way of specifying options for scoped reporters?
I have this reporter.config.json

{
  "reporterEnabled": ["mochawesome", "@org/custom-rep"],
  "mochawesomeReporterOptions": {
    "charts": true,
    "overwrite": false,
    "html": false,
    "json": true,
    "reportDir": "reports"
  },
  "@org/customRepReporterOptions": {
    "arg1": true
  }
}

Note: I have also tried using just "customRepReporterOptions".

Expected behavior

customRepReporterOptions should be passed to the @org/custom-rep reporter.

Actual behavior

The reporter doesn't receive the options.

Information about the Issue

Everything worked until I added the scope to my custom reporter.