Cypress Lighthouse test failing after latest Chrome Version Upgrade (Version 111.0.5563.64 ) about cypress-audit HOT 30 CLOSED

Seeing similar behavior. Lighthouse 1.3.1 appears to crash chrome (headless or not) with the following message:

The automation client disconnected. Cannot continue running tests.

On an older version of Cypress (12.3.0) it was closing the browser causing a different log message in the Cypress, equivalent to closing the browser manually while testing.

from cypress-audit.

Should be fixed in 1.4.2 sorry for that

from cypress-audit.

Yes @brandonlenz - This is exact issue I am having.

@paul-s1 - I believe if you run on any other browser except Chrome the cypress test might pass however it wont do the Lighthouse Audit. I tried now again with Edge, Electron and it gave message similar to below for those browsers.

log cy.lighthouse(), Edge is not supported. Skipping...

from cypress-audit.

@mfrachet my tests using cy.pa11y and cy.lighthouse are now working on 1.4.1 🚀

from cypress-audit.

humf, since there's not build step implied, the module tries to resolve the sibling workspace that is not shipped in the lighthouse artifact. I'll take a look tomorrow morning 🤦🏻

from cypress-audit.

Cool - Thanks
Let me check it out

from cypress-audit.

Thanks @mfrachet @MuckT - I probably should have done this sooner. :(
https://github.com/tsathaye/lighthouseaudit started working after upgrading node

"@cypress-audit/lighthouse": "^1.4.2", "@cypress-audit/pa11y": "^1.4.2", "cypress": "^12.9.0"

And with following

npm -v 9.5.1
node -v = v18.16.0

from cypress-audit.

I have also experienced the same. The error message is different in other Cypress versions and likely a similar issue. I was able to get it to run using Edge (Version 111.0.1661.41), although it fails the first attempt. subsequent attempts of the tests will run with results.

• OS: macOS 12.5.1
• Cypress version: 11.2.0
• Node: v16.18.0
  • when I upgraded to v18.13.0, I got this error on each test, thought maybe it was related to #62 :

From Node.js Internals:
      at <unknown> (      Error: connect ECONNREFUSED ::1:59700)
      at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1487:17)

I'm having better results when I download Chromium 110.0.5481.0 and Node v16.18.0 use that for browser running the test

from cypress-audit.

@brandonlenz I also remember seeing that error, I can't remember the combination of Cypress version and Chrome version but I do remember when I ran the headless version, i was able to console.log the Lighthouse results from the first test, then it just hung. were you able to see the same?

from cypress-audit.

Based on this issue filed in the Lighthouse GitHub repo, it looks like this is an issue with lighthouse version 9. It looks like moving to the newly-released version 10 of lighthouse may correct the issue.

from cypress-audit.

I am observing the same issue, it was working for me with cypress version 6 & chrome version 111 few weeks ago when I was exploring lighthouse option for our project, but we focused on upgrading cypress to version 12.7.0. After the upgrade when I am trying to execute my existing sample tests, they are erroring out with the same above attached screenshot.

from cypress-audit.

FYI I tested upgrading this package to version ^10.0.2 of lighthouse and it worked. The breaking changes announced for version 10 here don't seem to affect this package's functionality much.

from cypress-audit.

from cypress-audit.

I pushed the version to npm 5 mns ago

from cypress-audit.

It looks like the author of the last PR mentioned he was still seeing issues with this after his latest changes went it.

In my local testing for moving this library to [email protected], I had to change the require() statement in task.js to pull in the CommonJS module version of lighthouse since they converted the default import the ESM as part of the major verison 10 release. If issues are still being observed, we may need to tweak this line to be require('lighthouse/core/index.cjs') instead. You can see release notes here), specifically this part:

Converted from CommonJS to ES modules. You can still use lighthouse in CommonJS by using an dynamic import: await import('lighthouse'). For access to just the lighthouse function in CommonJS, you can also use require('lighthouse/core/index.cjs')

from cypress-audit.

Yeah, I had issues also between ESM and commonjs. Not sure which one to pick since I suppose to every project is different

I can do both, but I ll be forced to release two different majors.

And I don't have that much time to maintain this library as you can see

from cypress-audit.

@mfrachet tested with the change @430010838 mentioned and Cypress audit is working locally in my case. I missed this in my last PR and I think we just want the behavior in #213 instead of releasing to different majors. There were some additional changes in the PR to get it to work, such as using legacyNavigation, but hopefully this addresses the issue.

from cypress-audit.

Should be fixed in 1.4.1 can you confirm?

from cypress-audit.

Thanks again for your help 🙏🏻

from cypress-audit.

1.145s omerfeigenbaum ❯ npm i @cypress-audit/lighthouse
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:*

with yarn -W I can install but not with npm

from cypress-audit.

Thanks @mfrachet for the releasing 1.4.2

After upgrading dependencies to following
"@cypress-audit/lighthouse": "^1.4.2",
"@cypress-audit/pa11y": "^1.4.2",
"cypress": "^12.8.1",

I am NOT getting the error
The Cypress extension has disconnected.
Cypress cannot run tests without this extension.

However now the test is failing with below

cy.task('lighthouse') failed with the following error:

Unexpected token '||='Learn more
node_modules/@cypress-audit/lighthouse/src/command-handler.js:46:1
44 | cy.log("-------- cy.lighthouse --------");
45 | return cy
46 | .task("lighthouse", {
| ^
47 | url,
48 | thresholds: thresholds || configThresholds || defaultThresholds,
49 | opts: opts || globalOptions,

Is there any change required in the calling function?
Or is this a separate issue which needs to be logged?

from cypress-audit.

@tsathaye have you tried removing your node modules with rm -rf ./node_modules and then reinstalling?

from cypress-audit.

Yes @MuckT - Tried installing after deleting node modules however still having the issue.
Is it possible to share a Cypress test repo where you have this working with latest version of Lighthouse?

from cypress-audit.

This repository https://github.com/mfrachet/cypress-audit/tree/master/examples

$ git clone https://github.com/mfrachet/cypress-audit
$ cd cypress-audit
$ pnpm i
$ cd examples/nextjs
$ pnpm run dev
$ pnpm run e2e:audit # in another terminal

from cypress-audit.

I tried the above repo also tried to replicate that in our test - However wasn't able to get it working

1. Then I created a sample repo https://github.com/tsathaye/lighthouseaudit
2. Copied files & single test from NextJS Example folder
3. Last thing I did was update the test to use http://cypress.io/ as base url

And am still getting the same error in this repo too - Can you please review when you have a moment

Thanks,
Tushar

from cypress-audit.

I honestly don't know what's going on, your repo works on my side 😓

from cypress-audit.

had the same issue, this settings help me, in cypress 12.9V working well, (lego.com performance is really low :) )

https://mfrachet.github.io/cypress-audit/guides/lighthouse/installation.html#installing-the-dependency

from cypress-audit.

@tsathaye have you tried clearing your Cypress cache? Cypress troubleshooting.

from cypress-audit.

Yes @MuckT - Tried clearing / reinstalling Cypress on my Mac now however the issue persists. :(

from cypress-audit.

@tsathaye I've tried your branch locally and it's working well. Have you tried force install to make sure you're getting the remote repos even if some exist on your local?
npm install --force?

Running:  homepage.cy.js                                                                  (1 of 1)


  The App
    1) audits the home page


  0 passing (38s)
  1 failing

  1) The App
       audits the home page:
     Error: cy.lighthouse - A threshold has been crossed.

performance record is 36 and is under the 50 threshold

from cypress-audit.