[Bug]: x1.5 CPU usage increase using Puppeteer between 20.8.3 to 20.9.0+ about puppeteer HOT 12 OPEN

Thanks for the details, could you reproduce the issue outside of the server? Because I won't be able to accurately measure cpu usage increase without the actual workload and the server configuration (which CPUs, how many cores, anything else running there). Could you please update the example to measure the time spent similar to #11944? Also, it would be great to understand if you see the cpu increase in the node process or in the chrome process.

from puppeteer.

@MathieuLamiot yes, you can specify the path to the browser you want to run via executablePath in the launch command. That should generally works unless there were some changes in the browser that required Puppeteer changes that are not backward compatible.

from puppeteer.

The issue has been labeled as confirmed by the automatic analyser.
Someone from the Puppeteer team will take a look soon!

Analyzer run

from puppeteer.

@mostafa-hisham could you check if you see the difference with 21.1.1 to 21.2.0? is it the same as #11944 ?

from puppeteer.

@mostafa-hisham could you please also clarify if the increased CPU usage is for the node process running Puppeteer or for the browser process(es)?

from puppeteer.

@OrKoN

could you please also clarify if the increased CPU usage is for the node process running Puppeteer or for the browser process(es)?

CPU usage in the graphs is for container CPU usage at this time.
it should include both node running Puppeteer and browser process(es)

• Green (mirror) -> v21.1.1 (mirror at 30% CPU usage)
  

CPU is lower but many pages crash with this error TargetCloseError Protocol error on stoping CSSpagecoverage
that means that pages crash before any processes are done on puppeteer like coverage and evaluation so it is not very representative.

TargetCloseError: Protocol error (CSS.stopRuleUsageTracking): Session closed. Most likely the page has been closed.
2024-06-03T12:52:35.147148385Z     at CDPSessionImpl.send (/app/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Connection.js:461:35)
2024-06-03T12:52:35.147151791Z     at CSSCoverage.stop (/app/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Coverage.js:293:57)
2024-06-03T12:52:35.147153755Z     at Coverage.stopCSSCoverage (/app/node_modules/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Coverage.js:116:40)

from puppeteer.

Hello @OrKoN I work with @mostafa-hisham.

could you check if you see the difference with 21.1.1 to 21.2.0? is it the same as #11944 ?

We though about that initially, but in #11944, it seems you found precisely the version update that introduce the issue. In our case, we see the issue when going from 20.8.3 to 20.9 so it seems different.
As @mostafa-hisham explained above, our code crashes with 21.1.1 so the test is not very useful, but since 20.9 is already impacted, I guess we have a different issue than #11944?

from puppeteer.

So I finally got a chance to run it. I have reduced the script a bit:

import puppeteer from "puppeteer";

async function main() {
  const browser = await puppeteer.launch({
    headless: true, 
    pipe: true,
  });
  const allPages = await browser.pages();
  const page = allPages.length ? allPages[0] : await browser.newPage();
  await page.setRequestInterception(true);
  page.on("request", (req) => {
    if ("wpr-saas-no-intercept" in req.headers()) {
      req.continue(); // Exception for LCP Beacon script.
    } else if (
      (req.resourceType() !== "script" &&
        req.resourceType() !== "stylesheet" &&
        req.resourceType() !== "document") ||
      (req.resourceType() === "script" && req.initiator().type === "script")
    ) {
      req.abort();
    } else {
      req.continue();
    }
  });
  const startTime = performance.now();
  await Promise.all([
    page.coverage.startCSSCoverage(),
    page.coverage.startJSCoverage(),
  ]);
  await page.goto("https://google.com", { waitUntil: "load" });

  const elements = await page.$$("*");
  await Promise.all(
    elements.map((elem) => elem.evaluate((el) => el.className)),
  );

  await Promise.all([
    page.coverage.stopCSSCoverage(),
    page.coverage.stopJSCoverage(),
  ]);
  const time = performance.now() - startTime;
  await browser.close();
  return time;
}

let n = 10;
let sum = 0;

for (let i = 0; i < n; i++) {
    sum += await main();
}

console.log(sum / n);

// 20.8.3 = 717.2703088 
// 20.9.0 = 707.1119905999997

I do not see a difference in time spent. Can you try the reduced script, do you still see the issue with? If yes, do you see a different time spent on the task or only the CPU usage?

Note that I removed custom args as the support level for them is varying and to rule out that they are not the cause of the issue.

UPD: same results with the same args

from puppeteer.

In the latest version, my average time is 1014 but it is likely the other issue.

from puppeteer.

Thanks for the update and looking into it.
We are working on providing more insights:

• We will check time measurements on our servers that report higher CPU, to see if time measurment would also spot the issue.
• Then we will rework the script above to reproduce locally. We think that we need to browse different pages, we have lists of what our servers work on so we can build from it.

We'll get back to you with that.

In the meantime, is there a way for us to run Puppeteer v20.9 with a browser version from Puppeteer v20.8? It would help us see if the issue comes from the puppeteer library or the browser itself?

from puppeteer.