everettss / puppeteer-har Goto Github PK

Please refers to sitespeedio/chrome-har#15 for details on the issue

chrome-har now supports returning response body as part of har. This option defaults to false in chrome-har. Can you add support to return response body via puppeteer-har?

Hello,

I'm using [email protected] [email protected] and [email protected]

const puppeteer = require('puppeteer');
const PuppeteerHar = require('puppeteer-har');

(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
console.log("create page");

const har = new PuppeteerHar(page);
await har.start({ path: 'results.har' });
console.log('Har started');

await page.goto('https://********/portail');

await har.stop();
await browser.close();
})();

How can I stop having empty "pageTimings" please ?

{"log":{"version":"1.2","creator":{"name":"chrome-har","version":"0.11.7","comment":"https://github.com/sitespeedio/chrome-har"},"pages":[{"id":"page_1","startedDateTime":"2020-04-29T14:00:06.681Z","title":"https://********/portail","**pageTimings":{}**}],"entries":[{"cache":{},"startedDateTime":"2020-04-29T14:00:06.682Z","_requestId":"5380B4DD08F8636FEB1F2EC3A5914CE6","_initialPriority":"VeryHigh","_priority":"VeryHigh","pageref":"page_1","request":{"method":"GET","url":

What do you think about adding support for playwright?

Maybe just another option to start(). Like start({path:.., playwright: true})

Then:

if (playwright) {
 this.client = await this.page.context().newCDPSession(this.page);
} else {
 this.client = await this.page.target().createCDPSession();
}

Hi, pageTimings is empty in har file, it cause I can not get onLoad time and onContendLoad time. Could you please help to have a look? Thanks in advance.
here is my script
describe("use puppeteer to get har file ", () => {
it("use more launch properties demo", async () => {
if (generate.getConfigs().isWindows) {
configs = generate.getConfigs().windows;
} else {
configs = generate.getConfigs().macOrLinux;
}
browser = await puppeteer.launch({slowMo: '1000', headless: false, executablePath: configs.chromePath,args: [--window-size=1920,1080]});
page = await browser.newPage();
har = new PuppeteerHar(page);
await page.setViewport({width: 1920, height: 1080});
await page.setDefaultTimeout(configs.timeout)
/*
* har1
* /
await har.start({path: path.join(path.resolve(), configs.bmcReportPath.scenario1+'1.har')});
await page.goto("https://www.biomedcentral.com/");
await page.waitFor(5000)
await har.stop();
/
* har2
* */
await har.start({path: path.join(path.resolve(), configs.bmcReportPath.scenario1+'2.har')});

    //get DOM element whose innertext is Search and clcik
    await page.evaluate(() => {
        const elements = [...document.querySelectorAll('span')];
        const targetElement = elements.find(e => e.innerText == 'Search');
        if (targetElement) targetElement.click();
    });
    await page.$eval('#publisherSearch', el => el.value = 'carbon neutral');
    await page.keyboard.press('Enter');
    await page.waitFor(5000)
    await har.stop();
    /*
    *
    * */
    await page.close();
    await browser.close();
});

});

here is my hare file with puppeteer-har tools, pageLoadTime is empty.
{
"log": {
"version": "1.2",
"creator": {
"name": "chrome-har",
"version": "0.11.12",
"comment": "https://github.com/sitespeedio/chrome-har"
},
"pages": [
{
"id": "page_1",
"startedDateTime": "2021-12-15T12:00:23.038Z",
"title": "https://www.biomedcentral.com/",
"pageTimings": {}
}
],
If I download har file manually, pageTimes is not empty. here is har file manually.
{
"log": {
"version": "1.2",
"creator": {
"name": "WebInspector",
"version": "537.36"
},
"pages": [
{
"startedDateTime": "2021-12-14T07:02:58.380Z",
"id": "page_17",
"title": "https://www.biomedcentral.com/",
"pageTimings": {
"onContentLoad": 263.5190000291914,
"onLoad": 22470.13099998003
}
}
],

I'm using the code from the example to generate a HAR for a specific page. The generated HAR is ~40 KB. When I use browsermob-proxy in python for the same page the generated HAR is ~1.5 MB. At least 1 html file is missing from the generated HAR from puppeteer-har.
Why is this? Is there a maybe an option to avoid it?
Even if use a wait function like in #5, I'm getting the same result.

My netsniffPrototype.js :

await this.page.goto(url, {timeout: 60000});
let cssSelectorconsent = '#consent-notice-agree-button';
let cookieAcceptBtn = await this.page.waitForSelector(cssSelectorconsent, { timeout: 60000, visible: true });
await cookieAcceptBtn.click();
await this.page.setCacheEnabled(false);
await this.harStart();
await this.page.reload({waituntil: networkidle2, timeout: 60000});
await this.harStop();

Add console.log in chrome-har/lib/entryFromResponse.js

const entrySecs =
page.__wallTime + (timing.requestTime - page.__timestamp);
console.log('page.__wallTime = ' + page.__wallTime + ' timing.requestTime = ' + timing.requestTime + ' page.__timestamp = ' + page.__timestamp)
entry.startedDateTime = dayjs.unix(entrySecs).toISOString();
console.log('dayjs.unix(entrySecs).toISOString() = ' + dayjs.unix(entrySecs).toISOString())

Stdout results (Python call nodejs script using subprocess.run() ) :

2020-02-24:12:39:28,092 INFO [netsniffPrototype.js:296] <Thread(Thread-1, started daemon 140015018862336)> Stop Har events

page.__wallTime = 1582544363.28644
timing.requestTime = 6971.141358
page.__timestamp 6971.138063
dayjs.unix(entrySecs).toISOString() = 2020-02-24T11:39:23.289Z

page.__wallTime = undefined
timing.requestTime = 6971.149286
page.__timestamp = undefined

2020-02-24:12:39:28,101 ERROR [netsniffPrototype.js:303] <Thread(Thread-1, started daemon 140015018862336)> Process exitHarNotStopped event Har not stopped properly !
e.stack => RangeError: Invalid time value
at Date.toISOString ()
at h.d.toISOString (/home/pptruser/node_modules/dayjs/dayjs.min.js:1:6372)
at module.exports (/home/pptruser/node_modules/chrome-har/lib/entryFromResponse.js:161:53)
at harFromMessages (/home/pptruser/node_modules/chrome-har/index.js:310:15)
at PuppeteerHar.stop (/home/pptruser/node_modules/puppeteer-har/lib/PuppeteerHar.js:109:21)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:97:5)

Dear Michał,

thank you for providing puppeteer-har. The European Data Protection Supervisor uses this in its free software tool to help website developers and data protection professional to detect data protection compliance issues.

In the past, I have used a fork of your code to include a patched version from chrome-har, which ensures that HARs can be opened within Chrome/Chromium. However, the fix sitespeedio/chrome-har#44 has now been integrated and released as v0.11.3.

Can you please update the dependency of puppeteer-har and release a new version, so I can fix EU-EDPS/website-evidence-collector#9 ?

Some response bodies are truncated when using saveResponse option.

This is due to the fact that you are not waiting for Network.loadingFinished

See:
ChromeDevTools/devtools-protocol#44 (comment)

I want to create har file of slow-loading website.
But created har file contents is empty.
I added sleep() function after har.start() , then har file contents is not empty.

Question

1. Why is sleep function needed?
2. I do not want to specify milliseconds for sleep function. What function do I use ?
   For example, should I use await page.waitForNavigation({ waitUntil: 'networkidle2' }); ?
   puppeteer/puppeteer#1916 (comment)

{"log":{"version":"1.2","creator":{"name":"chrome-har","version":"0.4.1","comment":"https://github.com/sitespeedio/chrome-har"},"pages":[],"entries":[]}}

const puppeteer = require('puppeteer');
const PuppeteerHar = require('puppeteer-har');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  const har = new PuppeteerHar(page);
  await har.start({ path: 'results.har' });
  
  // Added
  await sleep(2000);

  await page.goto('http://example.slowly.com');

  await har.stop();
  await browser.close();
})();


function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

Hey,

Are there any package that has types for this lib? would it be better to include them within this package or to contribute them to DefinitelyTyped package?

I am using har-puppeteer to retrieve network metrics from Chrome on page load.
I created a collection time that tracks the time.
har-puppeteer also collects its own collection time metric "startedDateTime": but it shows 1-2 hours behind the actual collection time.
Can someone help explain/fix this gap? And also provide documentation on the har-puppeteer metrics definitions.

Attached code here:
collectionwithWL - Copy.zip

Right now you have to start the record before you launch the browser, is it posibble to do it after? while the browser already open?