everettss / puppeteer-har Goto Github PK
View Code? Open in Web Editor NEWGenerate HAR file with puppeteer
License: MIT License
Generate HAR file with puppeteer
License: MIT License
Please refers to sitespeedio/chrome-har#15 for details on the issue
chrome-har
now supports returning response body as part of har. This option defaults to false in chrome-har
. Can you add support to return response body via puppeteer-har?
Hello,
I'm using [email protected] [email protected] and [email protected]
const puppeteer = require('puppeteer');
const PuppeteerHar = require('puppeteer-har');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
console.log("create page");
const har = new PuppeteerHar(page);
await har.start({ path: 'results.har' });
console.log('Har started');
await page.goto('https://********/portail');
await har.stop();
await browser.close();
})();
How can I stop having empty "pageTimings" please ?
{"log":{"version":"1.2","creator":{"name":"chrome-har","version":"0.11.7","comment":"https://github.com/sitespeedio/chrome-har"},"pages":[{"id":"page_1","startedDateTime":"2020-04-29T14:00:06.681Z","title":"https://********/portail","**pageTimings":{}**}],"entries":[{"cache":{},"startedDateTime":"2020-04-29T14:00:06.682Z","_requestId":"5380B4DD08F8636FEB1F2EC3A5914CE6","_initialPriority":"VeryHigh","_priority":"VeryHigh","pageref":"page_1","request":{"method":"GET","url":
What do you think about adding support for playwright?
Maybe just another option to start(). Like start({path:.., playwright: true})
Then:
if (playwright) {
this.client = await this.page.context().newCDPSession(this.page);
} else {
this.client = await this.page.target().createCDPSession();
}
Hi, pageTimings is empty in har file, it cause I can not get onLoad time and onContendLoad time. Could you please help to have a look? Thanks in advance.
here is my script
describe("use puppeteer to get har file ", () => {
it("use more launch properties demo", async () => {
if (generate.getConfigs().isWindows) {
configs = generate.getConfigs().windows;
} else {
configs = generate.getConfigs().macOrLinux;
}
browser = await puppeteer.launch({slowMo: '1000', headless: false, executablePath: configs.chromePath,args: [--window-size=1920,1080
]});
page = await browser.newPage();
har = new PuppeteerHar(page);
await page.setViewport({width: 1920, height: 1080});
await page.setDefaultTimeout(configs.timeout)
/*
* har1
* /
await har.start({path: path.join(path.resolve(), configs.bmcReportPath.scenario1+'1.har')});
await page.goto("https://www.biomedcentral.com/");
await page.waitFor(5000)
await har.stop();
/
* har2
* */
await har.start({path: path.join(path.resolve(), configs.bmcReportPath.scenario1+'2.har')});
//get DOM element whose innertext is Search and clcik
await page.evaluate(() => {
const elements = [...document.querySelectorAll('span')];
const targetElement = elements.find(e => e.innerText == 'Search');
if (targetElement) targetElement.click();
});
await page.$eval('#publisherSearch', el => el.value = 'carbon neutral');
await page.keyboard.press('Enter');
await page.waitFor(5000)
await har.stop();
/*
*
* */
await page.close();
await browser.close();
});
});
here is my hare file with puppeteer-har tools, pageLoadTime is empty.
{
"log": {
"version": "1.2",
"creator": {
"name": "chrome-har",
"version": "0.11.12",
"comment": "https://github.com/sitespeedio/chrome-har"
},
"pages": [
{
"id": "page_1",
"startedDateTime": "2021-12-15T12:00:23.038Z",
"title": "https://www.biomedcentral.com/",
"pageTimings": {}
}
],
If I download har file manually, pageTimes is not empty. here is har file manually.
{
"log": {
"version": "1.2",
"creator": {
"name": "WebInspector",
"version": "537.36"
},
"pages": [
{
"startedDateTime": "2021-12-14T07:02:58.380Z",
"id": "page_17",
"title": "https://www.biomedcentral.com/",
"pageTimings": {
"onContentLoad": 263.5190000291914,
"onLoad": 22470.13099998003
}
}
],
I'm using the code from the example to generate a HAR for a specific page. The generated HAR is ~40 KB. When I use browsermob-proxy in python for the same page the generated HAR is ~1.5 MB. At least 1 html file is missing from the generated HAR from puppeteer-har.
Why is this? Is there a maybe an option to avoid it?
Even if use a wait function like in #5, I'm getting the same result.
My netsniffPrototype.js :
await this.page.goto(url, {timeout: 60000});
let cssSelectorconsent = '#consent-notice-agree-button';
let cookieAcceptBtn = await this.page.waitForSelector(cssSelectorconsent, { timeout: 60000, visible: true });
await cookieAcceptBtn.click();
await this.page.setCacheEnabled(false);
await this.harStart();
await this.page.reload({waituntil: networkidle2, timeout: 60000});
await this.harStop();
Add console.log in chrome-har/lib/entryFromResponse.js
const entrySecs =
page.__wallTime + (timing.requestTime - page.__timestamp);
console.log('page.__wallTime = ' + page.__wallTime + ' timing.requestTime = ' + timing.requestTime + ' page.__timestamp = ' + page.__timestamp)
entry.startedDateTime = dayjs.unix(entrySecs).toISOString();
console.log('dayjs.unix(entrySecs).toISOString() = ' + dayjs.unix(entrySecs).toISOString())
Stdout results (Python call nodejs script using subprocess.run() ) :
2020-02-24:12:39:28,092 INFO [netsniffPrototype.js:296] <Thread(Thread-1, started daemon 140015018862336)> Stop Har events
page.__wallTime = 1582544363.28644
timing.requestTime = 6971.141358
page.__timestamp 6971.138063
dayjs.unix(entrySecs).toISOString() = 2020-02-24T11:39:23.289Z
page.__wallTime = undefined
timing.requestTime = 6971.149286
page.__timestamp = undefined
2020-02-24:12:39:28,101 ERROR [netsniffPrototype.js:303] <Thread(Thread-1, started daemon 140015018862336)> Process exitHarNotStopped event Har not stopped properly !
e.stack => RangeError: Invalid time value
at Date.toISOString ()
at h.d.toISOString (/home/pptruser/node_modules/dayjs/dayjs.min.js:1:6372)
at module.exports (/home/pptruser/node_modules/chrome-har/lib/entryFromResponse.js:161:53)
at harFromMessages (/home/pptruser/node_modules/chrome-har/index.js:310:15)
at PuppeteerHar.stop (/home/pptruser/node_modules/puppeteer-har/lib/PuppeteerHar.js:109:21)
at runMicrotasks ()
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Dear Michał,
thank you for providing puppeteer-har. The European Data Protection Supervisor uses this in its free software tool to help website developers and data protection professional to detect data protection compliance issues.
In the past, I have used a fork of your code to include a patched version from chrome-har, which ensures that HARs can be opened within Chrome/Chromium. However, the fix sitespeedio/chrome-har#44 has now been integrated and released as v0.11.3.
Can you please update the dependency of puppeteer-har and release a new version, so I can fix EU-EDPS/website-evidence-collector#9 ?
Some response bodies are truncated when using saveResponse
option.
This is due to the fact that you are not waiting for Network.loadingFinished
I want to create har file of slow-loading website.
But created har file contents is empty.
I added sleep()
function after har.start()
, then har file contents is not empty.
sleep
function needed?sleep
function. What function do I use ? await page.waitForNavigation({ waitUntil: 'networkidle2' });
?{"log":{"version":"1.2","creator":{"name":"chrome-har","version":"0.4.1","comment":"https://github.com/sitespeedio/chrome-har"},"pages":[],"entries":[]}}
const puppeteer = require('puppeteer');
const PuppeteerHar = require('puppeteer-har');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
const har = new PuppeteerHar(page);
await har.start({ path: 'results.har' });
// Added
await sleep(2000);
await page.goto('http://example.slowly.com');
await har.stop();
await browser.close();
})();
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
Hey,
Are there any package that has types for this lib? would it be better to include them within this package or to contribute them to DefinitelyTyped package?
I am using har-puppeteer to retrieve network metrics from Chrome on page load.
I created a collection time that tracks the time.
har-puppeteer also collects its own collection time metric "startedDateTime": but it shows 1-2 hours behind the actual collection time.
Can someone help explain/fix this gap? And also provide documentation on the har-puppeteer metrics definitions.
Attached code here:
collectionwithWL - Copy.zip
Right now you have to start the record before you launch the browser, is it posibble to do it after? while the browser already open?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.