Working for only first link, not for the links that are opened in new tabs (via website). about puppeteer-extra HOT 11 OPEN

This is a known limitation of puppeteer puppeteer/puppeteer#3667

from puppeteer-extra.

FYI, this is on the list for Playwright. I'm eager to see how they manage to resolve it.

I'm from the Playwright team. Playwright supports emulation properties at a browser context level, which enables all pages within the context to work with the said properties. These pages can be opened through browserContext.newPage() or as a result of page navigations.

from puppeteer-extra.

Hi @arjunattam, reading your comment I didn't fully understand if this works out of the box on playwright or what would the proper set up be to have the plugins work on every tab.

On this minimal example, I'm having trouble getting the anonymize-ua plugin to work on a page navigation, using playwright.

const { chromium } = require('playwright-extra')
const plugin = require('puppeteer-extra-plugin-anonymize-ua')(
    { customFn: (ua) => 'MyCoolAgent/' + ua.replace('Chrome', 'Beer') }
)

chromium.use(plugin)
chromium.launch({ headless: false }).then(async browser => {
    const page = await browser.newPage()
    await page.goto('https://www.whatismybrowser.com/detect/what-is-my-user-agent/', { waitUntil: 'domcontentloaded' })
    await page.click('[href="/detect/what-is-my-user-agent/"]', { button: 'middle' })
    // wait 10 seconds before closing the browser
    // manually switch to the new tab and check that the ua is different between both
    await new Promise((resolve) => setTimeout(resolve, 10000));
    await browser.close()
})

Do you have any idea on what's missing?

from puppeteer-extra.

Hey @millionwords could you provide a code example of the issues you have?

The following simple test case with multiple pages works as expected:

const puppeteer = require("puppeteer-extra")

const UserAgentPlugin = require("puppeteer-extra-plugin-anonymize-ua")
const uaPlugin = UserAgentPlugin({
  customFn: ua => ua.replace("Chrome", "Beer")
})

puppeteer.use(uaPlugin)

const checkPageForBeer = name => async page => {
  await page.goto("https://httpbin.org/headers", {
    waitUntil: "domcontentloaded"
  })
  const content = await page.content()
  console.log(`- ${name}: User agent includes beer:`, content.includes("Beer"))
}

puppeteer.launch({ headless: true }).then(async browser => {
  const page1 = await browser.newPage()
  await checkPageForBeer("page1")(page1)

  const page2 = await browser.newPage()
  await checkPageForBeer("page2")(page2)

  await browser.close()
})

Output:

❯❯❯ node newtab.js
- page1: User agent includes beer: true
- page2: User agent includes beer: true

Are you by chance dealing with popups? I recall that there where several issues surrounding popup handling in pptr, but it's been a while. 😄

from puppeteer-extra.

yes, It's a popup in a new tab, the headers do not work well :(

from puppeteer-extra.

I have the same issue. The first page is okay with all the settings but after clicking a link and opening to a new window. it is not configured as the first page.

from puppeteer-extra.

Back when I did the bulk of the code, popups where a known issue with puppeteer. A lot has happened since then and I'm gonna check if we aren't able to find a workaround here. 😄👍

from puppeteer-extra.

I also have the same question.
like this

` const page = await browser.newPage();

await page.goto('https://example.com/');
await page.evaluate(() => {
    var a = document.createElement('a');
    a.href = 'https://example.com/?404';
    a.target = '_blank';
    a.innerHTML = 'Click me';
    a.id = 'click';
    document.body.appendChild(a);
});

await page.waitFor(1000);

await page.click('#click');`

the new page is opend by a[target='_blank']，then puppeteer-extra-plugin-stealth does not work.
so is the puppeteer's bug? has other method to hack this problem?

from puppeteer-extra.

I'm working on a fix for this using new CDP hooks (puppeteer/puppeteer#3667 (comment)).

Right now I got a PoC working which will intercept implicitly created targets but I currently need to reload the popup/page to make it continue loading (which is less than optimal). I'll continue researching this, also happy to discuss potential ways to fix that.

But all in all it should be possible to fix this behaviour in the not too distant future :)

from puppeteer-extra.

Hey @berstend !
You are doing a great job here
It would be great if you can share any progress connected with this issue

from puppeteer-extra.

FYI, this is on the list for Playwright. I'm eager to see how they manage to resolve it.

from puppeteer-extra.