NoDriver: how to get response body about undetected-chromedriver HOT 7 OPEN

await page.get_content()

Here is how to get the title of page:

....
from selectolax.parser import HTMLParser
....

html = await page.get_content()
tree = HTMLParser(html)
title = tree.head.css("title")[0].html

from undetected-chromedriver.

@Hillcow Bruh, seems like you have not idea what I am talking about

from undetected-chromedriver.

Well, i know what you are talking about.

This is the wrong repositor for nodriver. It is not even released yet.

I didn't find docs or examples.

Apparently /example/network_monitor.py (https://github.com/ultrafunkamsterdam/nodriver/blob/main/example/network_monitor.py) is not clear enough.

from nodriver import *
async def main():
    browser = await start()
    tab = await browser.get('data:,')   # bogus page to get a tab object. could also use browser.main_tab
    tab.add_handler(cdp.network.ResponseReceived, myhandler)
    await tab.get('https://www.google.com')
    
async def myhandler(event: cdp.network.ResponseReceived):
    print(event.response)

# could also use regular callback like
#
#  def myhandler(event:  cdp.network.ResponseReceived):
#        print(event.response)


loop().run_until_complete(main())

from undetected-chromedriver.

Well, i know what you are talking about.

This is the wrong repositor for nodriver. It is not even released yet.

I didn't find docs or examples.

Apparently /example/network_monitor.py (https://github.com/ultrafunkamsterdam/nodriver/blob/main/example/network_monitor.py) is not clear enough.

from nodriver import *
async def main():
    browser = await start()
    tab = await browser.get('data:,')   # bogus page to get a tab object. could also use browser.main_tab
    tab.add_handler(cdp.network.ResponseReceived, myhandler)
    await tab.get('https://www.google.com')
    
async def myhandler(event: cdp.network.ResponseReceived):
    print(event.response)

# could also use regular callback like
#
#  def myhandler(event:  cdp.network.ResponseReceived):
#        print(event.response)


loop().run_until_complete(main())

Can you help me move to a certain frame and click on an element?

from undetected-chromedriver.

Just search for the element. It searches in frames too

from undetected-chromedriver.

I may be just dense today but I have tried a number of ways to use get_response_body() both on cdp.network and cdp.fetch to get the actual body of the response without luck. sure, it is no problem to get the entire html of the page. but suppose some data comes through in a XHR that is much easier to parse in it's native json than rendered as html. is this possible? the returned objects are of type 'generator'

this is almost entirely from the network_monitor.py example:

from nodriver import start, cdp, loop

async def main():
    browser = await start()
    tab = browser.main_tab
    # tab.add_handler(cdp.network.RequestWillBeSent, send_handler)

    tab.add_handler(cdp.network.ResponseReceived, receive_handler)
    tab = await browser.get("https://example.com")
    tab.wait()
    await tab.wait(20)

async def receive_handler(event: cdp.network.ResponseReceived):
    print("---------------------------------------")
    print(event.response)

    response_body = cdp.fetch.get_response_body(event.request_id)

    print(type(response_body))
    # <class 'generator'>

regarding the nodriver repo: An owner of this repository has limited the ability to open an issue to users that are collaborators on this repository.

from undetected-chromedriver.

is this possible? the returned objects are of type 'generator'

you should pass this generator to send function (see pyCDP docs about commands and many examples in nodriver source code)

cdp_cmd = cdp.network.get_response_body(event.request_id)
res = await browser.main_tab.send(cdp_cmd)

but in this case it won't help ))) (see my issue #1832). Please let me know if you find a way to fix it

from undetected-chromedriver.