jakubkulhan / chrome-devtools-protocol Goto Github PK

View Code? Open in Web Editor NEW

166.0 166.0 50.0 5.93 MB

Chrome Devtools Protocol client for PHP

License: MIT License

Makefile 0.02% PHP 99.94% Shell 0.02% Dockerfile 0.02%

automation chrome chrome-devtools chrome-devtools-protocol headless headless-chrome web

chrome-devtools-protocol's People

Contributors

Stargazers

Watchers

chrome-devtools-protocol's Issues

Make launcher executable configurable

I'm likely just not seeing the mechanism to do this, but I would like to specify the chromium-browser executable as I will not be installing google-chrome unless need-be.

Perhaps just switch out the const and use a setter method?

Intercepting/testing network requests

Hi,

quick question - does this library support setting up intercepting, or otherwise debugging, network requests?

Basically, I'm trying to write an integration test to check that our content security policy headers are correctly blocks some external requests, so I'd like to check that:

calls to unapproved Javascript files don't succeed.
after that request is blocked, there is an appropriate call to our content security policy reporting end-point.

cheers
Dan

Chrome version 111+ disallows POST method into CDT

See https://chromium.googlesource.com/chromium/src/+/1b944abea3ffeef729174b6a802d2d805a38ce2c%5E%21/#F1

Since this wrapper uses POST, it stops working, logging the error:

Fatal error: Uncaught GuzzleHttp\Exception\ClientException: Client error: POST http://127.0.0.1:28909/json/new resulted in a 405 Method Not Allowed response: Using unsafe HTTP verb POST to invoke /json/new. This action supports only PUT verb.

Context::background() does not work

This always fails:

$ctx = Context::background();
$instance->createSession($ctx);

Context::background() has deadline null - it sets stream_set_timeout() to 0. Seems that http://php.net/manual/en/function.stream-set-timeout.php does not clear timeout, when setting to zero, but instead times out socket immediately.

Support for PHP 8.1

The latest version of PHP is 8.1 and this package does not support it.

I forced it to install to see if it would work, but seems that at least all the classes that implement jsonSerializable need to be updated to add a return type.

An example of the error is:

PHP Fatal error: During inheritance of JsonSerializable: Uncaught ErrorException: Return type of ChromeDevtoolsProtocol\Model\Target\CreateBrowserContextRequest::jsonSerialize() should either be compatible with JsonSerializable::jsonSerialize(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /chrome-devtools-protocol/gen-src/ChromeDevtoolsProtocol/Model/Target/CreateBrowserContextRequest.php:52

There's also some deprecations like:

PHP Deprecated:  Return type of ChromeDevtoolsProtocol\Model\Network\Headers::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in chrome-devtools-protocol/gen-src/ChromeDevtoolsProtocol/Model/Network/Headers.php on line 75

Adding logEntry and message listener

HI there,

In the other issue, you suggested using a log and/or console listener to listen for CSP violation notices. I've tried setting them up, but they don't seem to be being triggers. Am I doing anything obviously wrong?

This is the test page, that is being served:

    public function getTestPage(RequestNonce $nonce)
    {
        $nonceRandom = $nonce->getRandom();

        $html = <<< HTML
<html>
<body>
  Hello, I am a test page, that tries to load some naughty javascript, which should trigger a CSP report.
</body>
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
  
  <script nonce="$nonceRandom">
    console.log("Test that logging is working.");
  </script>
</html>

HTML;

        return new HtmlResponse($html);
    }

This is the code I'm using to setup the log and console listeners, and then fetch the page.

          $messageListener = function() {
                    echo "messageListener \n";
                    var_dump(func_get_args());
                };

                $logEntryAddedListener = function () {
                    echo "logEntryAddedListener:\n";
                    var_dump(func_get_args());
                };

                $session->log()->addEntryAddedListener($logEntryAddedListener);
                $session->console()->addMessageAddedListener($messageListener);

                $session->page()->navigate(
                    $ctx,
                    NavigateRequest::builder()
                        ->setUrl(self::CSP_VIOLATION_PAGE)
                        ->build()
                );
          $session->page()->awaitLoadEventFired($ctx);

The event callbacks never seem to be called, though that page is definitely sending some output to the console:

Upgrade wrench/wrench dependency to ^2.0.10

I am aware of #25 and this is not a duplicate.

When using this library with PHP 7.4 developers will run into the following: implode(): Passing glue string after array is deprecated. Swap the parameters

I've done some research and found that this issue arises from wrench/wrench when using PHP 7.4 and is already fixed in wrench/wrench: varspool/Wrench#130

The issue got solved with the following release of wrench: https://github.com/varspool/Wrench/releases/tag/v2.0.10

This library should use at least 2.0.10, if not 3.x as suggested in #25, to make sure it stays usable with PHP 7.4

Temporary fix

To temporarly fix this issue require wrench/wrench before jakubkulhan/chrome-devtools-protocol in your composer.json like so: "wrench/wrench": "^2.0.11",`

No PDF created after printToPDF

I have your Basic Usage demo working with the addition of the necessary autoload and use classes, but nothing happens on printToPDF. No PDF is created. $devtools->page()->printToPDF($ctx, PrintToPDFRequest::make()) returns a PrintToPDFResponse Object with data. I've tried saving that data using file_put_contents but it's not a valid PDF. Am I missing something?

Invalid argument supplied for foreach()

The culprit is here:

chrome-devtools-protocol/src/ChromeDevtoolsProtocol/DevtoolsClient.php

Lines 109 to 111 in d296e85

 foreach ($this->getWsClient()->receive() as $payload) { 

 /** @var Payload $payload */ 

 $message = json_decode($payload->getPayload());

Wrench\Client::receive() will return null if the websocket is no longer connected (I'm using a context with a timeout), throwing the foreach warning. I'm not sure what the best solution is here. Happy to write a patch though.

Do you have a document

printToPDF triggered before ajax request completion

$ctx = Context::withTimeout(Context::background(), 30);
$chrome_binary_file = '/usr/bin/google-chrome';
$launcher = new Launcher();
$launcher->setExecutable($chrome_binary_file);
$instance = $launcher->launch($ctx, '--user-data-dir=/tmp/chrome-user-data', '--disable-gpu --virtual-time-budget=5000');

$tab = $instance->open($ctx);
$tab->activate($ctx);
$devtools = $tab->devtools();

$devtools->page()->enable($ctx);
$devtools->page()->navigate($ctx, NavigateRequest::builder()->setUrl($source_url)->setReferrer($ref_url)->build());
$devtools->page()->awaitLoadEventFired($ctx);

$data = $devtools->page()->printToPDF($ctx, PrintToPDFRequest::fromJson((object)[
'displayHeaderFooter' => true,
'paperWidth' => 8.27,
'paperHeight' => 11.69,
'headerTemplate' => '

',
'footerTemplate' => '

Page of

'
]))->data;

file_put_contents($filename, base64_decode($data))

New stable release

The current stable release on packagist.org is over two years old and lacks support for PHP8+. Would it be possible to tag a new release, or are there any blocking issues? I'd be happy to help where needed. Thanks.

how to start using?

what steps do you need to follow from this resource? https://chromedevtools.github.io/devtools-protocol/
how to install via composer?

implode() deprecated in PHP 7.4

I have updated my dev environment to PHP 7.4 and am now getting an exception when running Tab->devtools();

The error occurs in Protocol.php line 330 - implode().

Error message:

implode(): Passing glue string after array is deprecated. Swap the parameters

Compatibility warning composer 2.0

Composer 2 autoloading errors occurs due to wrench package PSR-0 incompliance:
see: varspool/Wrench#133

Package seems to be abandoned, solution can be to use a fork. https://github.com/chrome-php/wrench is mentioned there.

Guzzle update is potentially breaking the Launcher

the launcher, launchWithExecutable method looks for Guzzle ConnectException and catches in that infinite loop
currently, on my machines, Guzzle is throwing explicit RequestException so Launcher always fails immediately on DevTools startup

Please advise if this behavior is confirmed for other users/consumers.

Oddly, it looks like a connect exception message, but for some reason, its a RequestException

Intercept Requests and cancel some

Hi Im trying to intercept requests and cancel some depending on the domain im trying to use the fetch method with addRequestPausedListener but not having much luck getting it working do you have an example?

Wait until network requests finished loading

I'm generating pdf from html webpage with custom fonts from google fonts.

<!DOCTYPE html>
<html>
<head>
    <meta charset='utf-8'>
    <meta http-equiv='X-UA-Compatible' content='IE=edge'>
    <title>Test</title>
    <link href="https://fonts.googleapis.com/css?family=Bahianita&display=swap" rel="stylesheet">
    <style>
        h1 {
            color: blue;
            font-family: 'Bahianita', cursive;
        }
    </style>
</head>
<body>
    <h1>Hello world</h1>
</body>
</html>

But the resulting pdf doesn't show custom fonts, since they are not loaded from the network.

Is it possible to implement something like waitUntil from puppeteer, or is there any other way to wait until all network requests are finished loading.

Upgrade wrench/wrench to prevent composer deprecation notices

When running the latest version of composer there are deprecation notices for the wrench/wrench dependency

Deprecation Notice: Class Wrench\Tests\SslTest located in ./vendor/wrench/wrench/lib/Wrench/Tests/Util/SslTest.php does not comply with psr-0 autoloading standard. It will not autoload anymore in Composer v2.0.

Deprecation Notice: Class Wrench\Tests\Protocol\HybiPayloadTest located in ./vendor/wrench/wrench/lib/Wrench/Tests/Payload/HybiPayloadTest.php does not comply with psr-0 autoloading standard. It will not autoload anymore in Composer v2.0.

I see that they have a v3.0.0-rc1 version that looks like fixes the issue. I am not sure how soon v3.0.0 will be released so it might be useful to try using the release candidate. It looks like they have been working on v3 since 2017 without a final release so I think we may be waiting a long time.

$instance = $launcher->launch( $ctx) hang up with "--remote-debugging-port=9222"

Hi, thanks for the great package.

with "--remote-debugging-port=9222", it create command line:

exec '/usr/bin/google-chrome' '--headless' '--window-size=1280,1024' '--disable-gpu' '--ignore-certificate-errors' '--no-sandbox' 'user-agent=Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.39 Safari/537.36' '--remote-debugging-port=9222' '--user-data-dir=/tmp/chrome-profile-39017'

"--remote-debugging-port=9222" and "--user-data-dir=/tmp/chrome-profile-39017"，port number is incorrect.

did i missed something？

thanks for your help.

Possible to run multiple tasks with 'daemonized' chrome-headless and CDP?

I would like to know if it is possible to run multiple tasks (f.e. PDF generation) using remote-debugging and daemonized chrome-headless (one instance running always in background)?

For every task there could be created a new tab kept open until the task is finished. Is this possible or doesn't it make sense at all?

Fail on intialization

Fatal error: Uncaught ChromeDevtoolsProtocol\Exception\RuntimeException: Executable [chrome] not found. in C:\xampp\htdocs\vendor\jakubkulhan\chrome-devtools-protocol\src\ChromeDevtoolsProtocol\Instance\Launcher.php:139 Stack trace: #0 C:\xampp\htdocs\index.php(17): ChromeDevtoolsProtocol\Instance\Launcher->launch(Object(ChromeDevtoolsProtocol\Context)) #1 {main} thrown in C:\xampp\htdocs\vendor\jakubkulhan\chrome-devtools-protocol\src\ChromeDevtoolsProtocol\Instance\Launcher.php on line 139

Code:

`<?php

require "./vendor/autoload.php";

use ChromeDevtoolsProtocol\Context;
use ChromeDevtoolsProtocol\Instance\Launcher;
use ChromeDevtoolsProtocol\Model\Page\PrintToPDFRequest;
use ChromeDevtoolsProtocol\Model\Page\NavigateRequest;

@Unlink(DIR . '/test.pdf');

// context creates deadline for operations
$ctx = Context::withTimeout(Context::background(), 30 /* seconds */);

// launcher starts chrome process ($instance)
$launcher = new Launcher();
$instance = $launcher->launch($ctx);

try {
// work with new tab
$tab = $instance->open($ctx);
$tab->activate($ctx);

$devtools = $tab->devtools();
try {
	$devtools->page()->enable($ctx);
	$devtools->page()->navigate($ctx, NavigateRequest::builder()->setUrl("https://www.google.com/")->build());
	$devtools->page()->awaitLoadEventFired($ctx);
    $data = $devtools->page()->printToPDF($ctx, PrintToPDFRequest::fromJson((object) [
        'displayHeaderFooter' => false
    ]))->data;
    file_put_contents(__DIR__ . '/../test.pdf', base64_decode($data));
} finally {
	// devtools client needs to be closed
	$devtools->close();
}

} finally {
// process needs to be killed
$instance->close();
}`

Any idea what could be going on? Is there a place where I set the Chrome address? On my system it is installed at the default address.

Support Guzzle 7

I would be nice if this package could also work with Guzzle 7. ^6.3|^7.0

Guzzle 7 Upgrade Guide

Socket exception makes it impossible to cleanly close session

If there's an issue that closes the WebSocket connection, it's not longer possible to cleanly close the session.

If you call Session::close() on the session object after an issue that closes the WebSocket connection (likely after catching an exception resulting from that closed WebSocket connection), Session::close() will attempt to send messages to the disconnected WebSocket. That will cause an exception.

If you catch that exception, execution in Session::close() will never get to the line that closes the DevtoolsClient: "$this->browser->close();"

If DevtoolsClient::close() is never called, the LogicException in DevtoolsClient::__destruct() will be thrown, complaining that the (broken) WebSocket connection was never released.

Put another way:

Instance::open() creates a Tab. You can create and get the DevtoolsClient using $tab->devtools(). You can call $devtools->close() directly.
On the other hand, Instance::createSession() creates a Session. The DevtoolsClient is created implicitly, and you cannot get it. You can only close in indirectly by closing the Session, and if the Session closing process throws an exception, code execution will never reach the line in Session::close() that calls DevtoolsClient::close(). This results in DevtoolsClient::__destruct() throwing the exception seen inside.

Conflict between await and execute command

First of all - thanks for the wonderful package!

Second, i think there is a flaw in how the messages are handled if you use callbacks and call another method from within a callback.. The bug is tricky, because it depends on the ordering of the messages between client and server, so it takes a few runs to catch it.

Relevant parts:

$ctx = Context::withTimeout(Context::background(), 30 /* seconds */);

Launcher::$defaultArgs = []; // run not headless -- bug happens often this way

...

$devtools->network()->enable(...);

$devtools->network()->addLoadingFinishedListener(function ($e) ... {
    $request = GetResponseBodyRequest::builder()->setRequestId($e->requestId)->build();
    $devtools->network()->getResponseBody($ctx, $request);
});

$pageDomain = $devtools->page();
$pageDomain->enable($ctx);
$url = 'https://www.google.com';
$pageDomain->navigate($ctx, NavigateRequest::builder()->setUrl($url)->build());
$pageDomain->awaitLoadEventFired($ctx);

What I expect this program to do:
to finish without timeout

What happens:
in some percentage of cases (50%?) it ends with a timeout on awaitLoadEventFired

The full source is here https://gist.github.com/slava-vishnyakov/057d2dfb0b9b1bad878130c9607e3179

Ok, so now we have something like this:

awaitLoadEventFired loops handleMessage waiting for Page.loadEventFired
at the same time we have a stream of incoming Network.loadingFinished messages, which trigger a callback, which then calls for getResponseBody method
(which starts another loop) and
occasionally Page.loadEventFired happens to be in this getReponseBody loop, not in awaitLoadEventFired and therefore awaitLoadEventFired never sees this Page.loadEventFired event.

I've added a few debug prints to see this:

So as I understand it, it goes like this:

awaitLoadEventFired (waiting for event YYY)
  -> awaitLoadEventFired calls handleMessage
      <- event which has a callback
    -> handleMessage sees there is a callback, calls callback
      -> callback calls executeCommand
        -> executeCommand calls handleMessage (waiting for command result)
        -> executeCommand calls handleMessage (waiting for command result)
           <- event YYY  (handleMessage is not interested, since it waits for command result)
           <- command result
        -> returns result
      -> callback ends
  we are back at awaitLoadEventFired loop, maybe there are more events, maybe not
  -> calls handleMessage

So, the fix should be something like "if we are awaiting for something and deep handleMessage gets it - we need to "bubble" it somehow"..

The problem gets deeper, since a callback might call executeCommand, which will trigger a callback, which will trigger another callback, which might call another executeCommand..

-> await XX
  -> callback
    -> executeCommand
      -> (in callback, calls await..)
         -> await XX
           -> callback
             -> executeCommand
                <-- XX happens
                <-- other XX happens

So, basically we can't have something like a $this->await[$method] = {null | XX_Response}, we need something more clever, which will say that we have 2 awaits upstream waiting for XX

So, my first stab at the fix (quite a bit ugly, but I don't know how to express it better):

(noticed that in this version I have $this->awaits -- it should be $this->awaitMethods, this is corrected in code below)

Basically we have two new instance variables - the number of awaits upstream waiting for particular method ($awaitMethods (naming is hard... $methodAwaitersCount ? :) )) and responses for those ($awaitMessages).

Now, when we enter handleMessage - we look whether there is an await upstream for this method. If there is - we don't process it and store to $awaitMessages variable..

After we return from callback - we look if there is a message for this method stored for us. If there is - we decrement the number of "awaiters" and return the message..

The new handleMessage:

private $awaitMethods = [];
private $awaitMessages = [];

private function handleMessage($message, ?string $returnIfEventMethod = null)
{
	if (isset($message->error)) {
		throw new ErrorException($message->error->message, $message->error->code);

	} else if (isset($message->method)) {
		if (isset($this->awaitMethods[$message->method]) && $this->awaitMethods[$message->method] > 0) {
			$this->awaitMessages[$message->method] [] = $message;

			return null;
		}

		if (isset($this->listeners[$message->method])) {
			if ($returnIfEventMethod !== null) {
				// add this method to await notifications (increment)
				if (!isset($this->awaitMethods[$returnIfEventMethod])) {
					$this->awaitMethods[$returnIfEventMethod] = 1;
				} else {
					$this->awaitMethods[$returnIfEventMethod]++;
				}
			}

			foreach ($this->listeners[$message->method] as $callback) {
				$callback($message->params);
			}

			if ($returnIfEventMethod !== null && count($this->awaitMessages[$returnIfEventMethod]) > 0) {
				// handleMessage inside callback got the message, take the first message from responses
				$message = array_shift($this->awaitMessages[$returnIfEventMethod]);
				// we are not waiting for this message anymore
				$this->awaitMethods[$returnIfEventMethod]--;

				return $message;
			}
		}

		if ($returnIfEventMethod !== null && $message->method === $returnIfEventMethod) {
			return $message;
		} else {
			if (!isset($this->eventBuffers[$message->method])) {
				$this->eventBuffers[$message->method] = [];
			}
			array_push($this->eventBuffers[$message->method], $message);
		}

	} else if (isset($message->id)) {
		$this->commandResults[$message->id] = $message->result ?? new \stdClass();

	} else {
		throw new RuntimeException(sprintf(
			"Unhandled message: %s",
			json_encode($message, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE)
		));
	}

	return null;
}

Allow use in PHP 7.2

Hi,

Does the library work okay in PHP 7.2? If so, would it be possible to change the PHP requirements to be either

"php": "~7.1 | ~7.2",

"php": "^7.1",

Or whatever is best for allowing installing on 7.2 ?

	foreach ($this->getWsClient()->receive() as $payload) {
	/** @var Payload $payload */
	$message = json_decode($payload->getPayload());

jakubkulhan / chrome-devtools-protocol Goto Github PK

chrome-devtools-protocol's People

Contributors

Stargazers

Watchers

Forkers

chrome-devtools-protocol's Issues

Recommend Projects

Recommend Topics

Recommend Org