Proxy unable to handle multiple consecutive requests about ios-webkit-debug-proxy HOT 23 CLOSED

I can reproduce the issue with the example wdp_client.js that comes with ios-webkit-debug-proxy. The first run will pass just fine. Try to run the second time and the navigate to url request reach the proxy and the browser navigates ok but the proxy does not respond to script at all so the script just hangs:

$ node wdp_client.js
open ws://localhost:9222/devtools/page/1
connected
send {"id": 1, "method": "Page.navigate", "params":{"url": "http://www.google.com"}}
recv {"result":{},"id":1}
disconnected
$ node wdp_client.js
open ws://localhost:9222/devtools/page/1
connected
send {"id": 1, "method": "Page.navigate", "params":{"url": "http://www.google.com"}}

from ios-webkit-debug-proxy.

We seem to be coming across the same issue where requests will fail intermittently and the socket gets disconnected.Any updates on this?

from ios-webkit-debug-proxy.

Thanks, yes, l just reproduced this issue. The proxy is likely missing
some socket-close cleanup. I plan to debug this today / early next week.

Todd
On May 17, 2013 5:45 AM, "swapna-chennadi" [email protected] wrote:

We seem to be coming across the same issue where requests will fail
intermittently and the socket gets disconnected.Any updates on this?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/15#issuecomment-18052596
.

from ios-webkit-debug-proxy.

Todd, any luck with this?

from ios-webkit-debug-proxy.

Hi Sergio, I just pushed a fix. Please try it out!

I'll tag & update brew once I finish an unrelated fix to support local DevTools "inspector.html" files.

from ios-webkit-debug-proxy.

Todd,

It seems like something's gone wrong. It works for multiple "navigate" requests but breaks for any of the requests that you need to break into the 8k chunks (e.g. the find_element example I added to the bug).

In addition, if you now try and do anything (e.g "document.title") through the console tab in the browser (e.g. "http://localhost:27753/devtools/devtools.html?host=localhost:27753&page=1") you get a message "Already sent close_frame".

Let me know if you're getting the same or if I am doing something wrong.

Thanks,

Sergio

from ios-webkit-debug-proxy.

Todd,

1. Connection issue - (OK)
   I did some extra investigating. The socket connection closing issue seems to have been resolved. Every time I run multiple navigate requests or document.title requests it works fine, whereas before it would work the first time but not the second time (due to the connection issue).
2. 8k requests - (NOK)
   However there is still an issue with larger than 8k requests as these are failing around 40% of the time at which time the proxy disconnects. (I keep seeing {"method":"Profiler.resetProfiles"} as the last received message in the proxy output, is this in anyway related?)
3. Dev Tools console requests - (NOK)
   As mentioned previously any request through the console on a chrome browser are not working at all.

Regards,

Sergio

from ios-webkit-debug-proxy.

Btw, if you would like to reproduce this use the example wdp_client.js and fire of the runtime.evaluate request below a couple of times:

'{"id": 1, "method": "Runtime.evaluate", "params":{"expression": "(function(name){return name})("large number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeatedlarge number of characters repeated")"}}'

Apologies for the large message.

Sergio

from ios-webkit-debug-proxy.

Thanks, debugging now.

Todd

from ios-webkit-debug-proxy.

Okay, I just pushed a (I hope) fix. I tested:

1. Connection issue -- verified via repeat NodeJS client navigate requests
2. 8k requests -- verified via a modified NodeJS client that passes a
   10k url parameter
3. Dev Tools console requests - verified in browser

I also tested and improved the handling of multiple conflicting inspectors
on the same page. E.g. if you leave a Dev Tools console open and then
request a NodeJS client navigate of that page, it now detects this, prints
a warning, and handles it cleanly.

Please test, and thanks for reporting these bugs!

from ios-webkit-debug-proxy.

Todd,

I tested the following scenarios:
1). Multiple short (less than 8k) requests (see example) work excellent.
I can run it multiple times in a row and it works great.

2). Small requests through the console work great. I put through tons of requests and it handles them all.

3). Multiple long (more than 8k) requests (see example and uncomment second set of requests) keep failing.

This is what the logging is returning without -d
Taking page 27753/1 from local DB091E3B-845B-41F4-9016-89CAFCD49F0E to 7FCBCE2A-7744-4C25-8D4B-06D0A24B45C2
Unable to send 7677 bytes to inspector

When I put the same large request through the console it fails as well. Something still doesn't seem right with the large 8k or more requests.

If you want to test with a real "Runtime.evaluate" example which we're looking to use its:
(< place the full content of <a =href="https://github.com/appium/appium/blob/master/app/hybrid/ios/webdriver-atoms/find_element.js">this file here >)("id", "< id of the element you're looking for >")
This code is the selenium web driver find element request.

Thanks for your help.

Sergio

from ios-webkit-debug-proxy.

Apologies, just saw that I had left an empty link for the find_element.js example:
https://raw.github.com/appium/appium/master/app/hybrid/ios/webdriver-atoms/find_element.js

You can test the request through the console by typing in:
( ...the content of the find_element.js...) ("id", " ... html id of the element you're looking for ... ")

This should return an wdc id for the found element of null if not found:
"{"status":0,"value":{"ELEMENT":":wdc:1369678003768"}}" OR {"status":0,"value":null}

Thanks,

Sergio

from ios-webkit-debug-proxy.

Hi Sergio,

Yes, this looks like a bug in the send code. I think it's unrelated to the
multiple consecutive requests issue.

The send is non-blocking and returns an "EWOULDBLOCK" error if you try to
send too much data. I'll fix the code to retry these failed requests.

Todd

from ios-webkit-debug-proxy.

Okay, I just pushed a fix.

I tested by modifying wdp_client.js:
var i;
var s = '';
for (i = 0; s.length < 50000; i++) {
  s += '+' + i;
}
var commands = []
for (i = 0; i < 100; i++) {
  commands.push('{"id": '+i+', "method": "Runtime.evaluate", "params":
{"expression": "(' + i + s + ')", "objectGroup": "console",
"returnByValue": true }}');
}
Please retest.

from ios-webkit-debug-proxy.

Here's the code again -- not sure why GitHub will let me update the above comment but not use Markdown :(

var i;
var s = '';
for (i = 0; s.length < 50000; i++) {
  s += '+' + i;
}
var commands = []
for (i = 0; i < 100; i++) {
  commands.push('{"id": '+i+', "method": "Runtime.evaluate", "params": {"expression": "(' + i + s + ')", "objectGroup": "console", "returnByValue": true }}');
}

from ios-webkit-debug-proxy.

Todd,

It looks like the fix has done the job. I ran most of the tests and the requests are flying through without any failures.

Will run some additional checks tomorrow to confirm, but i think you've cracked it.

Thank you very much! 👍

Sergio

from ios-webkit-debug-proxy.

Did some more tests and it looks all good.

Thanks again.

Sergio

from ios-webkit-debug-proxy.

I was testing the fix as well and seems to work ok.

However, I noticed the proxy is taking all my CPU while running - even in idle (no test ongoing). Is this intentional or is there some kind of busy polling loop left over in some thread for debugging purposes etc.

Anyone else having the same issue with latest master or is it just my env?

from ios-webkit-debug-proxy.

I did spot this once (but i didn't put the two together) as it wasn't affecting the performance of my mac or the proxy.

Just checked it again and it is taking 100% of one of the cores. However it is still very much responsive to incoming requests.

Do you think this is an issue Todd?

Sergio

from ios-webkit-debug-proxy.

Yes, it shouldn't be using lots of CPU when idle! It's likely a trivial
bug in the socket "select" loop. I'll fix it today.

Todd
On May 31, 2013 4:35 AM, "Sergio Neves Barros" [email protected]
wrote:

I did spot this once (but i didn't put the two together) as it wasn't
affecting the performance of my mac or the proxy.

Just checked it again and it is taking 100% of one of the cores. However
it is still very much responsive to incoming requests.

Do you think this is an issue Todd?

Sergio

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/15#issuecomment-18732755
.

from ios-webkit-debug-proxy.

All fixed, please test.

from ios-webkit-debug-proxy.

@wrightt , looks all good from my side. It's barely using any of the processor resources (less than 1%) for me and I am still able to make all the necessary calls and use the console. All good.

Nice spot @petrikii.

Sergio

from ios-webkit-debug-proxy.

Hallo @snevesbarros @zwri, i have a similiar issue. I'm using appium and ios-webkit-debug-proxy to test iOS 10.3.1 real device. i wrote a simple script with python. But sometimes it works sometimes it doesnt, if a run the same script for a couple times. If the script doesnt work. the error says webkit remote debugger socket disconnected.

I also already tried to install IWDP via Terminal, but it doesnt change anything

git clone https://github.com/google/ios-webkit-debug-proxy.git
cd ios-webkit-debug-proxy
./autogen.sh
./configure
make
sudo make install

here is the log file:
https://gist.github.com/mare1012/5ac509d11442583fb7d1fc8e4264bcee
here is my script:
https://gist.github.com/mare1012/1f67e5c10507932d9823651405a56aa8

from ios-webkit-debug-proxy.