Proxy with Selenium about mitmproxy-java HOT 21 CLOSED

    proxy = new MitmproxyJava("/usr/local/bin/mitmdump", (InterceptedMessage m) -> {
        System.out.println("intercepted request for " + m.requestURL.toString());
        messages.add(m);
        return m;
    });

this starts mitmproxy itself. you dont need to start mitmproxy in a different terminal. if you start a terminal run before the code, you can not get a result. ı faced this simple mistake, maybe yours is like this.

from mitmproxy-java.

@seshusangeetha ah, that's the problem then. The readme states:

The install method cannot be a prebuilt binary or homebrew, since those packages are missing the Python websockets module. Install via pip or from source.

from mitmproxy-java.

I have the same issue with appium, can not get network calls, would be great for some help !

from mitmproxy-java.

Hi @williamrussellajb, what do you mean when you say:
I can see the proxy exposing the network requests to google

If you run mitmproxy from the commandline before you run your test, and you run your test without the mitmproxy-java code, does mitmproxy log the requests in the UI in the terminal? That would be how to check that the requests from chromedriver are actually going through the proxy.

from mitmproxy-java.

Hi Jonahss,

Yes, I can see the network calls in mitmproxy from my terminal, manually started.
Likewise, I can see them in my intellij command line when using proxy.start().

I never have any break points hit this line of code here though:

 MitmproxyJava proxy = new MitmproxyJava("/usr/local/bin/mitmdump", (InterceptedMessage m) -> {
        System.out.println("intercepted request for " + m.requestURL.toString());
        messages.add(m);
        return m;
    });

Would you like to make a basic project/example?

from mitmproxy-java.

Hi @williamrussellajb, ok I was just checking about the proxy settings.
I have an example project here: https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition065_Capture_Network_Requests.java

Your use looks like everything is right. Could you include the logs you saw? Could help diagnose the problem.

from mitmproxy-java.

@Jonahss , I am having the same issue as @williamrussellajb in which the terminal shows the network requests being captured but the lambda function is never called so I can't store the messages that are being captured

from mitmproxy-java.

Hello @spinalk, @williamrussellajb, I spent some time recently with the library and my test case is very simillar to yours, this works for me:

org.openqa.selenium.Proxy proxyProxy = new Proxy();
proxyProxy.setHttpProxy("localhost:8080");
proxyProxy.setSslProxy("localhost:8080");

org.openqa.selenium.remote.DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("proxy", proxyProxy);

ChromeOptions options = new ChromeOptions();
options.setProxy(proxyProxy);

capabilities.setCapability(ChromeOptions.CAPABILITY, options);
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);

WebDriver driver = new ChromeDriver(options);

from mitmproxy-java.

Also @adamMo has made some major improvements recently, so try the newest version with the example above ^

from mitmproxy-java.

Hello @spinalk @williamrussellajb I think I finally know what is your issue! You probably doesn't have python websockets module installed. I am currently on fresh OS and I faced:

2019-08-01 23:08:34.249 INFO 30360 --- [ main] i.a.m.MitmproxyJava : Mitmproxy started on port 8890
2019-08-01 23:08:34.255 INFO 30360 --- [ Thread-13] i.a.m.MitmproxyJava : Loading script /tmp/mitmproxy-python-plugin8155624357208443309.py
2019-08-01 23:08:34.255 INFO 30360 --- [ Thread-13] i.a.m.MitmproxyJava : in script /tmp/mitmproxy-python-plugin8155624357208443309.py: No module named 'websockets'

you can fix it by calling pip3 install websockets
Later on I will update the library to check and install it automatically.

from mitmproxy-java.

@Jonahss I do have the same issue mentioned by the above users with Appium and Eclipse IDE is used. I am able to see the traffic in my console. But none of the messages get intercepted or added. the lambda function is never called so I can't store the messages that are being captured.
I have my web sockets installed too if we assume that as @adamMo mentioned the web sockets installation as the issue so I guess that could not be the one causing this issue.

Any Quick help or response would be highly helpful. Thanks

from mitmproxy-java.

So if packets are being sent through mitmproxy but they are not being forwarded to your Java lambda functions, there must be an error in the python plugin which takes the mitm-intercepted packets and forwards them via websocket to the Java library.

We can debug this by running mitmproxy with the Python plugin and see if any errors are logged.
Try running mitmdump --anticache -s scripts/proxy.py and then trigger some packets to be sent from the device. Hopefully something will be logged which highlights the issue.

scripts/proxy.py is located inside the mitmproxy-java jar file for mitmproxy-java.

from mitmproxy-java.

@Jonahss Yes I guess there is some issue with web sockets. When I run as mentioned with proxy.py, it says that in "scripts/proxy.py: No module named 'websockets' " and then the traffic flow continues. May be this is causing the issue.

I tried with the script in both 1.6.1 and 2.0.1 jars but the same issue exists

Something weird is like when I try pip3 install websockets - it says
Requirement already satisfied: websockets in /usr/local/lib/python3.7/site-packages (8.0.2)

The other thing is when I try initialising the proxy it gives the following output where it says web socket server started successfully while it says no web sockets module in the script in next line

[main] INFO io.appium.mitmproxy.MitmproxyJava - Starting mitmproxy on port 8080
[WebSocketSelector-14] INFO org.java_websocket.server.WebSocketServer - websocket server started successfully
[Thread-6] INFO io.appium.mitmproxy.MitmproxyJava - Loading script /private//mitmproxy-python-plugin5637973557438151506.py
[Thread-6] INFO io.appium.mitmproxy.MitmproxyJava - in script /private//mitmproxy-python-plugin5637973557438151506.py: No module named 'websockets'
[Thread-6] INFO io.appium.mitmproxy.MitmproxyJava - Proxy server listening at http://*:8080
[main] INFO io.appium.mitmproxy.MitmproxyJava - Mitmproxy started on port 8080

from mitmproxy-java.

And how did you install mitm proxy on your machine?

from mitmproxy-java.

I installed it through brew

from mitmproxy-java.

@seshusangeetha @spinalk Do You have some news about this problem?

from mitmproxy-java.

@seshusangeetha Any update on this issue?

from mitmproxy-java.

Closing due to inactivity. I think we did solve the original issue.

from mitmproxy-java.

I had the same issue. I was not not able to intercept network calls using java client but was able to get netowrk call when I run mitmproxy manually from terminal. I had installed mitmproxy using homebrew that why this problem was occuring > i installed itmproxy suing pip and its working fine now.

from mitmproxy-java.

You may get the wrong mitmproxypath.
because in my os, there are two mitmdump files to exec.

from mitmproxy-java.

Thank you so much, brother! It works, I just closed the terminal which runs mitmweb and it works !

from mitmproxy-java.