Git Product home page Git Product logo

mitmproxy-java's Introduction

Java rendition of node-mitmproxy

Make sure to read the prerequisites section below.

Detailed information included in the AppiumPro article which introduces this library.

Why?

mitmproxy is great for capturing network traffic, but has no easy interface for Java users. The software testing community, specifically Appium mobile testers, want to be able to capture network requests made by devices during their Java tests.

This library will run mitmproxy in the background as a separate process, and allow you to pass in a lambda function which gets called on every network request captured by mitmproxy. This allows you to check the captured requests within you Java code without having to stop the proxy or end capture.

See @jonahss's Appium Pro posts on this topic:

What can I use this for?

For transparently rewriting HTTP/HTTPS responses. The mitmproxy plugin lets every HTTP request go through to the server uninhibited, and then passes it to Java via a WebSocket for rewriting.

How does it work?

mitmproxy-java starts a Websocket server and a Python plugin for mitmproxy connects to it and sends requests over. The two communicate via binary messages to reduce marshaling-related overhead.

Pre-requisites

  • mitmproxy V9 must be installed and runnable from the terminal. The install method cannot be a prebuilt binary or homebrew, since those packages are missing the Python websockets module. Install via pip or from source.
  • Python 3.6 and above, since we use the new async/await syntax in the mitmproxy plugin
  • pip3 install websockets

Maven:

<dependency>
  <groupId>io.appium</groupId>
  <artifactId>mitmproxy-java</artifactId>
  <version>2.0.2</version>
</dependency>

Gradle:

testCompile group: 'io.appium', name: 'mitmproxy-java', version: '2.0.2'

Usage

List<InterceptedMessage> messages = new ArrayList<InterceptedMessage>();

//optional, default port is 8080
int mitmproxyPort = 8090;

//optional, you can pass null if no extra params
List<String> extraMitmproxyParams = Arrays.asList("param1", "value1", "param2", "value2");

// remember to set local OS proxy settings in the Network Preferences
proxy = new MitmproxyJava("/usr/local/bin/mitmdump", (InterceptedMessage m) -> {
    System.out.println("intercepted request for " + m.getRequest().getUrl());
    messages.add(m);
    return m;
}, mitmproxyPort, extraMitmproxyParams);

proxy.start();

// do stuff

proxy.stop();

If the above code doesn't work, and you are getting Error=2, No such file or directory please re-check your mitmdump path in new MitmproxyJava initialization. You can get mitmdump path using below command:

whereis mitmdump

See AppiumPro article for more guidelines: https://appiumpro.com/editions/65 Example can be found here: https://github.com/cloudgrey-io/appiumpro/blob/master/java/src/test/java/Edition065_Capture_Network_Requests.java

Your Java code is bad and you should feel bad

I'm no Java expert! You may see some bad patterns, like my terrible disregard for exception handling. Let's make it better. File your issues and land some PRs! I wrote this in just a couple days for our weekly newsletter.

Your Python plugin is bad and you should feel bad

See node-mitmproxy. Pull requests welcome!

Develop

Upload to Central Repository with command ./gradlew uploadArchives Set username and password in build.gradle file

mitmproxy-java's People

Contributors

czolg avatar himanshunikhare25 avatar jonahss avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mitmproxy-java's Issues

There will be a CROS problem

There is no problem running mitmproxy directly, but there will be a cros problem through mitmproxy-java

system: mac os 11
java:1.8

Support for Mitmproxy V5

Thanks for your wonderful work. But it has been a long time since last update and it can't be worked with latest mitmproxy v5.
Is there a plan for that?

can not install certificate

running your code
Path certificatePath = Paths.get(System.getProperty("user.home"), ".mitmproxy", "mitmproxy-ca-cert.pem"); Map<String, Object> args = new HashMap<>(); byte[] byteContent = Files.readAllBytes(certificatePath); args.put("content", Base64.getEncoder().encodeToString(byteContent)); androidDriver.executeScript("mobile: installCertificate", args);

and getting appium error to install cretificate click for log

Unable to reflect response in android app using appium and Java

I am using below configuration.

  1. Installed MITM certificate on my emulator.
  2. Installed MITM proxy, websocket and other related dependecy as mentioned in appiumPro article

Part of code dealing with proxy is as per below
`
List messages = new ArrayList<>();

    // remember to set local OS proxy settings in the Network Preferences
    proxy = new MitmproxyJava("C:\\Python311\\Scripts\\mitmdump.exe", (InterceptedMessage m) -> {
        if(

                m.getRequest().getUrl().toString().toLowerCase().startsWith(apiURL.toLowerCase()) &&
                m.getRequest().getMethod().equalsIgnoreCase("post")&&
                getJsonObject(new String(m.getRequest().getBody())).hasNonNull("operationName")&&
                getJsonObject(new String(m.getRequest().getBody())).get("operationName").asText().equalsIgnoreCase("getPharamcyTATInfo")
        )
        {

                JsonNode responseBodyJson = getJsonObject(new String(m.getResponse().getBody()));
                String modifiedResp = updateValuesInJson(responseBodyJson.toString(),
                        "data.getSkuInfo.tatInfo.tatResponse.inventoryExist",false,
                        "data.getSkuInfo.tatInfo.tatResponse.items[0].exist",false);
                m.getResponse().setBody(modifiedResp.getBytes(StandardCharsets.UTF_8));
                m.getResponse().getHeaders().add(new String[]{"My Test Resp Header","Header Value"});
                messages.add(m);
        }
        return m;
    });
    proxy.start();

    login(initializeDriver(testMethod),credentialManager);

`

now I have printed the intercepted message and modified response. but the app is still receiving the unmodified request.
when I do below
`
messages.forEach(
x->{
LOGGER.info("########Request Operation : "+x.toString() +getJsonObject(new String(x.getRequest().getBody())).get("operationName"));
LOGGER.info("########### Response Body :" +new String(x.getResponse().getBody()));
}

    );`

i see the proper response (modified one), this response never made it app, it is still showing the normal behaviour.

P.S. the app is Android native app
Please advice

Unable to set response body and apply it to UI, selenium driver

Hi, I am using selenium framework for web application, completed mitm setup, able to get calls and trying to edit the response body using setBody() but not getting the correct response message on the web application after changed the body(getting a common issue screen for all error JSON bodies). Able to edit the response code using setStatusCode(). Can anyone please help me to edit the response body so that it should show correct message on the UI, or if can we add setHeaders also to achieve this then how to use setHeaders()

我调试SimpleTest方法时messages总是0

SimpleTest方法中的messages没有数据。这是为什么呢?image
以下是日志:[main] INFO io.appium.mitmproxy.MitmproxyJava - Starting mitmproxy on port 8080
[WebSocketSelector-22] INFO org.java_websocket.server.WebSocketServer - websocket server started successfully
[main] INFO io.appium.mitmproxy.MitmproxyJava - Mitmproxy started on port 8080
么有信息!
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - Loading script C:\Users\12890\AppData\Local\Temp\mitmproxy-python-plugin78771757515683033.py
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - in script C:\Users\12890\AppData\Local\Temp\mitmproxy-python-plugin78771757515683033.py: No module named 'websockets'
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - Proxy server listening at http://*:8080

ModuleNotFoundError - websockets

Problem on Windows 11, python version 3.12

Encounter the following error:

[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava - [13:12:32.253] Loading script C:\Users\Horia\AppData\Local\Temp\mitmproxy-python-plugin1517772022857169551.py
[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava - [13:12:32.263] error in script C:\Users\Horia\AppData\Local\Temp\mitmproxy-python-plugin1517772022857169551.py
[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava - Traceback (most recent call last):
[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava -   File "C:\Users\Horia\AppData\Local\Temp\mitmproxy-python-plugin1517772022857169551.py", line 16, in <module>
[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava -     import websockets
[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava - ModuleNotFoundError: No module named 'websockets'
[Thread-18] INFO io.appium.mitmproxy.MitmproxyJava - [13:12:32.264] HTTP(S) proxy listening at *:8080.

I have seen this issue, and it seems I need to install mitmproxy via pip? I just don't understand how will that work, since pip install mitmproxy will download a python module, and I need the mitmdump.exe path when instatiating MitmproxyJava class.

I have also tried to install mitmproxy using:

  • the windows installer, same error as above
  • pipx, as hinted in here, same error as above
  • pip, but I cannot find the path to the mitmdump executable

What am I doing wrong?

No module named 'websockets'

[Win10]
Hello, I tried to run the library, it keeps telling me that:

22:45:33.231 [Thread-6] INFO io.appium.mitmproxy.MitmproxyJava - Loading script C:\Users\Adam\AppData\Local\Temp\mitmproxy-python-plugin5399613169945565656.py
22:45:33.231 [Thread-6] INFO io.appium.mitmproxy.MitmproxyJava - in script C:\Users\Adam\AppData\Local\Temp\mitmproxy-python-plugin5399613169945565656.py: No module named 'websockets'
22:45:33.232 [Thread-6] INFO io.appium.mitmproxy.MitmproxyJava - Proxy server listening at http://*:8080

I used pip3 to install websockets module but it looks like the script is somehow not using it, so at the end the handler function is not called. Do you have any idea what's going on?

Proxy with Selenium

Hello, I am making a common framework which integrates Appium and Selenium together.

I have mitmproxy installed (MAC) and am launching chrome browser.
I've included the proxy desired capabilities.
desiredCapabilities.addArguments("--proxy-server=localhost:8080");

@Test
public void validateChromeLaunch() {
    List<InterceptedMessage> messages = new ArrayList<InterceptedMessage>();

    MitmproxyJava proxy = new MitmproxyJava("/usr/local/bin/mitmdump", (InterceptedMessage m) -> {
        System.out.println("intercepted request for " + m.requestURL.toString());
        messages.add(m);
        return m;
    });
    try {
        proxy.start();
        WebDriver webDriver = DriverFactory.getDriver();
        webDriver.get("http://google.com");
        System.out.println(webDriver.getCurrentUrl());
        messages.forEach(interceptedMessage ->
                                 System.out.println(interceptedMessage.responseBody
                                 ));
        proxy.stop();
    } catch (IOException | TimeoutException | URISyntaxException | InterruptedException e) {
        e.printStackTrace();
    }
}

I can see the proxy exposing the network requests to google, but I cannot print out the intercepted message, it does not capture the network calls.

Have you successfully tested this with Selenium?
This project looks like a great replacement for browsermob.

Unable to capture request and response in iOS execution using appium and java

Problem Description
I am trying to run my java code from appium script in eclipse and using mitmproxy to capture the requests. Whenever I start the test execution, the eclipse console showing me all the request and response code but it is not capturing in InterceptedMessage.

System Information
The logs which I am receiving on eclipse console:
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - 127.0.0.1:52111: POST https://outlook.office.com/owa/service.svc?action=GetAt… HTTP/2.0
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - << HTTP/2.0 200 OK 1.37k
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - 127.0.0.1:52136: clientconnect
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - 127.0.0.1:52136: POST https://teams.events.data.microsoft.com/Collector/3.0/?…
[Thread-10] INFO io.appium.mitmproxy.MitmproxyJava - << 200 OK 0b

my mitmproxy version
Mitmproxy: 6.0.2
Python: 3.9.1
OpenSSL: OpenSSL 1.1.1j 16 Feb 2021
Platform: macOS-11.2-x86_64-i386-64bit

and the method which I am using to capture the request is:
private MitmproxyJava proxy;
public IOSDriver captureIosSimulatorTraffic(String appLocation)
throws IOException, InterruptedException, TimeoutException {
List messages = new ArrayList();
// remember to set local OS proxy settings in the Network Preferences
proxy = new MitmproxyJava("/usr/local/bin/mitmdump", (InterceptedMessage m) -> {
System.out.println("intercepted request for " + m.getRequest().getUrl().toString());
messages.add(m);
return m;
});
proxy.start();
}

Update Readme Version

Hello, I wanted to alert you to update the README version.
I had copied the maven dependency from the readme 1.6.1 (missing package support)

Maven, MitmProxy version 9: AttributeError: module 'mitmproxy.http' has no attribute 'HTTPResponse'

Hello,

I recently started a Maven project using mitmproxy-java. Anytime I try to change the status code of an intercepted call, I get this error: AttributeError: module 'mitmproxy.http' has no attribute 'HTTPResponse'

`List messages = new ArrayList<>();

    MitmproxyJava proxy = new MitmproxyJava("/Library/Frameworks/Python.framework/Versions/3.10/bin/mitmdump", (InterceptedMessage m) -> {
        messages.add(m);
        m.getResponse().setBody("Hi from Test".getBytes(StandardCharsets.UTF_8));
        m.getResponse().getHeaders().add(new String[]{"myTestResponseHeader", "myTestResponseHeaderValue"});
        m.getResponse().setStatusCode(500);
        return m;
    });
    proxy.start();

    Unirest.setProxy(new HttpHost("localhost", 8080));
    HttpResponse<String> response = Unirest.get("http://appium.io").asString();

    driver.get("http://appium.io");

    System.out.println("RESPONSE");
    System.out.println(messages.get(0));

    TimeUnit.SECONDS.sleep(15);

    proxy.stop();

    assertThat(response.getBody()).isEqualTo("Hi from Test");

    final InterceptedMessage firstMessage = messages.get(0);

    assertThat(firstMessage.getRequest().getUrl()).startsWith("http://appium.io/");
    assertThat(firstMessage.getRequest().getHeaders()).containsOnlyOnce(new String[]{"myTestHeader", "myTestValue"});
    assertThat(firstMessage.getResponse().getHeaders()).containsOnlyOnce(new String[]{"myTestResponseHeader", "myTestResponseHeaderValue"});
    System.out.println(firstMessage.getResponse().getStatusCode());
    assertThat(firstMessage.getResponse().getStatusCode()).isEqualTo(500);`

I see that the newest version is 2.0.2 and I'm hoping it has the fix. But currently only 2.0.1 is available on Maven to insert into the POM file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.