tafia / hyper-proxy Goto Github PK

I was able to do this with reqwest ( though I wasn't able to tunnel) - I could use it for HTTP endpoints only using standard http not connect. Anyway, Is there something like verify in python requests or another way to pass the cert file using your hyper-proxy. Most of my proxies are built with reqwest but my forward/firewall-ish one uses auth and requires a cert.

Thanks for your help!

Would you consider making your ProxyStream implementation public and also switching form using tokio_tls to tokio_native_tls?

The reason I'm asking is because I'd like to be able to access the underlying TLS stream of a proxied connection, to do some further validation of the server certificate. This is almost possible using the Upgrade mechanism in hyper but to downcast the hyper::upgrade::Upgraded back to its underlying stream requires your ProxyStream struct to be made public and requires it to use a tokio_native_tls::TlsStream not a tokio_tls::TlsStream as the latter doesn't allow access to the underlying native_tls stream (and also looks to be deprecated in favour of the former).

The documentation around what comprises a "Secured" vs "Regular" proxy is a bit thin. I was trying to track down some issues around the fact that I use a proxy wrapper whether or not a proxy is in use (which keeps our types a bit simpler -- we just create a proxy with Intercept::None set and provide a bogus proxy server). But if you do this without using the from_proxy_unsecured ProxyConnector constructor, a BadDER error gets thrown.

Obviously, I fixed my particular implementation, but the documentation for both ProxyConnector::from_proxy and ProxyConnector::from_proxy_unsecured is the same ("Create a proxy connector and attach a particular proxy").

With an unsecured proxy, hyper_proxy appears to work fine for HTTP destinations (e.g. http://github.com), but fails for HTTPS sites (e.g. https://github.com).

It fails with either connection closed before message completed or invalid HTTP version parsed, depending on the site.

Am I doing something wrong in configuring the client/connector, or is this a bug?

Minimal repro

Proxy setup

squid proxy in my LAN at 192.168.1.20 with the following in squid.conf:

http_port 3128
http_access allow all
cache deny all  # don't cache anything

Code

Example repo
Direct file link

use hyper::client::HttpConnector;
use hyper::Body;
use hyper::Client;
use hyper_proxy::Intercept;
use hyper_proxy::Proxy;
use hyper_proxy::ProxyConnector;

#[tokio::main]
async fn main() {
    let proxy = {
        let proxy_uri = "http://192.168.1.20:3128".parse().unwrap();
        let proxy = Proxy::new(Intercept::All, proxy_uri);
        let connector = hyper::client::HttpConnector::new();
        let proxy_connector = ProxyConnector::from_proxy_unsecured(connector, proxy);
        proxy_connector
    };

    let client: Client<ProxyConnector<HttpConnector<_>>, Body> =
        hyper::Client::builder().build(proxy);

    let req = client.get("http://github.com".parse().unwrap());
    let res = req.await;

    println!("HTTP Uri");
    println!("========");
    match res {
        Ok(body) => println!("{:?}", body),
        Err(x) => eprintln!("{:}", x.message()),
    }

    let req = client.get("https://github.com".parse().unwrap());
    let res = req.await;

    println!("\n\nHTTPS Uri (github)");
    println!("==================");
    match res {
        Ok(body) => println!("{:?}", body),
        Err(x) => eprintln!("{:}", x.message()),
    }

    let req = client.get("https://instagram.com".parse().unwrap());
    let res = req.await;

    println!("\n\nHTTPS Uri (instagram)");
    println!("=====================");
    match res {
        Ok(body) => println!("{:?}", body),
        Err(x) => eprintln!("{:}", x.message()),
    }
}

Run output

HTTP Uri
========
Response { status: 301, version: HTTP/1.1, headers: {"date": "Sat, 09 Jul 2022 01:31:18 GMT", "content-length": "0", "location": "https://github.com/", "x-cache": "MISS from 6bd9b44c254b", "x-cache-lookup": "MISS from 6bd9b44c254b:3128", "via": "1.1 6bd9b44c254b (squid/3.5.12)", "connection": "keep-alive"}, body: Body(Empty) }


HTTPS Uri (github)
==================
connection closed before message completed


HTTPS Uri (instagram)
=====================
invalid HTTP version parsed

Seems the example is not compatible with the new version of hyper

Is there a way to diagnose where a proxy connector might be introducing corruption when working with TLS? I'm running the example http_proxy and am able to successfully connect through reqwest but not this crate. My setup is

let mut http = hyper::client::connect::HttpConnector::new();
http.enforce_http(false);

let proxy_uri = "https://127.0.0.1:8100".to_string();
let mut proxy = hyper_proxy::Proxy::new(
    hyper_proxy::Intercept::All,
    proxy_uri.parse::<http::Uri>()
        .map_err(|err| Error::InvalidUri(proxy_uri, err.to_string()))?,
);
proxy.set_header(
    http::header::HeaderName::from_static("proxy-connection"),
    http::header::HeaderValue::from_static("Keep-Alive"),
);
endpoint.connect_with_connector(
    hyper_proxy::ProxyConnector::from_proxy(http, proxy)?).await.unwrap()

where endpoint is a tonic Channel. The handshake seems to kick off but then a tonic::transport::Error(Transport, hyper::Error(Connect, Custom { kind: InvalidData, error: CorruptMessage })) is returned.

Hi,

libproxy is a multi-platform library that provides dynamic proxy management. It's extremely useful in big companies that force users to use different proxies based on a PAC file. @jplatte made rust bindings for libproxy already. Do you think it would be possible to support libproxy in hyper-proxy?

In set_authorization if the intercept is not http or https, it sets both Authorization and Proxy-Authorization headers.

It's not clear to me why this should be the case and strikes me as a bug. It's not clear to me why we would need to set different headers according to the scheme of the intercept at all. I'm not finding any relevant information in the RFC.

Do you have any ideas? Thanks

I'm looking to integrate hyper-proxy into egg-mode (see egg-mode-rs/egg-mode#101). However, i also recently had an issue reported where a user couldn't use the native certificate store as loaded by rustls-native-certs (see egg-mode-rs/egg-mode#92). I fixed this at the time by exposing a feature from hyper-rustls that made it use statically-linked certificates from the webpki-roots crate instead of rustls-native-certs. Would you be open to exposing a Cargo feature to switch between these certificate stores?

This seems to be fixed in master, but not yet published. Probably you just forgot it after the release of 0.9.0.

Hello!

When I copy the example code from the readme I get the following compilation error:

34  |             proxy.set_authorization(auth);
    |                   ----------------- ^^^^ expected `Authorization<_>`, found `Authorization<Basic>`
    |                   |
    |                   arguments to this method are incorrect
    |
    = note: `Authorization<Basic>` and `headers::common::authorization::Authorization<_>` have similar names, but are actually distinct types

Code used:

        let proxy = {
            let proxy_uri = "http://my-proxy:8080".parse().unwrap();
            let mut proxy = Proxy::new(Intercept::All, proxy_uri);
            let auth = Authorization::basic("John Doe", "Agent1234");
            proxy.set_authorization(auth);
            let connector = HttpConnector::new();
            let proxy_connector = ProxyConnector::from_proxy(connector, proxy).unwrap();
            proxy_connector
        };

Rust version: 1.75.0

Is this an error in my environment/code or the current lib? Thanks!

If a provided URL is http rather than https, this bit of code will accidentally set the outgoing port to 443 without checking the scheme if force_connect is enabled

            if uri.scheme() == Some(&httpur::i::Scheme::HTTPS) || p.force_connect {
                let host = host.to_owned();
                let port = uri.port_u16().unwrap_or(443);

It's a major first release which the whole ecosystem is supporting, so we should probably follow suit in order to provide better compatibility for lib users. I'm creating this issue so I can link and track from the web3 upgrade issue.

On crates.io, the current version is 0.9.1: https://crates.io/crates/hyper-proxy
But here on GitHub it's 0.9.0 only: https://github.com/tafia/hyper-proxy/blob/master/Cargo.toml#L3
When I download the code from crates.io, it's different to the code here on GitHub. Did you forget to push to GitHub? Please do it :)

I can't access the https site after hooking up the http proxy, without hooking up the proxy the https site works fine.

Hi,

there are no examples folders in the source folder and I could not find anything related to socks5 on the issues page so I better ask to clarify.

I need a library to create a Hyper connector that supports HTTPS (through rustls) and Socks5, do you support this? if yes, could you please point me to a code example and or documentation?

thank you

This crate depends on the http crate (for obvious reason). That crate has had a major version bump to v1.0.0. This crate should track this update to allow interoperability with other crates that are now bumping their dependency as well.

I'm doing something like the following, so that I can intentionally mitm tls connections from my program:

let tls = TlsConnector::builder()
      .danger_accept_invalid_hostnames(true)
      .danger_accept_invalid_certs(true)
      .build();
let mut http = HttpConnector::new(4);
http.enforce_http(false);
let proxy_uri = proxy_uri.parse().unwrap();
let proxy = Proxy::new(Intercept::All, proxy_uri);
let connector = HttpsConnector::from( (http, tls.unwrap()) );
Client::builder()
      .build::<_, hyper::Body>( ProxyConnector::from_proxy(connector, proxy).unwrap() )

and the tls certificate for the proxy is still being checked for validity. An attempt to switch to '.from_proxy_unsecured(...)' on the ProxyConnector is also failing, in that connections to secured hosts are being downgraded to mere http connections vs port 443.