When proxying, I get additional "\r\n" added at the end of headers, which break request to sensitive servers (like Icecast Streaming Media Server).
How to reproduce:
I download this version of http_proxy.rb (latest):
https://github.com/igrigorik/em-proxy/blob/443da1df719ea62b6803dbf4f605a79da3b01f72/examples/http_proxy.rb
To make it work on ruby 1.8 (which is default on Centos 6) I only add this code at the top:
require 'rubygems'
class String
instance_method(:clear) rescue \
def clear
replace ""
end
end
Then run ruby http_proxy.rb
, otherwise unmodified, and and check packet data with tcpdump -X
.
Also, for bug visibility I modify /usr/lib/ruby/gems/1.8/gems/em-proxy-0.1.8/lib/em-proxy/connection.rb
by adding into relay_to_servers()
debugging p [:relay_to_servers, data]
before s.send_data
loop like this:
def relay_to_servers(processed)
if processed.is_a? Array
data, servers = *processed
# guard for "unbound" servers
servers = servers.collect {|s| @servers[s]}.compact
else
data = processed
servers ||= @servers.values.compact
end
p [:relay_to_servers, data] # <-- debugging for your pleasure
servers.each do |s|
s.send_data data unless data.nil?
end
end
Then, I connect to proxy and copy-paste simple request, like this:
$ telnet proxy 9889
GET http://www.google.com/test HTTP/1.1
Host: www.google.com
Proxy-Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
DNT: 1
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ru;q=0.6,es;q=0.4
(Request headers are taken from real browser request.)
Results
This is what I see in the stdout:
# ruby http_proxy.rb
listening on 0.0.0.0:9889...
[:relay_to_servers, "GET http://www.google.com/test HTTP/1.1\r\n"]
[:relay_to_servers, "Host: www.google.com\r\n"]
[:relay_to_servers, "Proxy-Connection: keep-alive\r\n"]
[:relay_to_servers, "Cache-Control: max-age=0\r\n"]
[:relay_to_servers, "Accept: */*\r\n"]
[:relay_to_servers, "User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\n"]
[:relay_to_servers, "DNT: 1\r\n"]
[:relay_to_servers, "Accept-Encoding: gzip, deflate, sdch\r\n"]
[:relay_to_servers, "Accept-Language: en-US,en;q=0.8,ru;q=0.6,es;q=0.4\r\n"]
New session: 7bffb510-9687-0133-c161-0dc54655ffbe ({"Accept-Encoding"=>"gzip, deflate, sdch", "Host"=>"www.google.com", "Cache-Control"=>"max-age=0", "DNT"=>"1", "User-Agent"=>"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36", "Proxy-Connection"=>"keep-alive", "Accept-Language"=>"en-US,en;q=0.8,ru;q=0.6,es;q=0.4", "Accept"=>"*/*"})
GET http://www.google.com/test HTTP/1.1
Host: www.google.com
Proxy-Connection: keep-alive
Cache-Control: max-age=0
Accept: */*
User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
DNT: 1
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ru;q=0.6,es;q=0.4
[:relay_to_servers, "GET http://www.google.com/test HTTP/1.1\r\nHost: www.google.com\r\nProxy-Connection: keep-alive\r\nCache-Control: max-age=0\r\nAccept: */*\r\nUser-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\r\nDNT: 1\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: en-US,en;q=0.8,ru;q=0.6,es;q=0.4\r\n\r\n"]
[:relay_to_servers, "\r\n"]
[:on_connect, "7bffb510-9687-0133-c161-0dc54655ffbe", nil]
[:on_response, etc.....
As you can see there is invalid additional [:relay_to_servers, "\r\n"]
.
Tcpdump output:
12:40:45.755302 IP xxxxxxxx.53790 > 173.194.71.106.http: Flags [P.], seq 1:350, ack 1, win 115, options [nop,nop,TS val 1531852163 ecr 2940865000], length 349
0x0000: 4500 0191 82af 4000 4006 3b89 xxxx xxxx E.....@.@.;....C
0x0010: adc2 476a d21e 0050 6622 b7dd 951f c590 ..Gj...Pf"......
0x0020: 8018 0073 0d75 0000 0101 080a 5b4e 3583 ...s.u......[N5.
0x0030: af4a 09e8 4745 5420 6874 7470 3a2f 2f77 .J..GET.http://w
0x0040: 7777 2e67 6f6f 676c 652e 636f 6d2f 7465 ww.google.com/te
0x0050: 7374 2048 5454 502f 312e 310d 0a48 6f73 st.HTTP/1.1..Hos
0x0060: 743a 2077 7777 2e67 6f6f 676c 652e 636f t:.www.google.co
0x0070: 6d0d 0a50 726f 7879 2d43 6f6e 6e65 6374 m..Proxy-Connect
0x0080: 696f 6e3a 206b 6565 702d 616c 6976 650d ion:.keep-alive.
0x0090: 0a43 6163 6865 2d43 6f6e 7472 6f6c 3a20 .Cache-Control:.
0x00a0: 6d61 782d 6167 653d 300d 0a41 6363 6570 max-age=0..Accep
0x00b0: 743a 202a 2f2a 0d0a 5573 6572 2d41 6765 t:.*/*..User-Age
0x00c0: 6e74 3a20 4d6f 7a69 6c6c 612f 352e 3020 nt:.Mozilla/5.0.
0x00d0: 2857 696e 646f 7773 204e 5420 352e 3129 (Windows.NT.5.1)
0x00e0: 2041 7070 6c65 5765 624b 6974 2f35 3337 .AppleWebKit/537
0x00f0: 2e33 3620 284b 4854 4d4c 2c20 6c69 6b65 .36.(KHTML,.like
0x0100: 2047 6563 6b6f 2920 4368 726f 6d65 2f34 .Gecko).Chrome/4
0x0110: 372e 302e 3235 3236 2e31 3036 2053 6166 7.0.2526.106.Saf
0x0120: 6172 692f 3533 372e 3336 0d0a 444e 543a ari/537.36..DNT:
0x0130: 2031 0d0a 4163 6365 7074 2d45 6e63 6f64 .1..Accept-Encod
0x0140: 696e 673a 2067 7a69 702c 2064 6566 6c61 ing:.gzip,.defla
0x0150: 7465 2c20 7364 6368 0d0a 4163 6365 7074 te,.sdch..Accept
0x0160: 2d4c 616e 6775 6167 653a 2065 6e2d 5553 -Language:.en-US
0x0170: 2c65 6e3b 713d 302e 382c 7275 3b71 3d30 ,en;q=0.8,ru;q=0
0x0180: 2e36 2c65 733b 713d 302e 340d 0a0d 0a0d .6,es;q=0.4.....
0x0190: 0a .
As you can see there is triple times 0d 0a
(which is "\r\n"), where it should be only two to finish HTTP request header.
Bug. ๐