abhinavsingh / proxy.py Goto Github PK

curl --proxy 127.0.0.1:1080 -v https://www.baidu.com

2015-07-22 23:13:53,431 - ERROR - pid:366 - Exception while handling connection <socket._socketobject object at 0x257abb0> with reason ValueError('need more than 1 value to unpack',)
Traceback (most recent call last):
File "/usr/bin/proxy.py", line 494, in run
self._process()
File "/usr/bin/proxy.py", line 479, in _process
if self._process_rlist(r):
File "/usr/bin/proxy.py", line 447, in _process_rlist
self._process_request(data)
File "/usr/bin/proxy.py", line 367, in _process_request
host, port = self.request.url.path.split(COLON)
ValueError: need more than 1 value to unpack
2015-07-22 23:13:53,433 - INFO - pid:366 - 127.0.0.1:38824 - CONNECT None:None

For a POC I've been working on, I added a couple lines to redirect certain requests to a mocked destination where the requests can be verified by an automated test suite.
What do you think would be a good way to add this ability as a feature, like a hook function or a user-definable hash map?

For example- In my implementation, most requests simply pass through without being touched, however, when a request to api.segment.io enters the proxy, it gets redirected to a mocked api.

If you want, I could take a stab at it and submit a PR for you to review.

when I run your proxy.py program when I only sniff HTTP URL not https. what's the problem

python proxy.py --port 12500 --ipv4

then configure Firefox proxy 127.0.0.1:12500

it gives errors and not working

 return recvfds(s, 1)[0]
  File "/usr/lib/python3.7/multiprocessing/reduction.py", line 161, in recvfds
    len(ancdata))
RuntimeError: received 0 items of ancdata

For brief background see #68 (comment)

What do the different log levels mean? BTW Excellent Work.

Hi,
I want to know how can I control client request is HTTP or HTTPS ?

I suspect due to underlying system calls expecting a 4-tuple IPv6 address, though it seems to work fine on macOS. Needs more investigation.

$ ./proxy.py --log-level d
2019-07-30 13:03:26,589 - INFO - pid:25332 - setup:730 - Starting 1 workers
2019-07-30 13:03:26,591 - ERROR - pid:25332 - run:698 - Exception while running the server error(99, 'Cannot assign requested address')
Traceback (most recent call last):
  File "./proxy.py", line 690, in run
    self.socket.bind((self.hostname, self.port))
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 99] Cannot assign requested address
2019-07-30 13:03:26,592 - INFO - pid:25332 - shutdown:742 - Shutting down 1 workers
2019-07-30 13:03:26,595 - INFO - pid:25332 - run:701 - Closing server socket

HTTPS Proxy connection from client
I bind a domain to my vps, how to set up https connection between client and vps?

Reproducible by running proxy.py on smallest GCP instance (or a system with <500Mb of free RAM) and stress testing it via fast.com speed tests.

Traceback (most recent call last):
File "./proxy.py", line 1082, in run
  teardown = self.run_once()
File "./proxy.py", line 1067, in run_once
  teardown = plugin.read_from_descriptors(readable)
File "/home/mailsforabhinav/proxy.py/proxy.py", line 834, in read_from_descriptors
  self.response.bytes += raw
MemoryError

The subprocess consume 100% CPU of one core and memory grows to hundreds of megabytes, and the speed falls as low as 100kB/s .

When configuring a browser to use proxy.py, domain names defined in /etc/hosts do not work. I think that's a common issue with a lot of local proxies. It may seem superfluous, but it would be really useful for automated testing of web applications. How might one add that feature?

Some clients (especially Android Emulator) are requesting to establish HTTPS tunnel over IPv6. socket.create_connection doesn't seem to work with IPv6 addresses until a 4-tuple is passed. Relevant code here https://github.com/abhinavsingh/proxy.py/blob/develop/proxy.py#L389

Proxy currently also errors out with reason ValueError('too many values to unpack (expected 2)') due to IPv4 expectations here https://github.com/abhinavsingh/proxy.py/blob/develop/proxy.py#L474

In macOS 10.14.2, run with docker

2019-02-18 10:16:35,884 - ERROR - run:604 - Exception while handling connection <socket.socket fd=1024, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('172.17.0.2', 8899), raddr=('172.17.0.1', 50078)> with reason ValueError('filedescriptor out of range in select()')
Traceback (most recent call last):
  File "./proxy.py", line 600, in run
    self._process()
  File "./proxy.py", line 575, in _process
    r, w, x = select.select(rlist, wlist, xlist, 1)
ValueError: filedescriptor out of range in select()

I was just wondering how the request for information from another server should be formed when it reaches the proxy. Do I send a request to localhost:8899 and include a query=remote_address parameter, or just q=remote_address?

It gets caught in a loop when processing chunked requests where there are additional headers fields after the message body. Occurs because:

1. The self.size in ChunkParser.process is 0
2. Thus the self.state is set to COMPLETE, but the len(data) > 0

great code and praise you!

i have some question.

see the code,
proxy.py:495

            if self.request.method == b'CONNECT':
                self.client.queue(PROXY_TUNNEL_ESTABLISHED_RESPONSE_PKT)
            # for usual http requests, re-build request packet
            # and queue for the server with appropriate headers
            else:
                self.server.queue(self.request.build(
                    del_headers=[b'proxy-authorization', b'proxy-connection', b'connection', b'keep-alive'],
                    add_headers=[(b'Via', b'1.1 proxy.py v%s' % version), (b'Connection', b'Close')]
                ))

when request is https, it send CONNECT to Proxy Server.
here is problem, why we(the proxy) tell client success?
why not let the remote server tell to client result?

we should just do transpond, so i modify the code:
self.client.queue(PROXY_TUNNEL_ESTABLISHED_RESPONSE_PKT)
to
self.server.queue(data)
it's work right!

what about you think?
look forward to your reply!

have a pc with two network card,each crad bind five ip
could add a function to choose out-go-ip when send data to server?
sorry for my poor-ability in english

When the request method is POST, the content-length is not set and the post data is empty, the proxy will not reach HTTP_PARSER_STATE_COMPLETE. Maybe it's the standard of RFC. If it's not, the solution below is practical:

    if self.state == HTTP_PARSER_STATE_HEADERS_COMPLETE and \
    self.type == HTTP_REQUEST_PARSER and \
    self.method == b"POST" and \
    len(data) == 0:
        self.state = HTTP_PARSER_STATE_COMPLETE

#66 is a side-effect of proxy.py maintaining raw bytes for requests and responses between client and server. After successful parsing we MUST ignore raw bytes to reduce the operational memory footprint.

Specifically following line stores raw response bytes only for dumping response length in access logs.

Can use browser by this proxy?
Just like I want use firefox Network Settings by this proxy
Thanks

As Descriped in multiprocessing module issues in windows, the parameters of multiprocessing.Process.init() must be pickleable , while the socket.socket object is not. So running this proxy.py in Windows will raise a Error.

This error is much like the issues discussed in the following stackoverflow post: http://stackoverflow.com/questions/33064982/how-can-i-shred-socket-object-over-python-multiprocess-it-raises-eoferror.
And the official python guide for the reason of this issue is as follows: https://docs.python.org/2/library/multiprocessing.html#windows

If the data ends with the size without CRLF, this patch can prevent the parser from raising an error.

Test program:

from proxy import *
cp = ChunkParser()
data1 = b'3\r\nabc\r\n4'
data2 = b'\r\nabcd\r\n'
cp.parse(data1)			# error
# would be correct if do `cp.parse(data1 + data2)`

As per title: it would be useful to have such a feature

Great work, anyway!

how can i set my public ip? my server has thousands of ipv6. is it possible to choose one for each proxy?

Hi Abhinav and others,

I'd like to build a customised proxy by subclassing your proxy.py but as it's a single file in bin/I believe I need to copy it out, complicating my workflow compared with simply pip install proxy.py.
Have you considered splitting it into a module (usefull in my case) and a runner with main method to be placed in bin?

Thanks for a nice piece of python :)

The parser part expects a GET request or a POST request with unempty body. Should change this line to make it work

Seen following logs when running Android emulator via proxy.py

2019-09-02 21:11:15,572 - INFO - pid:6909 - set_host_port:409 - b'[2607:f8b0:4005:802::2003]:443'
2019-09-02 21:11:15,572 - ERROR - pid:6909 - run:1083 - Exception while handling connection <socket.socket fd=44, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.0.0.236', 8899), raddr=('10.0.0.236', 59065)> with reason ValueError('too many values to unpack (expected 2)')
Traceback (most recent call last):
  File "./proxy.py", line 1077, in run
    teardown = self.run_once()
  File "./proxy.py", line 1042, in run_once
    self.request.parse(client_data)
  File "./proxy.py", line 429, in parse
    more, raw = self.process(raw)
  File "./proxy.py", line 460, in process
    self.process_line(line)
  File "./proxy.py", line 501, in process_line
    self.set_host_port()
  File "./proxy.py", line 410, in set_host_port
    self.host, self.port = self.url.path.split(COLON)
ValueError: too many values to unpack (expected 2)

Hello, I'm new in python and I am using this proxy but I have a problem, when lots of requests are being performed the proxy stops working because of the file @#descriptor error (FD_SETSIZE limit).

I was reading what this problem only happens with select.select() and a workaround or solution may be using select.epoll() instead of select.select() but I didn't know how to do this correctly.

Thanks

How can I access HTML Source from proxy.py?

I tried to print() data and buffer and body but failed to print any of them..

tornado 6.0.2 has breaking change for @web.asynchronous decorator
tornadoweb/tornado#2641

I tried to install the code from pypi as written in README. Everything seemed to work fine but I got spurious connection problems in certain situations and I wasted about one hour with debugging. Eventually I tried to git version and everything worked just fine.

Is there anything blocking a new pypi release? It seems the current git code contains important fixes when using the proxy with modern browsers.

Running IPv4 only server on Python 2.7.16 on Mac OS X (installed via homebrew) and trying to perform the following request:

$ export http_proxy=http://127.0.0.1:8899
$ curl -v checkip.amazonaws.com
* Uses proxy env variable http_proxy == 'http://127.0.0.1:8899'
*   Trying 127.0.0.1:8899...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8899 (#0)
> GET http://checkip.amazonaws.com/ HTTP/1.1
> Host: checkip.amazonaws.com
> User-Agent: curl/7.65.3
> Accept: */*
> Proxy-Connection: Keep-Alive
> 

It hangs.

On the server side this is what I see:

$ python proxy.py --ipv4
2019-08-10 09:36:25,689 - INFO - pid:19824 - setup:730 - Starting 12 workers
2019-08-10 09:36:25,697 - INFO - pid:19824 - run:692 - Started server on port 8899
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.16/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 268, in _feed
    send(obj)
TypeError: expected string or Unicode object, NoneType found

tinyproxy has the DisableViaHeader option. Could this be added to this implementation?

When work with --basic-auth, Chrome always give an error: ERR_PROXY_AUTH_UNSUPPORTED

HttpParser fails to reach state COMPLETE for following scenario:

2019-09-17 16:23:54,643 - DEBUG - pid:11933 - run_once:1159 - Client is ready for reads, reading
2019-09-17 16:23:54,643 - DEBUG - pid:11933 - recv:123 - received 31 bytes from b'client'
2019-09-17 16:23:54,644 - DEBUG - pid:11933 - recv:124 - b'CONNECT pypi.org:443 HTTP/1.0\r\n'
2019-09-17 16:23:54,644 - DEBUG - pid:11933 - run_once:1177 - 2
2019-09-17 16:23:54,671 - DEBUG - pid:11933 - run_once:1159 - Client is ready for reads, reading
2019-09-17 16:23:54,671 - DEBUG - pid:11933 - recv:123 - received 2 bytes from b'client'
2019-09-17 16:23:54,672 - DEBUG - pid:11933 - recv:124 - b'\r\n'

I tested through IE, curl, firefox. It works good. But on safari, it always get an error.

i am trying to integrate v2 of luis api in my bot but it not work the sample code is as below

var bot = new builder.UniversalBot(connector);
server.post('/api/messages', connector.listen());
var recognizer = new builder.LuisRecognizer('https://api.projectoxford.ai/luis/v2.0/apps/72556e01-956c-4784-a26d-b7fb3e00df7b?subscription-key=15f282c492514272a354088915b56321');
var intents = new builder.IntentDialog({ recognizers: [recognizer] });
bot.dialog('/', intents);
intents.matches('TSUGGEST', [
function (session, args, next) {
var task = builder.EntityRecognizer.findEntity(args.entities);
var task1=builder.EntityRecognizer.findEntity(args.entities,'builtin.number');
if (!task1) {
builder.Prompts.text(session, "Please tell article name");
} else {
next({ response: task1.entity });
}
},
function (session, results) {
if (results.response) {
// ... save task
session.send("Ok... '%s' task.", results.response);
} else {
session.send("Ok");
}
}]);

2016-06-10 11:45:42,920 - INFO - pid:2296 - Starting server on port 8899
2016-06-10 11:45:45,696 - ERROR - pid:2296 - Exception while running the server PicklingError("Can't pickle <built-in method recvfrom_into of socket.socket object at 0x024692C0>: it's not found as __
main_.recvfrom_into",)
Traceback (most recent call last):
File "proxy.py", line 529, in run
self.handle(client)
File "proxy.py", line 545, in handle
proc.start()
File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
self._popen = Popen(self)
File "C:\Python27\lib\multiprocessing\forking.py", line 277, in init
dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 554, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 754, in save_global
(obj, module, name))
PicklingError: Can't pickle <built-in method recvfrom_into of _socket.socket object at 0x024692C0>: it's not found as main.recvfrom_into
2016-06-10 11:45:45,698 - INFO - pid:2296 - Closing server socket
Drücken Sie eine beliebige Taste . . . Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
self = load(from_parent)
File "C:\Python27\lib\pickle.py", line 1384, in load
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 864, in load
dispatchkey
File "C:\Python27\lib\pickle.py", line 886, in load_eof
raise EOFError
EOFError

Using Firefox 47.0

hello,
how listen only on specifiq ipv6 address

python /tmp/proxy.py --hostname [fd00::8e2:97ff:fe2e:f1ea] --port 8081 --log-level DEBUG
2019-04-18 15:32:05,116 - INFO - run:633 - Starting server on port 8081
2019-04-18 15:32:05,125 - ERROR - run:643 - Exception while running the server gaierror(-2, 'Name or service not known')
Traceback (most recent call last):
  File "/tmp/proxy.py", line 636, in run
    self.socket.bind((self.hostname, self.port))
  File "/usr/lib64/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
gaierror: [Errno -2] Name or service not known
2019-04-18 15:32:05,126 - INFO - run:645 - Closing server socket

python /tmp/proxy.py --hostname fd00::8e2:97ff:fe2e:f1ea --port 8081 --log-level DEBUG
2019-04-18 15:32:17,082 - INFO - run:633 - Starting server on port 8081
2019-04-18 15:32:17,082 - ERROR - run:643 - Exception while running the server gaierror(-9, 'Address family for hostname not supported')
Traceback (most recent call last):
  File "/tmp/proxy.py", line 636, in run
    self.socket.bind((self.hostname, self.port))
  File "/usr/lib64/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
gaierror: [Errno -9] Address family for hostname not supported
2019-04-18 15:32:17,082 - INFO - run:645 - Closing server socket

The server starts as normal. But when sending traffic through it the proxy crashes with this error provided below. It is runs under Windows 7 if that helps. Where is the problem? The client displays message "Connection reset".

C:\Users\IEUser>python -m proxy --host 192.168.1.5 --port 8899
2018-01-25 00:16:30,895 - INFO - pid:3684 - Starting server on port 8899
2018-01-25 00:16:59,334 - ERROR - pid:3684 - Exception while running the server
PicklingError("Can't pickle <built-in method recvfrom_into of _socket.socket object at 0x000000000334CBF0>: it's not found as main.recvfrom_into",)
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\proxy.py", line 529, in run
self.handle(client)
File "C:\Python27\lib\site-packages\proxy.py", line 545, in handle
proc.start()
File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
self._popen = Popen(self)
File "C:\Python27\lib\multiprocessing\forking.py", line 277, in init
dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 425, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 554, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 754, in save_global
(obj, module, name))
PicklingError: Can't pickle <built-in method recvfrom_into of _socket.socket object at 0x000000000334CBF0>: it's not found as main.recvfrom_into
2018-01-25 00:16:59,348 - INFO - pid:3684 - Closing server socket

C:\Users\IEUser>Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
self = load(from_parent)
File "C:\Python27\lib\pickle.py", line 1384, in load
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 864, in load
dispatchkey
File "C:\Python27\lib\pickle.py", line 886, in load_eof
raise EOFError
EOFError

it was working fine with commits on Jun 4, 2019, but now throws below error when accessing http://httpbin.org/ip

Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 266, in _feed
    send(obj)
TypeError: expected string or Unicode object, NoneType found

Use the proxy.py library.
python 2.7
the proxy successfully started via Pycharm with parameters, host:port: localhost:8890
From any browsers proxy-process stopped and raised the following exception, where I'm wrong?:
016-01-04 23:44:44,589 - INFO - pid:10340 - Starting server on hostname localhost and port 8890
2016-01-04 23:44:49,150 - DEBUG - pid:10340 - Accepted connection <socket._socketobject object at 0x024154C8> at address ('127.0.0.1', 61650)
2016-01-04 23:44:49,164 - ERROR - pid:10340 - Exception while running the server PicklingError("Can't pickle <built-in method recvfrom_into of _socket.socket object at 0x024EA440>: it's not found as main.recvfrom_into",)
Traceback (most recent call last):
File "C:/Program Files (x86)/Python27/Lib/site-packages/proxy.py", line 529, in run
self.handle(client)
File "C:/Program Files (x86)/Python27/Lib/site-packages/proxy.py", line 545, in handle
proc.start()
File "C:\Program Files (x86)\Python27\lib\multiprocessing\process.py", line 130, in start
self._popen = Popen(self)
File "C:\Program Files (x86)\Python27\lib\multiprocessing\forking.py", line 277, in init
dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Program Files (x86)\Python27\lib\multiprocessing\forking.py", line 199, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 548, in save_tuple
save(element)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Program Files (x86)\Python27\lib\pickle.py", line 748, in save_global
(obj, module, name))
PicklingError: Can't pickle <built-in method recvfrom_into of _socket.socket object at 0x024EA440>: it's not found as main.recvfrom_into
2016-01-04 23:44:49,167 - INFO - pid:10340 - Closing server socket

@abhinavsingh
i use python requests like this:
res = requests.get(url='https://httpbin.org/ip', proxies={'http':'ip:port', 'https':'ip:port'}, timeout=10, verify=False), got a timeout error, python program get no response, but found proxy.py debug log info is ok. and if request http://httpbin.org/ip, it's fine.
then i use internet explorer with the proxy, working correctly. It looks like a certificate problem when running in centos system. how to solve it.

I want to setup a proxy server, it access resource regard http_proxy, https_proxy, no_proxy settings, is it possible ?

When it receives an OPTIONS request it sends back an OPTIONS response twice even though it has received a following GET request. Caused by the HttpParser not being reset or properly being repopulated in process due to its state.