Comments (5)
This issue is slightly alarmist. It is correct that http://0177.0.0.1/
isn't blocked but it doesn't seem to "work". Under Python 3.9.13:
>>> import advocate
>>> print(advocate.get("http://0177.0.0.1"))
0177.0.0.1 80
((<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', (IPv4Address('177.0.0.1'), 80)),)
('177.0.0.1', 80)
As you can see this resolves down to 177.0.0.1
which isn't a private IP.
from advocate.
I think since the decimal and hex versions of http://127.0.0.1
are blocked (2130706433, 0x7F000001), it would follow that the octal version should be blocked.
@spladug @JordanMilne are you still maintaining this library? Is it possible for someone at Reddit to maintain this since they make use of the lib? I might also be willing to maintain it if there's no one available.
from advocate.
@okyanusoz Thanks for filing an issue! I'm not able to reproduce advocate.get("http://0177.0.0.1/")
successfully connecting to localhost
on my system, it raises a validation exception. Are you sure that advocate.get("http://0177.0.0.1/")
is actually trying to connect to 127.0.0.1
on your system and not 177.0.0.1
? What OS are you on, and how did you install Python? What does this output when pasted in a python
shell?
import socket
socket.getaddrinfo("0177.0.0.1", 8000)
from advocate.
Hi @ColdHeat, sorry for the delay here, unfortunately I haven't had much time to work on OSS recently 😞. I'm in the middle of dropping support for Python 2 as part of cutting a 2.0 version of Advocate, and that's knocked loose a bunch of other TODOs like getting off of Travis and making the urllib3
patching less brittle. I expect to have a WIP branch up in the coming weeks though.
This issue is slightly alarmist. It is correct that http://0177.0.0.1/ isn't blocked but it doesn't seem to "work". [...] As you can see this resolves down to 177.0.0.1 which isn't a private IP.
Yep, though depending on your OS and what libc you're using, 0177.0.0.1
may either be interpreted as octal or decimal with a leading 0. On my system, the getaddrinfo()
impl treats it as an octal address, so the socket.getaddrinfo()
call returns 127.0.0.1
and advocate
blocks it appropriately:
>>> import advocate
>>> advocate.get("http://0177.0.0.1")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/src/advocate/advocate/api.py", line 108, in get
return request('get', url, **kwargs)
File "/src/advocate/advocate/api.py", line 94, in request
response = sess.request(method=method, url=url, **kwargs)
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/urllib3/connection.py", line 239, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/src/.virtualenvs/advocate/lib/python3.10/site-packages/urllib3/connection.py", line 205, in connect
conn = self._new_conn()
File "/src/advocate/advocate/connection.py", line 152, in _validating_new_conn
conn = conn_func(
File "/src/advocate/advocate/connection.py", line 123, in validating_create_connection
raise err
advocate.exceptions.UnacceptableAddressException: ('0177.0.0.1', 80)
>>> import socket
>>> socket.getaddrinfo("0177.0.0.1", 8000)
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8000)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_DGRAM: 2>, 17, '', ('127.0.0.1', 8000)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_RAW: 3>, 0, '', ('127.0.0.1', 8000))]
# ^ 0177.0.0.1 is equivalent to 127.0.0.1 on this system, so advocate refuses to connect.
I think since the decimal and hex versions of http://127.0.0.1 are blocked (2130706433, 0x7F000001), it would follow that the octal version should be blocked.
So Advocate doesn't really know anything about IP encodings, it just takes whatever host is provided as part of the URL and hands it off to socket.getaddrinfo()
, which will do any DNS lookups and IP canonicalization as appropriate. It can't even really tell the difference between an IP or a hostname in the host portion of a URI.
Rather than play whack-a-mole with all the different possible IP representations (https://android.googlesource.com/platform/bionic/+/master/tests/arpa_inet_test.cpp#27 has a handful for musl libc, I'm sure there are more depending on the system,) we just focus on addresses that we know resolve to a banned address at the point the connection is being made per getaddrinfo()
. Barring something truly bizarre, although all bets are off with computers, the IP returned by getaddrinfo()
should be unambiguous, and that's ultimately what we use for validation and connection.
from advocate.
Closing since this doesn't appear to be reproducible, please re-open if you have more details that could help reproduce!
For future readers, there are several system configurations (like OS X, perhaps on Linux with an unusual libc) where "0177.0.0.1" will resolve to "177.0.0.1" rather than "127.0.0.1". advocate
will only block requests to "0177.0.0.1" on systems where it resolves to "127.0.0.1" because, by design, advocate
only checks what an address resolves to at the point the request is being made.
from advocate.
Related Issues (18)
- Tests are broken with `setup.py test`
- got exception HOT 2
- New release? HOT 5
- Make netifaces dependency optional HOT 8
- Add support for user-defined DNS64 prefixes
- Rip out Python 2.x support
- type hints HOT 2
- Upload sdist and wheel to PyPI in CI on release creation
- Python versions
- hostname tests are skipped due to incorrect type of `canonname`
- Is advocate dead? Are there alternatives? HOT 1
- What should be done about IPv6? HOT 1
- Tests broken with requests 2.9.1
- Strange behavior with RequestsAPIWrapper.head() HOT 4
- Refactor requests compatibility tests to work with new requests versions
- Advocate stops working with requests HOT 10
- Unvendor ipaddress module
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from advocate.