dmippolitov / pydnsbl Goto Github PK
View Code? Open in Web Editor NEWAsync dnsbl spam lists checker based on asyncio/aiodns.
License: MIT License
Async dnsbl spam lists checker based on asyncio/aiodns.
License: MIT License
Hi,
I am using pydnsbl in one of our application and it was working fine during the development stage. The results returned were accurate (i.e. it returned True for domains and IPs that exists in Blacklists).
However, we recently deployed our product to an EC2 Instance and no matter what domain or IP is provided, pydnsbl returns True saying the domain / IP is blacklisted (even though it is not).
I am just wondering if I am doing something wrong. Do I need to make some changes?
Any help would be appreciated.
Thank You
Working in JupyterLab, Python 3.8.
Using pydnsbl.DNSBLDomainChecker().check() function and it throws Runtime error.
<ipython-input-11-bdf0be99db20> in <module>
1 checker = pydnsbl.DNSBLDomainChecker()
----> 2 result = checker.check(domain)
/volumes/code/.venv/lib/python3.8/site-packages/pydnsbl/checker.py in check(self, request)
146
147 def check(self, request):
--> 148 return self._loop.run_until_complete(self.check_async(request))
149
150 def bulk_check(self, requests):
/usr/local/lib/python3.8/asyncio/base_events.py in run_until_complete(self, future)
590 """
591 self._check_closed()
--> 592 self._check_running()
593
594 new_task = not futures.isfuture(future)
/usr/local/lib/python3.8/asyncio/base_events.py in _check_running(self)
550 def _check_running(self):
551 if self.is_running():
--> 552 raise RuntimeError('This event loop is already running')
553 if events._get_running_loop() is not None:
554 raise RuntimeError(
RuntimeError: This event loop is already running```
When running with unittest.IsolatedAsyncioTestCase and feeding async_check tasks into an asyncio.Queue to be consumed, the following error occurs:
Traceback (most recent call last):
File "\lib\unittest\async_case.py", line 65, in _callTestMethod
self._callMaybeAsync(method)
File "\lib\unittest\async_case.py", line 88, in _callMaybeAsync
return self._asyncioTestLoop.run_until_complete(fut)
File "\lib\asyncio\base_events.py", line 616, in run_until_complete
return future.result()
File "\lib\unittest\async_case.py", line 102, in _asyncioLoopRunner
ret = await awaitable
...
File "\lib\site-packages\pydnsbl\checker.py", line 140, in check_async
results = await asyncio.gather(*tasks)
File "\lib\site-packages\pydnsbl\checker.py", line 122, in dnsbl_request
response = await self._resolver.query(dnsbl_query, 'A')
RuntimeError: Task <Task pending name='Task-2' coro=<dnsbl_request() running at \lib\site-packages\pydnsbl\checker.py:122> cb=[gather.<locals>._done_callback() at \lib\asyncio\tasks.py:751] created at \lib\asyncio\tasks.py:806> got Future <Future pending> attached to a different loop
This happens when not manually creating and passing a loop into BaseDNSBLChecker
: BaseDNSBLChecker
will then create and set a new loop in the main thread, even if an existing loop is already running.
The fix for this is to change the code in BaseDNSBLChecker
(line 95 of file):
if not loop:
self._loop = asyncio.new_event_loop()
asyncio.set_event_loop(self._loop)
to this:
if not loop:
self._loop = asyncio.get_event_loop()
asyncio.set_event_loop(self._loop)
Hello,
Could you publish a new release on PyPI with the fix for broken provider?
Thank you.
Hi, Sir
Thank you very much for this great tools. but i saw some providers on your list are unreachable now. could you help to remove it from the list ?
thank you so much.
multi.uribl.com
Using:
example1.com (no listing)
example.com (whitelisted)
'blacklisted': True, 'detectedBY': {'multi.uribl.com': ['unknown']}, 'notation': <DNSBLResult: example1.com [BLACKLISTED] (1/58)>}
and if "Whitelisted" same:
'blacklisted': True, 'detectedBY': {'multi.uribl.com': ['unknown']}, 'notation': <DNSBLResult: example.com [BLACKLISTED] (1/58)>}
module 'pydnsbl' has no attribute 'DNSBLIpChecker'
it was made sure that pip install command was run along iwth other libs
I would love to integrate pydnsbl into a project of mine but I need ipv6 support. Would it be possible to add ipv6 support easily?
domain names can be checked in the dbl.spamhaus.org zone for instance
https://www.spamhaus.org/faq/section/Spamhaus%20DBL
Is it possible to add other providers like the ones listed in the list of multirbl.valli.org?
AttributeError: 'DNSBLDomainChecker' object has no attribute 'detected_by'
I get this on any domain I try to check even with the example in the code.
Hello!
Under Windows 10 x64 (build 2004) and Python 3.8.1 x64, an error appears:
From cffi callback <function _sock_state_cb at 0x000002A1125D0310>:
Traceback (most recent call last):
File "C:\Python\Python38\lib\site-packages\pycares\__init__.py", line 91, in _sock_state_cb
sock_state_cb(socket_fd, readable, writable)
File "C:\Python\Python38\lib\site-packages\aiodns\__init__.py", line 104, in _sock_state_cb
self.loop.add_reader(fd, self._handle_event, fd, READ)
File "C:\Python\Python38\lib\asyncio\events.py", line 501, in add_reader
raise NotImplementedError
NotImplementedError
For Python 3.8 for Windows and above is required:
import sys
import aiodns
import asyncio
if sys.platform == 'win32' and sys.version_info >= (3, 8):
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
# Done...
https://docs.python.org/3.8/library/asyncio-platforms.html#asyncio-platform-support
Best Regards,
Andrey Klimov
Can you remove http://dynip.rothen.com/ from the BASE_PROVIDERS
?
The domain is down/under construction.
According to https://datatracker.ietf.org/doc/html/rfc5782#section-2.4 as mentioned in the project, the method for converting IPv6 address to IPv6 DNSxLs should be done by first expanding 0's -> reverse -> remove :
-> concatenate each digit with .
For example
2001:db8:1:2:3:4:567:89ab
-> b.a.9.8.7.6.5.0.4.0.0.0.3.0.0.0.2.0.0.0.1.0.0.0.8.b.d.0.1.0.0.2
Also need to consider the case where ::
exists
2600:XXXX::f03c:91ff:fe50:d2
-> 2.d.0.0.0.5.e.f.f.f.1.9.c.3.0.f.0.0.0.0.0.0.0.0.X.X.X.X.0.0.6.2
In line 165 of pydnsbl/pydnsbl/checker.py
elif address.version == 6:
# according to RFC: https://tools.ietf.org/html/rfc5782#section-2.4
request_stripped = request.replace(':', '')
return '.'.join(reversed([x for x in request_stripped]))
It seems like the project didn't expand 0's first. Perhaps it should expand 0's first before continuing the rest steps.
Running from a venv on unbuntu and flask, I have imported pycares alongside python 3.7
Previously 3.6.9 it worked fine.
Now I am getting an error:
File "/var/www/ctea/app/venv/lib/python3.7/site-packages/pydnsbl/checker.py", line 17, in
import aiodns
File "/var/www/ctea/app/venv/lib/python3.7/site-packages/aiodns/init.py", line 4, in
import pycares
File "/var/www/ctea/app/venv/lib/python3.7/site-packages/pycares/init.py", line 2, in
from ._cares import ffi as _ffi, lib as _lib
ModuleNotFoundError: No module named 'pycares._cares'
If I run python within the venv I can import pycares fine. The issue is when importing pycares from wsgi. I am not the only one with this issue:
constverum/ProxyBroker#124
constverum/ProxyBroker#157
Thanks in advance!
Within checker.py line 86:
Replace with:
This fixed my issue. See below thread for more info:
https://stackoverflow.com/questions/41594266/asyncio-with-django/46140106
Great job and thanks! Would you mind updating and pushing to PIP?
in the end of script always: AttributeError: 'Channel' object has no attribute 'destroy'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.