sdgathman / pyspf Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
I believe that the example below should error out due to the '|' in the included domain before attempting a DNS lookup:
$ pyspf "v=spf1 include:a.b.com|c.d.com|e.f.com ~all" 1.1.1.1 [email protected] test.kitterman.com
result: ('permerror', 550, 'SPF Permanent Error: No valid SPF record for included domain: a.b.com|c.d.com|e.f.com: include:a.b.com|c.d.com|e.f.com') ~all
Since neither ALPHA nor DIGIT have "|", I think it's invalid per the ABNF.
The HELO field is ignored by Big Email, and rDNS demanded instead - mostly because so few admins could manage to actually put the hostname in HELO. I never understood why.
There have been some demands to remove +all - because some admins are using it dishonestly or incorrectly. Having examples of how to use it effectively (and honestly) might help forestall +all meeting the same fate as HELO.
Initially submitted to https://bugs.launchpad.net/pypolicyd-spf/+bug/1842005
Aug 30 06:09:43 central1 policyd-spf[29530]: Traceback (most recent call last):
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/bin/policyd-spf", line 809, in
Aug 30 06:09:43 central1 policyd-spf[29530]: instance_dict, configData, peruser, peruserconfigData)
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/bin/policyd-spf", line 623, in _spfcheck
Aug 30 06:09:43 central1 policyd-spf[29530]: mres = mfromquery.check()
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/lib/python3.4/site-packages/spf.py", line 547, in check
Aug 30 06:09:43 central1 policyd-spf[29530]: rc = self.check1(spf, self.d, 0)
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/lib/python3.4/site-packages/spf.py", line 586, in check1
Aug 30 06:09:43 central1 policyd-spf[29530]: return self.check0(spf, recursion)
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/lib/python3.4/site-packages/spf.py", line 906, in check0
Aug 30 06:09:43 central1 policyd-spf[29530]: if self.cidrmatch([arg], cidrlength): break
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/lib/python3.4/site-packages/spf.py", line 1348, in cidrmatch
Aug 30 06:09:43 central1 policyd-spf[29530]: for netwrk in [ipaddress.ip_network(ip) for ip in ipaddrs]:
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/lib/python3.4/site-packages/spf.py", line 1348, in
Aug 30 06:09:43 central1 policyd-spf[29530]: for netwrk in [ipaddress.ip_network(ip) for ip in ipaddrs]:
Aug 30 06:09:43 central1 policyd-spf[29530]: File "/usr/lib64/python3.4/ipaddress.py", line 84, in ip_network
Aug 30 06:09:43 central1 policyd-spf[29530]: address)
Aug 30 06:09:43 central1 policyd-spf[29530]: ValueError: '119.18.39.164\n' does not appear to be an IPv4 or IPv6 network
The txt record triggers the crash:
"v=spf1 ip4:203.143.89.146 ip4:203.27.138.188 ip4:119.18.39.164\010 include:spf.protection.outlook.com ~all"
I'm using pyspf to validate a list of domains include our servers in their SPF and it gives unknown
instead of permerror
for badly formed SPF records
I've seen this in the following instances:
def check_spf(address, domain):
""" Check a domain SPF record for the given IPs """
chk = spf.check(i=address, s=domain, h=domain)
if __name__ == '__main__':
print("{}:{}".format(address, chk[0]))
return chk[0]
check_spf("pyspf.clancs.co.uk", "10.10.10.10") // or any IP address,
I would expect the above to return permerror
, not unknown
... This doesn't make it unusable but is a confusing result as the RFC doesn't support an unknown result.
➜ pyspf git:(master) ✗ python3 spf.py 1.2.3.4
Traceback (most recent call last):
File "spf.py", line 95, in DNSLookup
from SPF.pydns import DNSLookup
File "/path/pyspf/SPF/pydns.py", line 28
raise spf.TempError, 'DNS: TCP Fallback error: ' + str(x)
^
SyntaxError: invalid syntax
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "spf.py", line 1886, in <module>
print(q.dns_spf(argv[0]))
File "spf.py", line 1086, in dns_spf
a = [t for t in self.dns_txt(domain) if RE_SPF.match(t)]
File "spf.py", line 1135, in dns_txt
dns_list = self.dns(domainname, rr,ignore_void=ignore_void)
File "spf.py", line 1266, in dns
for k, v in DNSLookup(name, qtype, self.strict, timeout):
File "spf.py", line 97, in DNSLookup
from SPF.dnspython import DNSLookup
File "/path/pyspf/SPF/dnspython.py", line 27
except dns.exception.DNSException,x:
^
SyntaxError: invalid syntax
Changed: raise spf.AmbiguityWarning, 'DNS: Truncated UDP Reply, SPF records should fit in a UDP packet, retrying TCP'
To
raise spf.AmbiguityWarning('DNS: Truncated UDP Reply, SPF records should fit in a UDP packet, retrying TCP')
To make it work on python3 on my setup.
(This is a proposed solution to the issue that I encountered and described in SpamExperts/OrangeAssassin#81)
While I was trying to get OrangeAssassin running on my OSX machine I noted that there were some strange namespace issues, caused by the fact that both py3dns and dnspython were installed in parallel. py3dns installs itself in top level module DNS
and dnspython in dns
, which turns out to be the same directory on a default install OSX machine due to filesystem case insensitivity.
I understand that pyspf has not seen a lot of development in the last few years, and my understanding is that dnspython is by far the more widely used library these days. It also happens to be the dns library used by other parts of OrangeAssassin.
I would be happy to create a diff that changes the dns library dependency in pyspf from py3dns to dnspython (along the lines of the code present in master) and also introduces a dependency declaration on dnspython (so that pip will handle the dependency for you) if you were interested in accepting it and releasing a new pyspf. What do you think?
RFC7208 requests that the "Received-SPF" header provides details about the reason the SPF result was what it was. See https://tools.ietf.org/html/rfc7208#section-9.1 . I don't see any API call in pyspf able to provide that kind of detail.
Since I am using pyspf from a MILTER filter and I can add an appropriate "Received-SPF" header to incoming emails, it would be very nice to be able to get the SPF tests details.
Thanks.
As noted in the readme, cvs2git chose the wrong master. While the normal checkout/commit/push/pull of git is pretty intuitive, some of the more subversive operations are not easy to understand. I'm getting some lessons from dkg in sdgathman/pymilter#27
If using pyspf on python2.7.5 (though think that all python2 is affected) the execution fails if spf.py
finds ipaddress
package and imports it. In my case ipaddress
came as dependency of salt-minion
import spf
spf.check2('XX.YY.ZZ.YY', '[email protected]', 'example.com')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/spf.py", line 297, in check2
receiver=receiver,timeout=timeout,verbose=verbose,querytime=querytime).check()
File "/usr/lib/python2.7/site-packages/spf.py", line 378, in __init__
self.set_ip(i)
File "/usr/lib/python2.7/site-packages/spf.py", line 405, in set_ip
self.ipaddr = ipaddress.ip_address(i)
File "/usr/lib/python2.7/site-packages/ipaddress.py", line 163, in ip_address
' a unicode object?' % address)
ipaddress.AddressValueError: 'XX.YY.ZZ.YY' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?
I tried to make it unicode
import spf
spf.check2(unicode('XX.YY.ZZ.YY'), '[email protected]', 'example.com')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/spf.py", line 297, in check2
receiver=receiver,timeout=timeout,verbose=verbose,querytime=querytime).check()
File "/usr/lib/python2.7/site-packages/spf.py", line 547, in check
rc = self.check1(spf, self.d, 0)
File "/usr/lib/python2.7/site-packages/spf.py", line 586, in check1
return self.check0(spf, recursion)
File "/usr/lib/python2.7/site-packages/spf.py", line 871, in check0
res, code, txt = self.check1(d,arg, recursion + 1)
File "/usr/lib/python2.7/site-packages/spf.py", line 586, in check1
return self.check0(spf, recursion)
File "/usr/lib/python2.7/site-packages/spf.py", line 895, in check0
if self.cidrmatch(self.dns_a(arg,self.A), cidrlength):
File "/usr/lib/python2.7/site-packages/spf.py", line 1348, in cidrmatch
for netwrk in [ipaddress.ip_network(ip) for ip in ipaddrs]:
File "/usr/lib/python2.7/site-packages/ipaddress.py", line 199, in ip_network
' a unicode object?' % address)
ipaddress.AddressValueError: 'XX.XX.XX.XX' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?
note that using unicode()
the reported IP address is NOT the one tested by spf.check2()
but an ipv4 address from the senders domain spf record.
After trying some I found that if I use an inport
like this
try:
import ipaddr as ipadress
in /usr/lib/python2.7/site-packages/spf.py
"solves" the issue and spf.check2()
works as expected again
If we're going to leave ipaddr (and possibly python2) behind, it might be a good idea to bump the version to 2.1. I'd suggest renaming the existing 2.1 branch for clarity, creating a new pysfp_2.0 branch as of the last release. Then master would be for new development towards the new 2.1 (I think the old one is dead at this point).
File "/opt/homebrew/lib/python3.10/site-packages/spf.py", line 595, in check
spf = self.dns_spf(self.d)
File "/opt/homebrew/lib/python3.10/site-packages/spf.py", line 1164, in dns_spf
a = [t for t in self.dns_txt(domain) if RE_SPF.match(t)]
File "/opt/homebrew/lib/python3.10/site-packages/spf.py", line 1219, in dns_txt
if isinstance(a[0],bytes):
IndexError: list index out of range
- Python 3.10.4
- dnspython 2.2.1
- pyspf 2.0.14
python spf.py 0.0.0.0 [email protected] openprocurement.io
dig openprocurement.io TXT
;; Warning: Message parser reports malformed message packet.
;; WARNING: Message has 11 extra bytes at end
When using check2 in python3 using the builtin ipaddress, with i='list', if the spf record has a network of prefix 32 or more, the code crashes.
my code:
`q = spf.query(i='list', s='[email protected]', h=None, local=None, receiver=None, timeout=spf.MAX_PER_LOOKUP_TIME,
verbose=False, querytime=20)
print(q.check())`
Traceback of the error:
Traceback (most recent call last):
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 1433, in cidrmatch
self.iplist.append(network.ip)
AttributeError: 'IPv4Network' object has no attribute 'ip'During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/shay/PycharmProjects/testing/sp.py", line 7, in
print(q.check())
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 600, in check
rc = self.check1(spf, self.d, 0)
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 639, in check1
return self.check0(spf, recursion)
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 924, in check0
res, code, txt = self.check1(d,arg, recursion + 1)
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 639, in check1
return self.check0(spf, recursion)
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 959, in check0
if self.cidrmatch([arg], cidrlength): break
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 1435, in cidrmatch
for netwrk in [ipaddress.IPNetwork(ip,strict=False) for ip in ipaddrs]:
File "/home/shay/PycharmProjects/testing/venv/lib/python3.8/site-packages/spf.py", line 1435, in
for netwrk in [ipaddress.IPNetwork(ip,strict=False) for ip in ipaddrs]:
AttributeError: module 'ipaddress' has no attribute 'IPNetwork'
This is because of line 1433 in spf.py:
self.iplist.append(network.ip)
which uses non existing ip attirbute for netwrok in the ipaddress python3 library.
proposed fix:
change to network.exploded
Hi,
When DNSLookup_dnspython()
is called with qtype
option including lowercase letter, empty list is always returned as if there's no record, regardless of the actual server response.
dnspython
accepts both upper/lowercase rdtype
and internally normalizes to uppercase.
I suppose letting DNSLookup_dnspython()
behave alike would be the best fix.
This one-line patch works well in my environment:
--- a/spf.py
+++ b/spf.py
@@ -120,6 +120,7 @@ def DNSLookup_pydns(name, qtype, strict=True, timeout=20):
def DNSLookup_dnspython(name, qtype, tcpfallback=True, timeout=30):
retVal = []
try:
+ qtype = qtype.upper()
# FIXME: how to disable TCP fallback in dnspython if not tcpfallback?
dns_version = dns.version.MAJOR+dns.version.MINOR/100
if dns_version<1.16:
Thanks.
Hello
pip install pyspf download 2.0.11 version
looking at the pypi data it seems that 2.0.12t is present 2 times, maybe that's the reason and removing the extraneous version could fix it. Thanks!
So this is based on a report I got for my SPF validator because someone had an SPF record that really did this.
Imagine the following record set:
example.com IN TXT "v=spf1 include:a.example.com include:b.example.com -all
a.example.com IN TXT "v=spf1 a -all"
b.example.com IN CNAME a.example.com
What appears to happen is that pyspf follows the CNAME and thinks there are two SPF records at a.example.com. I've set up test records in kitterman.org to demonstrate the problem:
pyspf 1.1.1.1 [email protected] test.kitterman.org
result: ('permerror', 550, 'SPF Permanent Error: Two or more type TXT spf records found.') None
Any SPF test of parallel.kitterman.org raises this error because of the CNAME following issue.
15|smtp | Error: /home/deploy/.local/lib/python3.6/site-packages/spf.py:1844: FutureWarning: split() requires a non-empty pattern match.
15|smtp | ln, reverse, delimiters = RE_ARGS.split(str)[1:4]
spf.py:1827: FutureWarning: split() requires a non-empty pattern match.
ln, reverse, delimiters = RE_ARGS.split(str)[1:4]
I would like to contribute to this project by making the code PEP8 compliant:
https://www.python.org/dev/peps/pep-0008/
I have never contributed to an open source project before so I'm hoping this is the right place to post and simple thing to start on.
hi all,
when I test this SPF "v=spf1 v=spf1 redirect=spf.protection.outlook.com -all"
with IP 40.92.1.1 in MXtool box, it return pass. However, when I do the same thing with this module, it returns Fail.
I did some extra test on this case, look like because it end with -all, so that the redirect part become useless.
Github wants markdown, PyPi wants RST. My attempt to make README.md compatible with both failed. Do we maintain both README.md and README.rst ? Is there a convert ? Is there a previewer (web or otherwise) for RST? Can I reupload versions to debug README to test.pypi.org? So many questions.
Search path contains gathman.org. Gathman.org has a wildcard that returns "v=spf1 -all" for any otherwise unspecified TXT queries.
$ python spf.py aws.telekom153.com
using pydns
None
*** install dnspython
$ python -m spf aws.telekom153.com
using dnspython
v=spf1 -all
$ python -m spf aws.telekom153.com.
using dnspython
None
$ host -t txt aws.telekom153.com
aws.telekom153.com.gathman.org descriptive text "v=spf1 -all"
$ host -t txt aws.telekom153.com.
Host aws.telekom153.com. not found: 3(NXDOMAIN)
One solution may be to always append a dot for the dnspython driver.
Hi,
I'd like to use this tool to validate an spf record. It's not really clear to me the best and easiest way to do this. It seems like a useful feature to support easily and perhaps document in the README.md.
I did look at the Kitterman code which seems to validate spf records, but didn't quite figure out their invocation yet.
Several checks fails with this traceback:
: Traceback (most recent call last):
: File "/var/policyd-spf/env/bin/policyd-spf", line 809, in <module>
: instance_dict, configData, peruser, peruserconfigData)
: File "/var/policyd-spf/env/bin/policyd-spf", line 623, in _spfcheck
: mres = mfromquery.check()
: File "/var/policyd-spf/env/lib/python3.5/site-packages/spf.py", line 591, in check
: spf = self.dns_spf(self.d)
: File "/var/policyd-spf/env/lib/python3.5/site-packages/spf.py", line 1160, in dns_spf
: a = [t for t in self.dns_txt(domain) if RE_SPF.match(t)]
: File "/var/policyd-spf/env/lib/python3.5/site-packages/spf.py", line 1210, in dns_txt
: dns_list = self.dns(domainname, rr,ignore_void=ignore_void)
: File "/var/policyd-spf/env/lib/python3.5/site-packages/spf.py", line 1354, in dns
: for k, v in DNSLookup(name, qtype, self.strict, timeout):
: File "/var/policyd-spf/env/lib/python3.5/site-packages/spf.py", line 106, in DNSLookup_pydns
: if strict > 1:
: NameError: name 'strict' is not defined
Output:
warning: command /var/policyd-spf/env/bin/policyd-spf exit status 1
warning: premature end-of-input on private/policy-spf while reading input attribute name
More info:
I'm using this module from policyd-spf filter for postfix. Current installed version is: pyspf==2.0.13
pyspf is "Python Software Foundation License". That doesn't specify the version, but I think I should copy the latest PSL.
Greetings
This is a suggestion/question, not a bug report.
I just wonder if pyspf could not add some special (optional) handling for SPF records like this:
v=spf1 +a +mx +a:vpr60.prodius.be +all
Basically it would just ignore them.
this is the current behaviour,
A formally correct result, but a bit of a let down in the spirit of the standard (it's supposed to fight spam, right ? this SPF record is one among many other like that I have seen in my logs)
So the changed output could be - with an additional command switch (corresponding to a class parameter / parameter to check/check2 global functions):
I could do a PR along these lines, but before doing it I prefer to know if it would be rejected out of hand as not standard conformant.
The default behaviour would stay of course standard conformant.
I'm well aware that this could be handled at the mail server level, but it would be easier to do it at the library level.
Last night our SPF daemon crashed a few hundred times because of the invalid ip4 entry (note the \010
) in the following record:
$ dig +short kruisecloud.com txt
"v=spf1 +a +mx +ip4:79.124.76.140 +ip4:107.175.172.4 +ip4:107.175.172.41\010 +ip4:107.174.15.33 +ip4:107.174.15.34 +ip4:62.108.40.57 +ip4:62.108.40.58 ~all"
Scott Kitterman fixed this bug two years ago, but there hasn't been a release since =)
It'd make everything a lot easier if there was a new version we could just push out, rather than trying to patch over top of the distro package.
Please fix url for check spf record.
Domain http://openspf.net don't work.
Please, add option to use custom DNS server address.
I have split horizon DNS with SMTP server in my local infrastructure, thus i get local IP for A/MX records from system resolver and i cannot use this project to verify public IP with own SPF... Using custom DNS server will allow to bypass this restriction.
Sorry this is not really an issue.. just a question.
Looking at the code that perform dns queries, am not seeing how a CNAME response is detected and handled! Could someone explain please?
Looking at the code of DNSLookup_dnspython() function, the call to dns.resolver.query() does not include raise_on_no_answer=False, without which a CNAME response would come as NoAnswer exception, no?
So how does this code catch the CNAME responses?
In addition to "v=spf1 +all", any sequence of positive matches ending in +all will always return Pass. Admins may wish to impose additional restrictive treatment on such worse than non-existent sender policies. @DanielAbrecht
under python3, with systems having both dnspython and python3 dns libraries installed, the SPF code fails due to how the resolver dependencies are walked.
The specific bug raised is that the snd.resolver.resolve(name,qtype,lifetime) args are incompatible with python3-dns, and cause pyspf to crash.
Something like this (as seen online elsewhere) seems to resolve the issue:
spf.py @@ -120,10
def DNSLookup_dnspython(name, qtype, tcpfallback=True, timeout=30):
+ import dns.version #for compatibility testing
retVal = []
try:
# FIXME: how to disable TCP fallback in dnspython if not tcpfallback?
- answers = dns.resolver.query(name, qtype, lifetime=timeout)
+ dns_version = dns.version.MAJOR+dns.version.MINOR/100
+ if dns_version<1.16:
+ answers = dns.resolver.query(name, qtype)
+ elif dns_version<2.0:
+ answers = dns.resolver.query(name, qtype, lifetime=timeout)
+ else: # >=2.0
+ answers = dns.resolver.resolve(name, qtype, lifetime=timeout)
for rdata in answers:
if qtype == 'A' or qtype == 'AAAA':
retVal.append(((name, qtype), rdata.address))
(this is working on my local system, though i have some other local patches which may or may not intertwine with this, but not listed here)
When the spf record includes methods like a or mx and I do a query against an ipv6 address and the a or mx records do not include ipv6 adresses this the void counter is increased. I consider this a bug and the expected behavior would be to not increase the void counter
Example SPF:
"v=spf1 a mx include:spf.example.com ~all"
Example query:
>>> import spf
>>> spf.check2(i='2a01:aa:bb:cc::2',s='j,[email protected]',h='mx1.example.com')
('permerror', 'SPF Permanent Error: Void lookup limit of 2 exceeded')
I totally agree that methods a or mx should not be used, but on received mails I do not have control over dns
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.