Problems with ipv6 queries and void lookups about pyspf HOT 8 CLOSED

Thanks. Misunderstanding totally on my side.

I wrote an outgoing mail filter in python which tries to predict if a mail is deliverable. Based on the result routing decisions are made.
As I got discrepancies in the results and all those popular online checkers got the same result I took this as the democratic truth.
A little bit frustrating for the prediction part of my filter.

Sorry for wasting your time on this.

from pyspf.

The point of the VOID lookup limit is to limit DoS attacks. There is no way to know whether there are A records when doing an AAAA lookup - except by doing yet another lookup! Sometimes, servers provide additional records that it thinks might be useful. Those are cached by pyspf, and it might be reasonable to check the cache - but that would be inconsistent, and an optional feature.

from pyspf.

Note that you can easily just double MAX_VOID_LOOKUPS to get the equivalent of what you suggest. If your problem is badly done policy from a business partner, just increase the limit until their broken policy passes.

from pyspf.

I am still confident that this implementation handles it wrong. At least it handles it different than the online checks from spf-record.com and mxtoolbox.com and also different than the libspf2 implementation and the spf-ruby implementation.

I set up a dns entry for testing (spf3.gschistigschasti.at) and if you test this against an ipv6 address (eg 2a01:4f8:c012:c36b::1) all tools mentioned above say "pass", pyspf says "permerror".

# /usr/bin/spfquery.libspf2 --ip=2a01:4f8:c012:c36b::1 --sender=spf3.gschistigschasti.at
pass

spfquery: domain of spf3.gschistigschasti.at designates 2a01:4f8:c012:c36b::1 as permitted sender
Received-SPF: pass (spfquery: domain of spf3.gschistigschasti.at designates 2a01:4f8:c012:c36b::1 as permitted sender) client-ip=2a01:4f8:c012:c36b::1; [email protected];

# /usr/bin/spfquery.pyspf --ip=2a01:4f8:c012:c36b::1 --sender=spf3.gschistigschasti.at
permerror
SPF Permanent Error: Void lookup limit of 2 exceeded
spfquery: permanent error in processing domain of spf3.gschistigschasti.at: Void lookup limit of 2 exceeded
Received-SPF: PermError (spfquery: permanent error in processing domain of spf3.gschistigschasti.at: Void lookup limit of 2 exceeded) client-ip=2a01:4f8:c012:c36b::1; envelope-from=spf3.gschistigschasti.at; receiver=spfquery; identity=mailfrom

[23] test(main)> require 'spf'
spf_server = SPF::Server.new
request = SPF::Request.new(
  versions:      [1, 2],             # optional
  scope:         'mfrom',            # or 'helo', 'pra'
  identity:      '[email protected]',
  ip_address:    '2a01:4f8:c012:c36b::1',
  helo_identity: 'spf3.gschistigschasti.at'   # optional
)
result = spf_server.process(request)
puts result

pass (SPF::Result::Pass)

https://mxtoolbox.com/SuperTool.aspx?action=spf%3aspf3.gschistigschasti.at%3a2a01%3a4f8%3ac012%3ac36b%3a%3a1&run=toolpage

https://www.spf-record.com/spf-lookup/spf3.gschistigschasti.at?ip=2a01:4f8:c012:c36b::1&opt_out=on

from pyspf.

The standard is the RFC, not other implementations.

RFC 7208 4.6.4 ends with:

it is useful to limit the number of "terms" for which DNS queries return
either a positive answer (RCODE 0) with an answer count of 0, or a
"Name Error" (RCODE 3) answer.
These are sometimes collectively
referred to as "void lookups". SPF implementations SHOULD limit
"void lookups" to two. An implementation MAY choose to make such a
limit configurable. In this case, a default of two is RECOMMENDED.
Exceeding the limit produces a "permerror" result.

In this implementation, the limit defaults to two as recommended, and is configurable.

If you are receiving mail and get this error, the sender's policy is broken. There is no defensible reason for such a policy. If, for business reasons, you need to accept email from such broken senders anyway (I am sometimes in that position), then you can increase the limit. I normally flag the broken domain to accept "Permerror" (while sending a diagnostic mailer response warning).

You might want a little database of domains which sets the increased limit only for certain domains. I use the sendmail "access" file for these purposes.

from pyspf.

No doubt, the reference is the RFC and not other implementations.

If all other implementations behave different than yours, this could be an indicator for a bug or a misinterpretation of the RFC.
And I would say it is the second one.

RFC 7208 Section 5 Mechanism Definitions :
When any mechanism fetches host addresses to compare with , when
is an IPv4, "A" records are fetched; when is an IPv6
address, "AAAA" records are fetched.
...
If the server returns "Name Error" (RCODE 3), then evaluation of the
mechanism continues as if the server returned no error (RCODE 0) and
zero answer records.

Mechanisms are
a
mx
ptr (do not use)
ip4
ip6
exists

RCODEs are defined in RFC 1035 Section 4.1.1

So if there is an AAAA lookup for mechanism a and it returns RCODE 3, it should be treated as RCODE 0 "No error condition".

from pyspf.

There is no error condition, but RCODE 3 or 0 result records is the literal definition of a "void lookup".
The other implementations are mostly compliant as limiting void lookups is a SHOULD, not a MUST.

Hopefully, the other implementations allow you to configure some limit. If not, they are deficient. As mentioned already, it is easy to configure pyspf to raise the limit to whatever you deem sufficient. The default is the RFC recommended limit of two.

As an editor of the original RFC, I remember the heated debate over DoS protection vs certain corporate adopters (who love huge cumbersome policies that are almost a DoS in themselves). There was one guy who was like "SPF DoS will be the end of the internet as we know it". And another guy who was like "who cares about corporate adopters, they can go rot in the bad place". The debate is archived.

from pyspf.

While, it is hard to predict which receivers will limit void lookups, any sender policy with more than 2 void lookups is very badly written, and is less likely to be delivered in general.

Note there are other MUSTard limits for DNS lookups that are routinely violated for badly written policies. Note section 4.6.4 DNS Lookup Limits in RFC 7208. You might get better predictive results with those. In fact, we used to have a "hall of shame" list for policies that violate that MUSTard. Any that you find in your research could go in a hall of shame. pyspf also enforces those limits by default, but allows the caller to override for business reasons. Again, I have to allow really stupid policies from certain senders only because they are my client's business partners.

from pyspf.