SCRAM-SHA-1(-PLUS) + SCRAM-SHA-256(-PLUS) + SCRAM-SHA-512(-PLUS) + SCRAM-SHA3-512(-PLUS) supports about msmtp HOT 40 CLOSED

Simon Josefsson convinced me that msmtp should support SCRAM-SHA-256. It now does.

However, I am still convinced that the SCRAM-* methods are just an unnecessary complexity and offer no real benefit.

from msmtp.

I think SCRAM-SHA512, SCRAM-SHA3 and tls-server-end-point are all harmful to the overall ecosystem security of SCRAM. They don't offer anything convincing compared to what we already have, and adding more SCRAM-parametrization adds complexity and maintenance costs, which lead to worse security.

from msmtp.

I have no plans to implement any of this, for the following reason:

Nowadays every SMTP session must be protected with TLS. When TLS is active, there is no reason to use any SCRAM* method. (RFC 5802 says "there are some significant security concerns" with transmitting passwords over TLS-secured connections, but does not list any of them, and I have not seen a compelling argument for that statement yet.)

I therefore consider all methods except PLAIN, OAUTHBEARER, EXTERNAL (for TLS client certificates), LOGIN (strictly only for compatibility with Microsoft crap), and maybe GSSAPI (only in special situations) to be useless today.

Some of the deprecated methods like CRAM-MD5 etc are only left for backward compatibility and will probably be removed in the next larger update (1.10 or 2.0), so their presence is no reason to add SCRAM* methods.

So unless someone can convince me that any SCRAM* method over TLS offers a real (and not just theoretical) advantage over the simpler methods listed above, I will not add support for them.

from msmtp.

After receiving feedback on an article I wrote about my doubts about SCRAM, I changed my position. SCRAM is indeed useful. See this update for the reasons that changed my mind.

from msmtp.

@gowthamgts: Strange, your messages are not clear:

• duesee/imap-next#96 (comment)
• #36 (comment)
• https://news.ycombinator.com/item?id=39868682

You can follow my announcements here:

• https://mastodon.social/@neustradamus
• https://twitter.com/neustradamus
• https://bsky.app/profile/neustradamus.bsky.social
• https://news.ycombinator.com/user?id=neustradamus
• https://www.reddit.com/user/Neustradamus/
• ...

The good point, people speak about SCRAM "Salted Challenge Response Authentication Mechanism" security ;)

• scram-sasl/info#1

Badly, some people or projects like only old unsecure mechanisms, some would like security improvements.

from msmtp.

@Neustradamus Don't @ me.

from msmtp.

@LeoniePhiline: Sorry, thanks for your message.
It is important to understand all and not a little part :)
I have edited my messages.

from msmtp.

SCRAM support is now complete from my point of view, at least for now. Let's see if somebody actually uses it and provides useful feedback :)

from msmtp.

@marlam: Thanks for your answer :)

@jas4711: Thanks a lot for your SCRAM-SHA-256 improvements in msmtp!

Can you look for mpop too, to have same into mpop and msmtp?

• https://github.com/marlam/mpop-mirror/search?q=scram-sha-1
• https://github.com/marlam/mpop-mirror/search?q=scram-sha-256

@marlam has not created a mimap yet, maybe soon?

@marlam: SCRAM-SHA-* is the evolution of DIGEST-MD5, a real security, please read the RFC6331.

But the PR is not merged and it is closed:

• #40

And I do not find SCRAM-SHA-256:

• https://github.com/marlam/msmtp-mirror/search?q=scram-sha-1
• https://github.com/marlam/msmtp-mirror/search?q=scram-sha-256

from msmtp.

Hi!

Can you look for mpop too, to have same into mpop and msmtp?
[...]
But the PR is not merged and it is closed:
* #40.
And I do not find SCRAM-SHA-256:
* https://github.com/marlam/msmtp-mirror/search?q=scram-sha-256

Hi, I applied Simon's patch to both msmtp and mpop already. The reason you did not see it yet on Github is that the main repository is synced to the Github mirror only once per day. It should be there by now.

@MarlaN has not created a mimap yet, maybe soon?

What's a mimap?

@marlam: SCRAM-SHA-* is the evolution of DIGEST-MD5, a real security, please read the RFC6331.

I fully understand hat DIGEST-MD5 is terrible and obsolete; that's not my point. My point is that I don't see why anyone would need SCRAM*. Everybody uses TLS for everything today (or should be), and if that is the case, I see no practical benefit of SCRAM* vs good ol' PLAIN authentication:

• Since you can assume TLS, you don't need channel binding and server authentication. This is just unnecessary complexity.

• I don't see the use for authorization ids in the context of single-purpose protocols such as POP3, SMTP, IMAP, HTTP and so on (for which the SCRAM-* methods are intended). Again, unnecessary complexity.

• I don't see the point of the stringprep requirements. The authentication method should just treat username and password as byte sequences, much like the Linux kernel treats file names. It is clear that they should in fact be valid UTF8 strings, but that can be handled purely on the user side; I don't see why the authentication method should need to deal with that. PLAIN works just fine with UTF8 user names and passwords precisely because it does not care. Unnecessary complexity again!

• If the client stores the password encrypted and the server stores just a password hash, that's fine. I don't see why more "protection" of a password would be needed when TLS is active.

I may be wrong about these points, and if so, I would welcome to be corrected here. If you disagree, please give more information or a link.

from msmtp.

@marlam: Thanks for your reply!
Sorry for "N", I have edited.

I have not seen the @jas4711 PR here for mpop: https://github.com/marlam/mpop-mirror/pulls?q=is%3Apr
It was directly in the main original source ^^

mimap would be an IMAP client, like mpop is a POP3 client ;)
It will be really nice...

With SCRAM, the password on the server is not in PLAIN, better security, confidentiality.

from msmtp.

mimap would be an IMAP client, like mpop is a POP3 client ;)
It will be really nice...

Ah, ok. I'd like to do that, see here for a first thought:
marlam/mpop#4

With SCRAM, the password on the server is not in PLAIN, better security, confidentiality.

Sorry, not sufficient:

• With PLAIN, the password also does not need to be in clear text on the server, a hash is sufficient.
• The claim "better security, confidentiality" needs to be substantiated. As in "citation needed" (and not just a link to an RFC defining the method!).

To put it in other words, I'd like an answer to the question "In what way is SCRAM-SHA-256 better than PLAIN for password-based authentication in the context of a TLS-secured single-purpose protocol like IMAP or SMTP?" My current answer would be: in no way that is of practical relevance.

from msmtp.

No with SCRAM the password is not saved into a database in clear.

Time to look for IMAP, you can see the last Internet-Draft:

• https://tools.ietf.org/html/draft-ietf-extra-imap4rev2

I have added mpop and msmtp in the list, but you have not done releases yet.

Note:

• GNU SASL (gsasl) supports SCRAM-SHA-256(-PLUS) since 1.9.1: http://git.savannah.gnu.org/gitweb/?p=gsasl.git;a=blob;f=NEWS;hb=HEAD
• Dovecot supports SCRAM-SHA-256(-PLUS) since 2.3.10: https://doc.dovecot.org/configuration_manual/authentication/authentication_mechanisms/
• Cyrus SASL/IMAP supports SCRAM-SHA-256(-PLUS) and more since 2.1.27: https://www.cyrusimap.org/sasl/sasl/authentication_mechanisms.html
• List here: scram-sasl/info#1

from msmtp.

No with SCRAM the password is not saved into a database in clear.

And it is not with PLAIN either!

* GNU SASL (gsasl) supports SCRAM-SHA-256(-PLUS) since 1.9.1: http://git.savannah.gnu.org/gitweb/?p=gsasl.git;a=blob;f=NEWS;hb=HEAD

* Dovecot supports SCRAM-SHA-256(-PLUS) since 2.3.10: https://doc.dovecot.org/configuration_manual/authentication/authentication_mechanisms/

* Cyrus SASL/IMAP supports SCRAM-SHA-256(-PLUS) and more since 2.1.27: https://www.cyrusimap.org/sasl/sasl/authentication_mechanisms.html

I know that these support SCRAM-SHA-256, but that does not answer the question "why would anyone want to use it instead of the much simpler PLAIN-over-TLS".

This is going nowhere. I must conclude that there is indeed no benefit to SCRAM.

from msmtp.

Thanks for update! Big of you to reconsider. To be honest, I'm somewhere between your extreme positions. I do believe PLAIN is not ideal and that we can do better, and SCRAM is not bad. The complexity introduced with SCRAM over PLAIN is only worth it if it actually solves the problems we want to address: however, SCRAM does not REQUIRE servers to store passwords salted/hashed, and quite often they are stored in cleartext, which is WORSE than with PLAIN where deployment actually have started to have salted/hashed passwords fairly often. Secondly, the tls-unique channel binding is broken and does not offer the security guarantees that people thought. The idea of channel bindings is still good, IMHO, but there is two problems in SCRAM with it: 1) SCRAM chose to demand a broken channel binding, and updating it is not easy today, and 2) it was optional, so in practice its use won't be enforced. Finally, the unfortunate choice of having SCRAM be a family of mechanisms leads to plurality of hash methods being suggested which is detrimental to deployment.

from msmtp.

Thanks for your changes about SCRAM-SHA-*.

I think the order is not good here:

• https://github.com/marlam/msmtp-mirror/blob/master/src/smtp.h#L77

And CRAM-MD5 and DIGEST-MD5 are less good than PLAIN.

20 November 2008: CRAM-MD5 to Historic:

• https://tools.ietf.org/html/draft-ietf-sasl-crammd5-to-historic-00

29 June 2017: CRAM-MD5 to Historic:

• https://tools.ietf.org/html/draft-zeilenga-luis140219-crammd5-to-historic-00

July 2011: RFC6331: Moving DIGEST-MD5 to Historic:

• https://tools.ietf.org/html/rfc6331

from msmtp.

Closing the circle here, I blogged about SCRAM concerns here:
https://blog.josefsson.org/2021/06/08/whats-wrong-with-scram/

from msmtp.

Thank you for this article Simon!

from msmtp.

@marlam: We can thank @jas4711 who has worked on the support of the new security RFC in GNU SASL:

• RFC9266: Channel Bindings for TLS 1.3: https://tools.ietf.org/html/rfc9266

Little details, to know easily:

• tls-unique for TLS =< 1.2
• tls-server-end-point
• tls-exporter for TLS = 1.3

If you can test before merging and add support in msmtp?

• https://lists.gnu.org/archive/html/help-gsasl/2022-07/msg00003.html
• https://lists.gnu.org/archive/html/help-gsasl/2022-07/msg00004.html

Thanks in advance.

from msmtp.

@jas4711, @marlam: I think that you have seen the jabber.ru MITM:

• https://notes.valdikss.org.ru/jabber.ru-mitm/
• https://snikket.org/blog/on-the-jabber-ru-mitm/
• https://www.devever.net/~hl/xmpp-incident
• https://blog.jmp.chat/b/certwatch

from msmtp.

I am currently adding SCRAM-*-PLUS to msmtp (and later to mpop), using gnutls_session_channel_binding() to get the channel binding information, type EXPORTER for TLS 1.3 and UNIQUE for TLS <= 1.2.

I have a question regarding the AUTH capabilities advertised in a server greeting. Given that the SCRAM-* methods are able to protect against certain MITM attacks, Is it not a problem that a MITM could remove all the SCRAM-* methods from the server greeting, thereby making the client fall back to PLAIN or similar?

In other words, does a user have to enforce using a SCRAM-* method to benefit from its protections?

from msmtp.

It was some time since I was well versed on the SCRAM downgrade protection complexity, but I believe that generally you are right that an adversary is able to MITM the TLS connection and make data modifications, AND if PLAIN is acceptable to the user/application, there is nothing SCRAM can do to protect against this.

What SCRAM-PLUS can offer is the ability to DETECT (and abort) passive TLS MITM's, since the channel binding will fail. If both client and server supports SCRAM+SCRAM-PLUS, then even SCRAM will allow detection (and abort) of active TLS attacks.

So ideally you would want to enforce at least SCRAM to any server that at some point offers SCRAM, and ideally anything less than SCRAM should not ever be acceptable to users/applications by default. I do understand that the e-mail world is far behind the HTTPS world here though, so this is usually not that feasible.

from msmtp.

To compare with the jabber.ru issue: if clients (and servers) had supported SCRAM-PLUS, then you would have gotten connection failures and notice the passive TLS MITM before the certificate expired. The attacker could turn active and remove SCRAM+SCRAM-PLUS but this would be more costly than a pure passive attack.

from msmtp.

Thank you for this information, it helps.

So in a first step, to not break existing setups, users of msmtp and mpop will have to manually select a SCRAM-* method to prevent these kinds of downgrade attacks.

In a second step, I'd like to add a simpler flag that enforces SCRAM-*[-PLUS] authentication without requiring the user to select one specific method manually, and recommend that flag prominently in all documentation and examples.

In the future, when/if SCRAM-* is widely supported in servers, this flag should then become the default, but that would break setups where the server does not support SCRAM-*...

from msmtp.

@marlam: Thanks for your comments and thanks to @jas4711 for comments too :)

I take the opportunity to say that I have requested to @jas4711 to add "tls-server-end-point" support in GSASL too:

• https://gitlab.com/gsasl/gsasl/-/issues/13

In the specified ticket, I have added links to XEPs about SCRAM.

from msmtp.

I would ignore tls-server-end-point and focus on mandatory SCRAM-PLUS with tls-exporter. If there is ever a serious significant use-case for non-plus SCRAM or tls-server-end-point, it could be added later, but I believe adding complexity to support them is counter-productive to reach a SCRAM-PLUS with tls-exporter world.

from msmtp.

@jas4711, it is in XEPs:

• XEP-0388: Extensible SASL Profile: https://xmpp.org/extensions/xep-0388.html
• XEP-0440: SASL Channel-Binding Type Capability: https://xmpp.org/extensions/xep-0440.html
• XEP-0474: SASL SCRAM Downgrade Protection: https://xmpp.org/extensions/xep-0474.html
• XEP-0480: SASL Upgrade Tasks: https://xmpp.org/extensions/xep-0480.html

from msmtp.

@marlam: @mbhangui has added supports in indimail-mta, he has done talks with @jas4711 too:

• https://github.com/search?q=owner%3Ambhangui+scram&type=issues
• https://github.com/search?q=owner%3Ambhangui+scram&type=pullrequests

Manvendra has helped @schengawegga for Net_SMTP too.

from msmtp.

from msmtp.

Support for SCRAM-SHA-256-PLUS and SCRAM-SHA-1-PLUS is now in the git repository.

This is tested so far against Exim via TLS 1.2 and TLS 1.3, but this basically tests a GnuTLS/GSASL client against a GnuTLS/GSASL server, so any additional testing is very welcome.

A future patch will likely introduce auth=scram-plus and auth=scram support so that the user at least does not have to care about the hash-du-jour to get the security benefits of the scram methods.

from msmtp.

The same code is now also in mpop for POP3. If you can test this, please do.

from msmtp.

Support for SCRAM-SHA-256-PLUS and SCRAM-SHA-1-PLUS is now in the git repository.

This is tested so far against Exim via TLS 1.2 and TLS 1.3, but this basically tests a GnuTLS/GSASL client against a GnuTLS/GSASL server, so any additional testing is very welcome.

Yay! Dovecot support SCRAM (but not SCRAM-PLUS, I think) and is fairly easy to interop-test against: https://gitlab.com/gsasl/gsasl/-/blob/b21aa50acebcf1350832aba4af8fc7f84bb68f3b/.gitlab-ci.yml#L651

A future patch will likely introduce auth=scram-plus and auth=scram support so that the user at least does not have to care about the hash-du-jour to get the security benefits of the scram methods.

Makes sense, although I think it is one of the disadvantages with SCRAM: it leads to UX complexity like this, instead of just providing secure defaults and reduce UX complexity. Web browser security has solved that, but it seems SMTP/IMAP/POP3/XMPP is stuck in older design patterns. No fault of msmtp/mpop, of course, but just a general observation.

/Simon

from msmtp.

@marlam: Thanks for your improvements!

@jas4711: @stephanbosch has added SCRAM in Dovecot and he has commented here:

• python/cpython#95341 (comment)

At the same time, you can see tls-server-end-point (in more tls-unique and tls-exporter) in xmpp lib used ejabberd...

• https://github.com/search?q=org%3Aprocessone+tls-server-end-point&type=code

from msmtp.

I have more questions about SCRAM...

I was told that if a server stores the salted and hashed password, then stealing that allows impersonating the client, so it's no better than storing clear text passwords.

Then what is the server supposed to store instead?

In GSASL, what is SERVERKEY and STOREDKEY, and how do they relate to SCRAM_SALTED_PASSWORD? All of those seem to be derived from PASSWORD using SCRAM_ITER and SCRAM_SALT.

from msmtp.

I hope that the manual covers this, see: https://www.gnu.org/software/gsasl/manual/html_node/SCRAM.html#SCRAM

Yes it is preferential to store SERVERKEY/STOREDKEY on the server, however I believe this advantage is exagerated: if you steal them, you only need one active MITM-attack to be able to compute the password-equivalents anyway. I think this is another example of were the SCRAM design introduces complexity that have questionable security advantage in the real world. The complexity required to support SERVERKEY/STOREDKEY, and security concerns as a consequence, may be larger than the slight theoretical gain that is won.

But does msmtp/mpop implement a server?

from msmtp.

Msmtp/mpop both come with tiny servers, but these are intended only as a local bridge between a mail client such as Thunderbird and msmtp/mpop, so SCRAM is not relevant for them.

I asked because as far as I can see, both Exim and Dovecot can only work with either plaintext or the hashed/salted password, not with SERVERKEY/STOREDKEY. So it seems offering SCRAM on a mail server today involves some tradeoffs...

from msmtp.

Msmtp/mpop both come with tiny servers, but these are intended only as a local bridge between a mail client such as Thunderbird and msmtp/mpop, so SCRAM is not relevant for them.

I asked because as far as I can see, both Exim and Dovecot can only work with either plaintext or the hashed/salted password, not with SERVERKEY/STOREDKEY. So it seems offering SCRAM on a mail server today involves some tradeoffs...

I see. I think using hashed/salted passwords on the server is acceptable today -- if you are worried about that, the theoretical gain offered by moving to SERVERKEY/STOREDKEY is small, and you are better of moving to Kerberos, SRP or the forthcoming OPAQUE mechanism. Doing one failed authentication attempted using stolen SERVEKEY/STOREDKEY as an attacker is a cheap way to get a password-equivalent.

I think SERVERKEY/STOREDKEY is mostly a complexity distraction that causes people to not implement SCRAM because they think they need to implement SERVERKEY/STOREDKEY too. So they stick with CRAM-MD5 or TLS+PLAIN.

/Simon

from msmtp.

careful here - https://news.ycombinator.com/item?id=39868682. the author has been pushing for lot of sec related changes in lot of repositories.

from msmtp.

@gowthamgts: Do you attack me because I have requested the XZ update?
I am not linked to the XZ project.

from msmtp.

@marlam: In part yes, not full yet. It is good for SCRAM-SHA-1(-PLUS) + SCRAM-SHA-256(-PLUS) but not for SCRAM-SHA-512(-PLUS) + SCRAM-SHA3-512(-PLUS), I can create another ticket for it.

I will see with @jas4711 for "tls-server-end-point" in gsasl, the missing part.

from msmtp.