bebleo / smtpdfix Goto Github PK

Right now whether a session is authenticated is recorded in the controller, but the session object would be a more natural fit for this.

Example:

def handle_MAIL(self, server, session, envelope, args):
    if not session.authenticated:
        server.push(AUTH_REQUIRED)
        return
    ....
    # rest of the method to process the mail command

(This may be an upstream bug in aiosmtpd, but you'd be better at deducing whether that's the case than I am.)

Running the following test under Python 3.11.0 with smtpdfix 0.4.0 and aiosmtpd 1.4.2:

from email.message import EmailMessage
import smtplib

def test_smtpdfix_bug(smtpd):
    msg = EmailMessage()
    msg["Subject"] = "Hello"
    msg["To"] = "you@there"
    msg["From"] = "me@here"
    msg.set_content("Hello, world!\n")

    smtpd.config.use_starttls = True

    with smtplib.SMTP(smtpd.hostname, smtpd.port) as client:
        client.starttls()
        client.send_message(msg)

fails with the following error:

============================= test session starts ==============================
platform linux -- Python 3.11.0, pytest-7.2.0, pluggy-1.0.0
rootdir: /docked
plugins: smtpdfix-0.4.0
collected 1 item

test_smtpdfix.py F                                                       [100%]

=================================== FAILURES ===================================
______________________________ test_smtpdfix_bug _______________________________

smtpd = <smtpdfix.controller.AuthController object at 0x7f9619aed3d0>

    def test_smtpdfix_bug(smtpd):
        msg = EmailMessage()
        msg["Subject"] = "Hello"
        msg["To"] = "you@there"
        msg["From"] = "me@here"
        msg.set_content("Hello, world!\n")
    
        smtpd.config.use_starttls = True
    
        with smtplib.SMTP(smtpd.hostname, smtpd.port) as client:
>           client.starttls()

test_smtpdfix.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.11/smtplib.py:790: in starttls
    self.sock = context.wrap_socket(self.sock,
/usr/local/lib/python3.11/ssl.py:517: in wrap_socket
    return self.sslsocket_class._create(
/usr/local/lib/python3.11/ssl.py:1075: in _create
    self.do_handshake()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ssl.SSLSocket [closed] fd=-1, family=2, type=1, proto=6>, block = False

    @_sslcopydoc
    def do_handshake(self, block=False):
        self._check_connected()
        timeout = self.gettimeout()
        try:
            if timeout == 0.0 and block:
                self.settimeout(None)
>           self._sslobj.do_handshake()
E           ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:992)

/usr/local/lib/python3.11/ssl.py:1346: SSLEOFError
------------------------------ Captured log call -------------------------------
ERROR    asyncio:base_events.py:1757 Fatal error: protocol.data_received() call failed.
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f96193c5450>
transport: <_SelectorSocketTransport fd=16 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 1001, in _read_ready__data_received
    self._protocol.data_received(data)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'SSLProtocol' object has no attribute 'data_received'
=========================== short test summary info ============================
FAILED test_smtpdfix.py::test_smtpdfix_bug - ssl.SSLEOFError: EOF occurred in...
============================== 1 failed in 0.34s ===============================

The test works fine if StartTLS is not used (including if normal SSL is used) and under earlier versions of Python, however.

When the server receives email messages it should set a received header similar to the following:

Received:     from maroon.pobox.com (maroon.pobox.com [208.72.237.40]) by mailstore.pobox.com 
        (Postfix) with ESMTP id 847989746 for <address>; Wed, 15 Jun 2011 10:42:09 -0400 (EDT)

See this link for some examples.

Setting the environment variables after tests are initialized is ineffective, This means that using both monkeypatch and mock.patch.dict fails.

Examples of tests that fail:

# Using pytest.monkeypatch
def test_smtpd_monkeypatch(monkeypatch, smtpd):
    monkeypatch.setenv("SMTPD_PORT", "5025")
    assert smtpd.port == 5025  # Fails 8025 == 5025

# Using mock.patch.dict
@mock.patch.dict(os.environ, {"SMTPD_PORT": "5025"})
def test_smtpd_mock_patch(smtpd):
    assert smtpd.port == 5025  # Fails 8025 == 5025

The following test succeeds:

@pytest.fixture
def mock_port(monkeypatch):
    monkeypatch.setenv("SMTPD_PORT", "5025")

def test_smtpd_fixture(mock_port, smtpd):
    # The order of the arguments matters and the monkey patch fixture
    # must come first.
    assert smtpd.port == 5025  # Passes

You have some properties like this:

    @property
    def SMTPD_USE_STARTTLS(self):
        return strtobool(os.getenv("SMTPD_USE_STARTTLS", False))

where strtobool starts with

def strtobool (val):
    """Convert a string representation of truth to true (1) or false (0).

    True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values
    are 'n', 'no', 'f', 'false', 'off', and '0'.  Raises ValueError if
    'val' is anything else.
    """
    val = val.lower()

Because the default value is a boolean, not a string, val.lower() results in an error.

This can be fixed by changing False to "False" or by testing if type(val) is bool: return val before the val.lower() call.

Describe the bug
When I install this package using pip on a venv environment (i.e. virtual environment created with python -m venv .venv),
I get a folder called tests/ folder directly in my site-packages/ with files including test-smtpdfix and test-smtp and it clashes with my own tests package. I delete that folder and this fixes the issue.
To Reproduce

• python -m venv .venv
• python -m pip install smtpdfix
• look in .venv/lib/python<version>/site-packages and there is a tests folder.

Expected behavior
Are you adding tests in a root tests namespace on purpose?

Environment (please complete the following information):

• OS: MacOS 12.6.3
• Version: Python 3.10.0

I'm attempting to use smtpdfix as a pytest fixture, but can't get it to initialise correctly. When I attempt to do so on my macbook, the certificates fail to create because the socket.gethostbyname(hostname) call fails:

        hostname = socket.gethostname()
>       ip = socket.gethostbyname(hostname)
E       socket.gaierror: [Errno 8] nodename nor servname provided, or not known

I can easily get the same error by running the same code in a Python console:

$ python
Python 3.8.5 (default, Oct 30 2020, 11:19:12)
[Clang 12.0.0 (clang-1200.0.32.21)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.gethostbyname(socket.gethostname())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
socket.gaierror: [Errno 8] nodename nor servname provided, or not known
>>> socket.gethostname()
'Alexanders-MacBook-Pro'

Dear James,

thank you very much for the prompt improvement and the subsequent release. We can confirm the library is working without errors now, even on a misconfigured system.

However, we observed that a test case usually taking 0.43s on a properly configured system now only completes in 20.50s. While it is running, several stalls occur, indicating that the lack of the resolvable host name has some further drawbacks down the line while running the SMTP conversation.

So, with this insight, correct local hostname -> IP address resolution should be mandatory before doing any SMTP communication at all, either with a real server, or with this library. It may make sense to mention that detail within the README or another FAQ/troubleshooting section of the documentation, and maybe even drop a word in the corresponding error log message about it.

In any case, we wanted to share our observations with you and the community.

Thank you again and with kind regards,
Andreas.

Originally posted by @amotl in #195 (comment)

Right now we expose the generate_certs method as public from aiosmtpd.certs. Given the work involved in maintaining this and that it's outside the scope of the project, it will be removed as of version 0.4.0

A PendingDeprecationWarning will be added to the code.

The vast majority of pytest plugins use the "pytest-" prefix in their project name. This will make the purpose of the project more self-evident. The pytest documentation also lists plugins automatically based on this prefix: https://docs.pytest.org/en/latest/reference/plugin_list.html

Currently we use dependabot to verify that smtpdfix works with the latest releases of any given solution. This is being accomplished in by freezing our pip requirements to /requirements.txt which leaves a redundant file in the root of the project.

All dependencies for install are defined in /setup.cfg and this should be the only source for setup.

based on this the complete list of the latest versions, /requirements.txt, should be moved to the /.github folder so that it no longer resides in the project root. At the same time, tests should be added to check for the minimum versions of dependencies against which smtpdfix is known to work.

When the OSSF Scorecard actions runs on a schedule it fails.

Rather than trying to fix here we will remove the schedule and instead set it to run on the pull_request event. Once fixed in ossf/scorecard-action we'll reinstate the scheduled runs.

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

When authenticating using the PLAIN and LOGIN authentication mechanisms over SSL/TLS authentication fails. It will succeed when running the server is using STARTTLS. The following test will fail:

def test_AUTH_LOGIN_success(mock_smtpd_use_starttls, smtpd, user):
    with SMTP(smtpd.hostname, smtpd.port) as client:
        client.starttls()
        code, resp = client.docmd('AUTH', f'LOGIN {encode(user.username)}')
        assert code == 334
        assert resp == bytes(encode('Password'), 'ascii')
        code, resp = client.docmd(f'{encode(user.password)}')
        assert code == 235

The following, however, fails:

def test_AUTH_LOGIN_failure(mock_smtpd_use_ssl, smtpd, user):
    with SMTP_SSL(smtpd.hostname, smtpd.port) as client:
        code, resp = client.docmd('AUTH', f'LOGIN {encode(user.username)}')
        assert code == 334
        assert resp == bytes(encode('Password'), 'ascii')
        code, resp = client.docmd(f'{encode(user.password)}')
        assert code == 235  # Fails here because the code is "538 Authentication required."

Currently types are defined in a separate python file, typing.py. In reality this obscures that these types are used by only Class and should be defined in place.

To resolve this and to reduce any redundancy the contents of typing.py will be removed and transferred to the appropriate places in the code.

Additional context
Consideration should be made whether these types should be exposed through the public API in __all__.

When using opportunistic TLS under python 3.10 the connection fails with the error ssl.SSLError: Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801).

Minimal example to reproduce:

def test_init_ssl(smtpd, msg):
    smtpd.config.use_ssl = True
    with SMTP_SSL(smtpd.hostname, smtpd.port) as client:
        client.ehlo()

    assert smtpd

Install fails when using pypy3.9 due to problems with dependencies in cryptography v.36 which were preventing the wheels from being built. The problem has now been fixed with the release of pyo3 0.15.2 and we may be able to install successfully once a new version of cryptography is released.

NB: PyPy support is considered a nice to have, but not officially supported at this time.

This is a reminder checklist to ensure that everything has been done before release 0.4.0 is published:

• Bump versions in \requirements\latest to latest versions.
• __version__ updated to 0.4.0 in \smtpdfix\__init__.py
• Release date set in \docs\CHANGELOG.md

Add the OSSF Scorecard Action.

Currently configuration is handled through environment variables. In some cases these can be changed after the fixture has been initialized, but before it is called. In others this change is ignored.

The consistent behaviour should be that:

1. When initialized the existing environment variables are read
2. The configuration can then be modified. When doing so the fixture will reset as necessary to take advantage of the new configuration.

See also #15, #48, #50

Currently there's no support for terminating the process. This could be added using, for example, signals.SIGTERM.

Because of the asyncio implementation this won't work on windows, but for our use case this would be an acceptable solution.

Support for Python 3.6 ended on December 23, 2021. Drop support with version 0.4.0.

• Remove python 3.6 from CI and tox
• Remove the workarounds for asyncio features added in python 3.7

In a pypy:3 Docker container using PyPy3 7.3.3, pytest 6.2.2, and smtpdfix 0.2.9, the following code fails with a TypeError:

from email.message import EmailMessage
import smtplib
import pytest

@pytest.fixture
def mock_smtpd_use_ssl(monkeypatch):
    monkeypatch.setenv("SMTPD_USE_SSL", "True")

def test_smtpd_ssl(mock_smtpd_use_ssl, smtpd):
    client = smtplib.SMTP_SSL(smtpd.hostname, smtpd.port)
    msg = EmailMessage()
    msg["Subject"] = "Hello, World!"
    msg["To"] = "[email protected]"
    msg["From"] = "[email protected]"
    msg.set_content("Hi, world!  How you doin'?\n")
    client.send_message(msg)
    client.quit()
    assert len(smtpd.messages) == 1

The output from pytest is as follows:

============================= test session starts ==============================
platform linux -- Python 3.6.12[pypy-7.3.3-final], pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /docked
plugins: smtpdfix-0.2.9
collected 1 item

test_ssl2.py E                                                           [100%]

==================================== ERRORS ====================================
_______________________ ERROR at setup of test_smtpd_ssl _______________________

tmp_path_factory = TempPathFactory(_given_basetemp=None, _trace=<pluggy._tracing.TagTracerSub object at 0x00007f0eab940950>, _basetemp=PosixPath('/tmp/pytest-of-root/pytest-2'))

    @pytest.fixture
    def smtpd(tmp_path_factory):
        """A small SMTP server for use when testing applications that send email
        messages. To access the messages call `smtpd.messages` which returns a copy
        of the list of messages sent to the server.
    
        Example:
            def test_mail(smtpd):
                from smtplib import SMTP
                with SMTP(smtpd.hostname, smtpd.port) as client:
                    code, resp = client.noop()
                    assert code == 250
        """
        config = Config()
    
        if os.getenv("SMTPD_SSL_CERTS_PATH") is None:
            path = tmp_path_factory.mktemp("certs")
            generate_certs(path)
            os.environ["SMTPD_SSL_CERTS_PATH"] = str(path.resolve())
    
        # fixture = AuthController(
        #     hostname=config.SMTPD_HOST,
        #     port=int(config.SMTPD_PORT),
        #     config=config,
        #     authenticator=_Authenticator(config)
        # )
        # fixture.start()
        # yield fixture
        # fixture.stop()
    
>       with SMTPDFix(config.SMTPD_HOST, config.SMTPD_PORT) as fixture:

/opt/pypy/site-packages/smtpdfix/fixture.py:105: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/pypy/site-packages/smtpdfix/fixture.py:64: in __enter__
    authenticator=_Authenticator(self.config)
/opt/pypy/site-packages/smtpdfix/controller.py:48: in __init__
    ssl_context=context_or_none())
/opt/pypy/site-packages/smtpdfix/controller.py:40: in context_or_none
    return ssl_context or self._get_ssl_context()
/opt/pypy/site-packages/smtpdfix/controller.py:81: in _get_ssl_context
    context.load_cert_chain(cert_path, key_path)
/opt/pypy/lib_pypy/_cffi_ssl/_stdssl/__init__.py:1113: in load_cert_chain
    certfilebuf = _str_to_ffi_buffer(certfile)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

view = PosixPath('/tmp/pytest-of-root/pytest-2/certs0/cert.pem')

    def _str_to_ffi_buffer(view):
        if isinstance(view, str):
            return ffi.from_buffer(view.encode())
        elif isinstance(view, memoryview):
            # NOTE pypy limitation StringBuffer does not allow
            # to get a raw address to the string!
            view = view.tobytes()
        # dont call call ffi.from_buffer(bytes(view)), arguments
        # like ints/bools should result in a TypeError
>       return ffi.from_buffer(view)
E       TypeError: a bytes-like object is required, not PosixPath

/opt/pypy/lib_pypy/_cffi_ssl/_stdssl/utility.py:25: TypeError
=========================== short test summary info ============================
ERROR test_ssl2.py::test_smtpd_ssl - TypeError: a bytes-like object is requir...
=============================== 1 error in 0.51s ===============================

The fixture works on PyPy3 when SSL is not used. When StartTLS is used on PyPy3, pytest just hangs indefinitely; I suspect that the TypeError is killing the SMTPD job and nothing's cleaning it up.

This is a reminder checklist to ensure that everything has been done before release 0.4.1 is published:

• Bump versions in \requirements\latest to latest versions.
• __version__ updated to 0.4.1 in \smtpdfix\__init__.py
• Release date set in \docs\CHANGELOG.md

There are several errors and typos in README.md that need fixing. At a minimum:

1. Typo "As" to "A" in the opening sentence.
2. Reword opening paragraph to read more clearly
3. Change "Installing handled through PyPi:" to "Install using PyPi:"
4. Using remove conftest.py option and simplify to reflect a single file.
5. Add examples for SSL and StartTLS
6. Fix up the copyright footer and see about changing it to look better. This should include the <hr>

On Python 3.9.2 on macOS 11.2.1, using pytest 6.2.2 and smtpdfix 0.2.9, the following test fails with smtplib.SMTPNotSupportedError: SMTP AUTH extension not supported by server.. Uncommenting the line that sets SMTPD_ENFORCE_AUTH does not make a difference. The test succeeds if the client.login() line is removed.

import smtplib

def test_smtpdfix(monkeypatch, smtpd):
    #monkeypatch.setenv("SMTPD_ENFORCE_AUTH", "True")
    client = smtplib.SMTP()
    client.connect(smtpd.hostname, smtpd.port)
    client.login("user", "password")
    client.quit()

With the correction for the possibility that a FileNotFoundError for the SSL certificate would cause the fixture to hang on windows releasing ver. 0.2.2 is warranted. This version will also include implementing CRAM-MD5 and optionally enforcing authentication.

• Renew certificates to be valid for ten years
• Bump version number to 0.2.2 and tag
• Release package using twine
• Add release to Github repository

A number of issues exist with the CI actions currently running that need to be addressed:

• Checks in tests.yml currently run against the dev version of python 3.11 (py3.11-dev) and should be updated to run against the latest production version available.
• Checks in daily.yml do not run against python 3.11
• Checks for minimum versions may be overridden by the pip install command pip install .[dev] -r requirements/minimum/requirements.txt this should be changed to pip install --no-deps . and pip install -r requirements/minimum/requirements.txt

In cases where the server is set up to authenticate it's reasonable to assume that other commands would result in it refusing to respond unless AUTH had been completed first.

Currently the configuration for the certificate and key for the SSL context is fixed to load a cert.pem and key.pem file.

It should be more flexible to allow for files with different names to be loaded.

Currently attempting to authenticate twice returns a message of 530 Already authenticated. which doesn't meet the requirement in RFC4954 that any attempt to authenticate after a user has done so should be responded to with a 503 response.

Relevant error message is contained here:
https://github.com/bebleo/bebleo_smtpd_fixture/blob/8db69b4e13135dbeb35d9ab0e10309f8429b52db/bebleo_smtpd_fixture/handlers.py#L11

And the test should be modified to test for the appropriate code:
https://github.com/bebleo/bebleo_smtpd_fixture/blob/7a5ccae5942b7f583610152eeb0eae85846f6b7f/tests/test_fixture.py#L87-L93

Dear James,

first things first: Thanks a stack for conceiving and maintaining this excellent package. It is tremendously helpful for conducting full system tests.

We are currently in the process of giving some love to an abandoned Python project over at [1,2], where we would like to use smtpdfix to verify email delivery within the test harness.

When starting with smtpdfix, we quickly ran into #89. However, we were happy to find #95, which already removed that feature. Other than the potential hiccup for some users where socket.gethostbyname() croaks, we believe it will be a great enhancement because the service will be available for consumption instantly, without going through key material and certificate generation, right?

So, we already recognized you were preparing version 0.4.0, at #144. May we humbly ask if you could publish version 0.4.0 to PyPI to make it available for pulling it into downstream projects? We think it would be a great benefit for the community. Please let us know if there are some blockers where we may be able to help out.

With kind regards,
Andreas.

[1] https://github.com/isarengineering/SecPi
[2] SecPi/SecPi#120

Describe the bug
When the library is initializing, it croaks like:

        # Generate public certificate
        hostname = socket.gethostname()
>       ip = socket.gethostbyname(hostname)
E       socket.gaierror: [Errno 8] nodename nor servname provided, or not known

.venv/lib/python3.10/site-packages/smtpdfix/certs.py:35: gaierror

To Reproduce
It happens on slightly misconfigured systems where the designated hostname does not resolve to an IP address.

>>> import socket
>>> socket.gethostbyname(socket.gethostname())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

Expected behavior
The library should work instantly by starting an SMTP server on port 8025.

Environment (please complete the following information):

• macOS 10.15.7
• Python 3.10
• smtpdfix 0.3.3

When running under Windows and a FileNotFoundError is raised the fixture hangs. Since signals doesn't play nicely with asyncio on Winddows this is difficult to easily recover from (see issue #4).

The method raising the error:
https://github.com/bebleo/bebleo_smtpd_fixture/blob/88d518ec6fc852eb38cbf09a0105a98474cffc2a/bebleo_smtpd_fixture/controller.py#L52-L70

Describe the bug
The OSSF Scorecards Github action fails to run when the workflow is invoked.

To Reproduce
Create a PR or push a commit to an existing PR so that the workflow is triggered.

Expected behavior
Workflow should run and the results are reported.

Distutils will be removed from python version 3.12.

Remove the dependency from smtpdfix/configuration.

I just stumbled upon a test case on which I would need to check if a mail contains a given attachment or not, but I could not find documentation about this, and dir did not helped much neither.

I suggest to implement and document how to check the attachments on mails.

As of the date this issue is being created the default branch is named master which carries offensive connotations due to it's historical associations with slavery and other relationships of systemic racism. This means that this project is failing to meet it's own, currently unstated, principle of being inclusive of all contributors and users.

We also consider the absence of an explicit statement of principles to be a failure. By not including such a statement this project continues to maintain that the status quo is acceptable. Measures must be taken to explicitly normalize inclusiveness and state that this is our goal. Part of which is to state that actions which are counter to that goal will not be tolerated. And then, to follow through by addressing any such behaviour.

To address these failures we will:

• rename master to main,
• add a code of conduct, and
• explicitly and directly address any actions that are contrary to that code of conduct.

When using smtpdfix in addition with pytest-xdist, one smptd server is started for each test process. As the smtpdfix default port is 8025, only one fixture can bind to it, and the other will raise errors.

I would suggest instead to use a random unavailable port by default, instead of 8025. portpicker might be a lightweight tool for this.

This can be solved by using a custom fixture, but I though this issue would be generic enough to have a solution directly in smptdfix.

What do you think?

Add support for .env files as a way to set environment variables.

• #341
• #338
• #337

Datetime

datetime.utcnow() will be deprecated in python 3.12 and removed in a future version.

tests/test_controller.py::test_missing_auth_handler
tests/test_controller.py::test_custom_ssl_context
tests/test_controller.py::test_custom_cert_and_key
tests/test_controller.py::test_TLS_not_supported
tests/test_smtpdfix.py::test_misconfigured_socket
  /Users/james/repos/smtpdfix/smtpdfix/certs.py:66: DeprecationWarning: datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.now(datetime.UTC).
    .not_valid_before(datetime.utcnow())

tests/test_controller.py::test_missing_auth_handler
tests/test_controller.py::test_custom_ssl_context
tests/test_controller.py::test_custom_cert_and_key
tests/test_controller.py::test_TLS_not_supported
tests/test_smtpdfix.py::test_misconfigured_socket
  /Users/james/repos/smtpdfix/smtpdfix/certs.py:67: DeprecationWarning: datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.now(datetime.UTC).
    .not_valid_after(datetime.utcnow() + timedelta(days=days))

Asyncio Error

___________________________________________________________ ERROR at teardown of test_missing_certs ____________________________________________________________

self = <smtpdfix.controller.AuthController object at 0x106d80380>, no_assert = False

    def stop(self, no_assert: bool = False):
        """
        Stop the loop, the tasks in the loop, and terminate the thread as well.
        """
        assert no_assert or self._thread is not None, "SMTP daemon not running"
        self.loop.call_soon_threadsafe(self.cancel_tasks)
        if self._thread is not None:
>           self._thread.join()

.tox/py312/lib/python3.12/site-packages/aiosmtpd/controller.py:333: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.pyenv/versions/3.12.0b3/lib/python3.12/threading.py:1126: in join
    self._wait_for_tstate_lock()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Thread(Thread-6 (_run), stopped daemon 6110162944)>, block = True, timeout = -1

    def _wait_for_tstate_lock(self, block=True, timeout=-1):
        # Issue #18808: wait for the thread state to be gone.
        # At the end of the thread's life, after all knowledge of the thread
        # is removed from C data structures, C code releases our _tstate_lock.
        # This method passes its arguments to _tstate_lock.acquire().
        # If the lock is acquired, the C code is done, and self._stop() is
        # called.  That sets ._is_stopped to True, and ._tstate_lock to None.
        lock = self._tstate_lock
        if lock is None:
            # already determined that the C code is done
            assert self._is_stopped
            return
    
        try:
>           if lock.acquire(block, timeout):
E           Failed: Timeout >10.0s

../../.pyenv/versions/3.12.0b3/lib/python3.12/threading.py:1146: Failed
------------------------------------------------------------------- Captured stderr teardown -------------------------------------------------------------------

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Timeout ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Stack of Thread-6 (_run) (6110162944) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/threading.py", line 1009, in _bootstrap
    self._bootstrap_inner()
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/threading.py", line 1052, in _bootstrap_inner
    self.run()
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/threading.py", line 989, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/james/repos/smtpdfix/smtpdfix/controller.py", line 158, in _run
    self.loop.run_until_complete(self.server.wait_closed())
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/asyncio/base_events.py", line 651, in run_until_complete
    self.run_forever()
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/asyncio/base_events.py", line 618, in run_forever
    self._run_once()
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/asyncio/base_events.py", line 1913, in _run_once
    event_list = self._selector.select(timeout)
  File "/Users/james/.pyenv/versions/3.12.0b3/lib/python3.12/selectors.py", line 561, in select
    kev_list = self._selector.control(None, max_ev, timeout)

Recent changes to aiosmtpd released under version 1.2.3 cause smtpdfix to hang on an error when initializing AuthSession.

Workaround: fix aiosmtpd to be <= 1.2.2; this is what version 0.2.7 does by default if installed with pip install smtpdfix

The issue appears to be that:

1. Errors aren't propagating up when they occur on the loop running the AuthSMTP server.
2. smtpdfix.AuthSession sets self.authenticated = False in __init__ which conflicts with the self.authenticated property in aiosmtpd.Session which AuthSession inherits from. The property was added with version 1.2.3 of aiosmtpd.

If the start of the server is aborted because the ready_timeout is exceeded the socket is not consistently closed down. when this happens subsequent uses fail with a message that the host and port are in use.

Release version 0.5.1

• all versions are up to date in requirements
• __version__ is updated in __init__.py
• Changelog has been updated
• Release has been created
• Latest version has been uploaded to PyPi

On Python 3.9.2 on macOS 11.2.1, using pytest 6.2.2 and smtpdfix 0.2.9, the following fails on the client = smtplib.SMTP_SSL(...) line with an SSL error:

from email.message import EmailMessage
import smtplib

def test_smtpd_ssl(monkeypatch, smtpd):
    monkeypatch.setenv("SMTPD_USE_SSL", "True")
    client = smtplib.SMTP_SSL(smtpd.hostname, smtpd.port)
    msg = EmailMessage()
    msg["Subject"] = "Hello, World!"
    msg["To"] = "[email protected]"
    msg["From"] = "[email protected]"
    msg.set_content("Hi, world!  How you doin'?\n")
    client.send_message(msg)
    client.quit()
    assert len(smtpd.messages) == 1

Meanwhile, setting SMTPD_USE_SSL outside of the test in a fixture succeeds:

from email.message import EmailMessage
import smtplib
import pytest

@pytest.fixture
def mock_smtpd_use_ssl(monkeypatch):
    monkeypatch.setenv("SMTPD_USE_SSL", "True")

def test_smtpd_ssl(mock_smtpd_use_ssl, smtpd):
    client = smtplib.SMTP_SSL(smtpd.hostname, smtpd.port)
    msg = EmailMessage()
    msg["Subject"] = "Hello, World!"
    msg["To"] = "[email protected]"
    msg["From"] = "[email protected]"
    msg.set_content("Hi, world!  How you doin'?\n")
    client.send_message(msg)
    client.quit()
    assert len(smtpd.messages) == 1

This is presumably not the intended way to set SMTPD_USE_SSL, and so I am reporting it as a bug.

When running the tests we sometimes get a connection refused error. This may relate to the smtpd server not have successfully shut down by the start of the next test.

The behaviour is inconsistent and doesn't reproduce locally on a windows 10 machine.

Error message from the github action:

============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.2, py-1.8.1, pluggy-0.13.1
rootdir: /home/runner/work/bebleo_smtpd_fixture/bebleo_smtpd_fixture
collected 6 items

tests/test_fixture.py .                                                  [ 16%]
tests/test_smtpd.py .E...                                                [100%]

==================================== ERRORS ====================================
_______________________ ERROR at setup of test_alt_port ________________________

request = <SubRequest 'client' for <Function test_alt_port>>

    @pytest.fixture
    def client(request):
        host = getenv("SMTPD_HOST", "127.0.0.1")
        port = int(getenv("SMTPD_PORT", "8025"))
>       smtp = SMTP(host=host, port=port)

tests/conftest.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/smtplib.py:253: in __init__

The example given for using SMTPDFix not as a fixture includes msg, a variable that is a fixture used in the tests of the project itself, that should be removed.

load_dotenv() should not be executed on library loading
(see here ) because its loads the default .env file that we have inside our project.
If we need to load a specific .env file for testing, we are obliged to override all the values. It also can be a security issue.

I would recommend letting the user loading env file and then uses os.getenv() with default values.

Versions 0.2.4 through 0.2.8 allow an insecure version of the cryptography package to be used. As a piece of housekeeping we should yank these versions on pypi so that they're no longer implicitly available.