reanahub / pytest-reana Goto Github PK

Currently REANA-Workflow-Controller and REANA-Server hold DB fixtures which should be extracted to have a common code base.

For inspiration we have pytest-invenio.

if in-memory queues are not cleared tests fail.

The test fails for both MessageConsumer and scheduler. This is probably due to in_memory_queue_connection having a "session" scope. It means some messages can stay in the queue between tests and cause unexpected failures.

As for now, I solved it by cleaning messages at the end of each test. But, this is not a sustainable solution. It is not nice when you need to remember to clear stuff after the test. I propose to create an issue to address later.

Originally posted by @VMois in reanahub/reana-server#487 (comment)

Extract RWC's FS fixture to reuse it across components:

• In REANA-Server creating the user workspace
• In REANA-Workflow-Engine-_ workspace for a serial workflow, same will come later for REANA-Workflow-Engine-CWL/Yadage
• REANA-Job-Controller (this is not clear, since the shared volume path is just provided to the jobs so they know where to work on...)

reana-server has both reana-commons and reana-pytest-commons as dependencies.
reana-commons requires click>=7.0,<8.0 while reana-pytest-commons requires click>=6.7,<7.0 ending up in a version conflict. Should we unpin click in reana-pytest-commons?

Here is a link to GH actions run with python 2.7: https://github.com/audrium/pytest-reana/runs/1347493807
It finishes with errors which needs to be addressed

Currently we have:

~/reanahub $ rg 'pytest>|pytest<|pytest='
reana-commons/setup.py
31:    'pytest>=3.8'

reana-workflow-engine-cwl/setup.py
34:    'pytest>=2.8.0',

pytest-reana/setup.py
55:    'pytest>=3.8.0,<4.0.0',

reana-job-controller/setup.py
30:    'pytest>=2.8.0',

reana-message-broker/setup.py
29:    'pytest>=2.8.0',

reana-client/setup.py
27:    'pytest>=3.8.0,<4.0.0',

reana-workflow-monitor/setup.py
28:    'pytest>=2.8.0',

reana-workflow-controller/setup.py
30:    'pytest>=3.8.0,<4.0.0',

reana-server/setup.py
32:    'pytest>=3.8.0,<4.0.0',

reana-workflow-engine-serial/setup.py
30:    'pytest>=2.8.0,<3.0.0'

reana-db/setup.py
30:    'pytest>=2.8.0,<3.0.0'

reana-cluster/setup.py
29:    'pytest>=2.8.0'

reana/setup.py
29:    'pytest>=2.8.0',

reana-workflow-engine-yadage/setup.py
29:    'pytest>=2.8.0',

We should:

• Remove upper boundaries
• Set the same minimum version everywhere

Stemmed from reanahub/reana-workflow-controller#141 (comment).

There is a problem running tests and using session fixture. The changes added by the session contained in the Flask application cannot be seen with the session fixture inside the test case (in principle they should be the same or at least connect to the same DB).

For the time being, we are running the tests using a persisted DB (a file SQLite).

To reproduce on R-W-Controller:

1. Set back the DB to in memory:

diff --git a/tests/conftest.py b/tests/conftest.py
index 21f6ed6..a5da2bd 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -28,7 +28,7 @@ def base_app(tmp_shared_volume_path):
         "SECRET_KEY": "SECRET_KEY",
         "TESTING": True,
         "SHARED_VOLUME_PATH": tmp_shared_volume_path,
-        "SQLALCHEMY_DATABASE_URI": "sqlite:///testdb.db",
+        "SQLALCHEMY_DATABASE_URI": "sqlite://",
         "SQLALCHEMY_TRACK_MODIFICATIONS": False,
         "ORGANIZATIONS": ["default"],
     }

2. Run tests on the test_views.py module:

$ py.test -s tests/test_views.py

This should be ok and passing.
3. Run the whole test suite

$ py.test -s tests

There are two in-memory DB's instantiated instead of one.

In pytest-reana's setup.py we have upper bound [1] of mock dependency set to <4.0.

OTOH, in reana-commons' setup.py we don't have that upper limit
, which leads us to incompatibilities.

To reproduce it, go to a component that installs both r-commons and pytest-reana (e.g. r-server) and do:

$ pip install -e .[all]
...
# you'll see a warning over here already

$ pip check
pytest-reana 0.7.0a1 has requirement mock<4.0,>=3.0, but you have mock 4.0.2.

⚠️ In addtion, with the coming pip version 20.3, this kind of warning will become errors, so it's best to solve this ASAP.

It will be stricter - if you ask pip to install two packages with incompatible requirements, it will refuse (rather than installing a broken combination, like it did in previous versions).

[1] We set an upper bound to mock (<4.0) because some of our tests were breaking with 4.0.x and we were in a rush for Awesome Workshop.