Explore if it is possible to have test dependencies about memote HOT 5 CLOSED

http://stackoverflow.com/a/12579625

from memote.

The principle of test ordering is explained here: http://pytest-incremental.readthedocs.io/runner.html#how-to-order-tests

But of course we'd like to set the test-dependencies ourselves, instead of basing them on the import graphs, like said plugin.

Here is an example, of a hypothetical 'Test Pathway':
a) The model's biomass reaction is functional.
Model > Test Biomass Production > Pass > Termination of this Pathway

b) The model's biomass reaction is incomplete.
Mode > Test Biomass Production > Fail > Test Production of Biomass Precursors in Default Condition > Fail > Test Production of Biomass Precursors with all exchanges opened > Pass > Termination of this Pathway

I see the following benefits of 'Test Pathways':

• Shorter testing time for models that are well curated.
• Adaptive output that highlights a model's problem areas without spamming 'irrelevant' test results.

from memote.

I have to think more about test dependencies since it goes against general testing practices. General consensus is to test everything all the time but then again this is a very specialized case. We might want to test everything all the time, though, such that the data is generated and complete even for failing cases.

from memote.

I know we've talked about how it is better to just record all tests results no matter what, but just to provide closure to the discussion I saw that it is indeed possible to do that for which the issue was created:

incremental testing - test steps
Sometimes you may have a testing situation which consists of a series of test steps. If one step fails it makes no sense to execute further steps as they are all expected to fail anyway and their tracebacks add no insight. Here is a simple conftest.py file which introduces an incremental marker which is to be used on classes[.]

https://docs.pytest.org/en/latest/example/simple.html

from memote.

Final agreement that test suite is fast enough and we need to run tests for result generation.

from memote.