Git Product home page Git Product logo

pytest-ordering's Introduction

pytest-ordering is no longer maintained, please use https://pypi.org/project/pytest-order/

pytest-ordering

pytest plugin to run your tests in a specific order

Build Status

Have you ever wanted to easily run one of your tests before any others run? Or run some tests last? Or run this one test before that other test? Or make sure that this group of tests runs after this other group of tests?

Now you can.

Install with:

pip install pytest-ordering

This defines some pytest markers that you can use in your code.

For example, this:

import pytest

@pytest.mark.run(order=2)
def test_foo():
    assert True

@pytest.mark.run(order=1)
def test_bar():
    assert True

Yields this output:

$ py.test test_foo.py -vv
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
plugins: ordering
collected 2 items

test_foo.py:7: test_bar PASSED
test_foo.py:3: test_foo PASSED

=========================== 2 passed in 0.01 seconds ===========================

Check out the docs: http://pytest-ordering.readthedocs.org/

pytest-ordering's People

Contributors

a-tal avatar brianmaissy avatar ftobia avatar mattias-lundell avatar robdennis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pytest-ordering's Issues

Allow ordering on per-class or per-module basis, instead of just per-session

If I mark a test first within a class, it should be possible to have that test run first in that class, not first overall (unless that class is the first to run).

Similarly, if I mark a test first within a module, it should be possible to have that test run first in that module, but not first overall (unless that module is the first to run).

Perhaps the best way to specify this would be a keyword argument like @pytest.mark.first(scope='class') or @pytest.mark.second(scope='session').

No matching distribution found for pytest-ordering

below error when im trying to install the package

python version : 2.7

Collecting pytest-ordering
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x109fbb250>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',)': /artifactory/api/pypi/python/simple/pytest-ordering/
Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x109fbb2d0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',)': /artifactory/api/pypi/python/simple/pytest-ordering/
Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x109fc1290>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',)': /artifactory/api/pypi/python/simple/pytest-ordering/
Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x109fc1a90>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',)': /artifactory/api/pypi/python/simple/pytest-ordering/
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x109fc1d90>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',)': /artifactory/api/pypi/python/simple/pytest-ordering/
Could not find a version that satisfies the requirement pytest-ordering (from versions: )
No matching distribution found for pytest-ordering

Run tests in order of definition

Hello!

I wanted to run my tests in the order they are defined, rather than having to define every step myself. Is there some other way of achieving this, or would this be a useful addition to the module?

class Order:
    i = 1500

    @staticmethod
    def auto():
        Order.i +=  1
        return Order.i

Then I can add the following tag to all tests:

@pytest.mark.run(order=Order.auto())

Travis-CI integration

Tests should be running on travis-ci. Ideally we support Python 2.7 and 3.3+. This implies setting up tox.

Ordering ignored on specification of multiple testcases

While calling pytest with multiple modules/classes/test methods, pytest ordering is ignored and instead ordering such as on command line is being used.

py.test foo.py bar.py dood.py

Will be ordered tests from foo.py -> bar.py -> dood.py regardless of pytest-ordering plugin, when dood.py should have been running before foo and bar based on test being wrapped with order 1 and foo.py and bar.py test being wrapped with higher order.

Ordering is based on internal dictionary implementation

Correct me if I'm wrong, but doesn't the following code rely purely on the internal implementation of the dict to order the keys in the correct way (0, 1, 2, 3, ..., -3, -2, -1)?

for key, ordered_items in grouped_items.items():

            if unordered_items and prev_key >= 0 and key < 0:

                sorted_items.extend(unordered_items)
                unordered_items = None

            prev_key = key

            sorted_items.extend(ordered_items)

        if unordered_items:
            sorted_items.extend(unordered_items)

Test sparse ordinal behavior

There is a behavior I intended which I have not tested (see also: mattias-lundell@a29a07f#commitcomment-9290806). The behavior is that you need not specify all the ordinals from the beginning or end for order to be respected. That is, if you mark a test as "third", it will be the third test to run, regardless of whether you have tests marked "first" and "second" (assuming of course that you have at least 3 tests).

There should be tests for the from-the-beginning case, the from-the-end case, and various edge cases.

Standardize on a single marker name: "order"

Reading the documentation I noticed that there are a lot of potential marker names including run(), and even custom markers proposed in #10. It seems like this has the potential to result in confusing code. Say for instance that some one is new to a project and doesn't know about pytest-ordering. What does the run marker do? Run is a very ambiguous word. I would like to suggest choosing a more specific name. As an option order might be a more intuitive name for someone reading code. The marker signature might look like this: order(pos=None, before=None, after=None).

RemovedInPytest4Warning: MarkInfo objects are deprecated

Hi everybody!

Could someone update code regarding this warning:

/home/kradem/.virtualenvs/myenv/lib/python3.6/site-packages/pytest_ordering/__init__.py:54: 
RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain merged marks which are 
hard to deal with correctly.
Please use node.get_closest_marker(name) or node.iter_markers(name).
Docs: https://docs.pytest.org/en/latest/mark.html#updating-code
  order = mark.kwargs.get('order')

Doesn't work when using inside a class

If my code is something like this:

class Test(TestCase):
    i = 0

    @pytest.mark.run(order=1)
    def test_c(self):
        assert self.i == 0
        self.i += 1

    @pytest.mark.run(order=2)
    def test_b(self):
        assert self.i == 1
        self.i += 1

    @pytest.mark.run(order=3)
    def test_a(self):
        assert self.i == 2
        self.i += 1

It says 'Empty test suite.', and when I change the marker into something like @pytest.mark.first, it still doesn't work. It seems to me that pytest doesn't obey these marks anymore.

How do I solve this ? or I'm doing it wrongly ?

INTERNALERROR> AttributeError: 'Function' object has no attribute 'get_closest_marker' on pytest 3.10 with pytest-ordering in python 3.4 and 3.6

I'm getting an error since the release of pytest 3.10 on several of my repos that use pytest-ordering.

Python versions that are affected are python 3.4 and python 3.6.
Python 2.7 and 3.5 seem to be working fine.

Here are some examples running on travis-ci:
https://travis-ci.com/MarvinT/morphs/builds/90470351
https://travis-ci.com/MarvinT/morphs/builds/90486557
https://travis-ci.com/gentnerlab/behav-analysis/builds/90654511

I'll probably change them to force version 3.9.3 if anyone is reading this from the future but I've tried to link to the specific builds that failed for this reason. Some of them previously passed on 3.9.3 and I re-ran them when 3.10 came out just to make sure it wasn't my code that was causing it to fail.

example stack trace:

3.50s$ pytest --cov=morphs
============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.3.0, py-1.5.2, pluggy-0.6.0
rootdir: /home/travis/build/MarvinT/morphs, inifile:
plugins: ordering-0.6, cov-2.6.0
collected 10 items                                                             
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py", line 103, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py", line 140, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py", line 150, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py", line 643, in perform_collect
INTERNALERROR>     config=self.config, items=items)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/pytest_ordering/__init__.py", line 45, in pytest_collection_modifyitems
INTERNALERROR>     mark = item.get_closest_marker(mark_name)
INTERNALERROR> AttributeError: 'Function' object has no attribute 'get_closest_marker'

Ordering with multiple files

Use case:

  • creaet a test file "test_example1.py"
  • Mark order from 1 to 5 for different test_methods
  • Now create another test file "test_example2.py"
  • Mark order from 1 to 5 for different test_methods
  • py.test executes order 1 from both the files first even the test_method names are diffrent
  • And it goes in similar pattern, executing order=1 in all files then goes to order=2 in all files etc.

It becomes very difficutl when we have more than 100 test case , If any one test case is added in single file we need to change the order number in all test case files.
More info in "http://stackoverflow.com/questions/35909656/pytest-ordered-plugin-doesnt-work-with-combination-of-multiple-files"

support for python 3.6

It seems that the pytest ordering does not work with python 3.6. Is it possible to support this package for python 3.6?

@pytest.mark.last doesn't work

20:25:57@5 ~/lira/tmp  $ py.test -v proba.py 
============================================================================= test session starts ==============================================================================
platform darwin -- Python 2.7.9 -- py-1.4.26 -- pytest-2.7.0 -- /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
rootdir: /Users/lira/tmp, inifile: 
plugins: ordering
collected 4 items 

proba.py::test3 PASSED
proba.py::test1 PASSED
proba.py::test2 PASSED
proba.py::test4 PASSED

=========================================================================== 4 passed in 0.01 seconds ===========================================================================
20:25:59@5 ~/lira/tmp  $ cat proba.py 
import pytest


def test1():
    pass

def test2():
    pass

@pytest.mark.last
def test3():
    pass

def test4():
    pass

pytest ordering excecute in reverse order

import pytest

@pytest.mark.run(order=1)
def testA():
    print("Method A")

@pytest.mark.run(order=2)
def testB():
    print("Method B")

@pytest.mark.run(order=3)
def testC():
    print("Method C")

Output :
collected 3 items

pytest_basics.py::testC Method C
PASSED
pytest_basics.py::testB Method B
PASSED
pytest_basics.py::testA Method A
PASSED

Minimal documentation on readthedocs.org

I have pytest-ordering registered at readthedocs.org, but there are no docs to build. Set up sphinx, write some minimal "here is what pytest-ordering does" documentation, and make sure it's being built.

Ordering a non-continuous segment causes no ordering to happen

I've got a number of py.test suites written that test various aspects of a system. One file might have a few dozen tests in it, grouped logically. The next file might have only a dozen. The next might have a hundred or more.

I'd like to run file1 before file2 before file3, and I absolutely NEED to run the tests inside file1 in a particular order, and the tests inside file2 in a particular order, and same for file3. So if I start file1 at pytest.mark.run(order=1000) , then 1001, 1002, etc, and file2 at pytest.mark.run(order=2000) and file3 at pytest.mark.run(order=3000) I naively thought that this would result in 1000, 1001, 1002, etc then 2000, 2001, etc, 3000, 3001, etc all getting run in order of increasing order number.

But I'm not seeing that behavior at all when I run the tests as a suite. In order to get the files to execute in the correct order I've used a script to run py.test on each one of them individually. And that seemed to work on the files starting at order=1000 or order=2000. But now that I've got a file starting at order=3000, I'm seeing strange behavior where instead of

[3000,3001,30002,3003,3004,3005,3006,3007,3008,3009,3010,3011]

I'm instead seeing

[3008,3009,3010,3011,3000,3001,30002,3003,3004,3005,3006,3007]

I took a peek in https://github.com/ftobia/pytest-ordering/blob/develop/pytest_ordering/__init__.py but it wasn't immediately obvious to me what was getting done where. I'll probably take another look after I post this to see if I can't put together a pull request, but I'm definitely puzzled.

The main reason I would like this out-of-order business is so that if I add a couple of tests to the end of one file I don't have to go reset the ordering of every other file (by hand!) to make room in the perfectly ordered number line.

Simplify tests

The tests are a bit complicated. I want a function to collect the tests in a given module, and return a list of items that would be passed to the pytest_collection_modifyitems hook.

Since I couldn't figure out how to do that, I'm running py.test programmatically via pytest.main(), capturing standard out, and then checking the output to see if the tests were run in the correct order. It would be so much easier to just collect the tests without the plugin installed (or maybe generate a list of test items programmatically), pass them to an ordering function, and assert that the output is in the correct order.

Need to figure out if this is possible, and if so, how to make this happen.

Is this project still maintained?

I noticed the "aspirational" part in the documentation after seeing a post in SO (similar to #65) and thought if it could make sense to contribute to this--but there seems to be no activity here for a year now, and PRs are not handled.

I'm not sure if this is is just temporary, or the project is not maintained anymore - thus this question...

Ordering of test suite

Hi.
I use your plugin no so long. I didn't find opportunity running test suites.
For example:
I have folder with tests which interested me, i need to run them in special ordering. But also in class i need to run methods in special ordering

it seems like on:
@pytest.mark.cross_acceptance
@pytest.mark.run(order=5)
class TestAcceptanceFifth:
@pytest.mark.run(order=1)
class TestWebPart(Web):
pass
@pytest.mark.run(order=2)
class TestIOSPart(IOS):
pass

this example runs first not fifth.
Thanks

Allow ordering within a "group"

Allow custom markers to define a "group" when you use ordering keywords, for example:

@pytest.mark.foo(order=2)
def bar():
    pass

@pytest.mark.foo(order=1)
def baz():
    pass

The above example will ensure that baz() runs before bar(). The group foo will itself not be ordered with respect to other tests.

Version 0.6 can not be installed from pypi

Hi,

I am getting pytest warnings because I can not install version 0.6 from pypi.

From linux (centos, python2.7), I have run pip install pytest-ordering and I got the version 0.5.

When I try to install the version 0.6 from the command line:
pip install pytest-ordering==0.6

I am getting an error: Could not find a version that satisfies the requirement ...

Any idea ?

Thanks

Custom markers

As discussed on stack overflow, this is a request for new feature to define custom markers.
We need to consider the trailing digits of the marker as the ordering.
E.g If I say @pytest.mark.custom1 or @pytest.mark.custom(order=1) , then 1 should be the order.
Thank you!

The module relative order don't working???

this module don't working?

import pytest
@pytest.mark.run(after='test_second')
def test_third():
    assert True

def test_second():
    assert True

@pytest.mark.run(before='test_second')
def test_first():
    assert True

The order don't change all ???

E:\QT-learn>pytest -svv
============================================================================= test session starts =============================================================================
platform win32 -- Python 3.7.4, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- c:\users\jekoie\appdata\local\programs\python\python37-32\python.exe
cachedir: .pytest_cache
rootdir: E:\QT-learn, inifile: pytest.ini
plugins: ordering-0.6
collected 3 items                                                                                                                                                              

test_qt.py::test_third PASSED
test_qt.py::test_second PASSED
test_qt.py::test_first PASSED

============================================================================== 3 passed in 0.04s ==============================================================================

E:\QT-learn>

The test methods are not printed in order in the junit report

Hi,

I have been using pytest-ordering and it works/fits my usecase. Thank you for providing this plugin.

Whenever I generate the junit xml report, the tests are not in order they are executed in the report. Is it possible to re-order based on the order they are executed ?

pytest-ordering doesn't honor test dependencies

import pytest

def test_a():
    pass

@pytest.mark.dependency(depends=['test_a'])
@pytest.mark.first
def test_b():
    pass
$ pytest -v -x -s test.py
================================== test session starts ==================================
platform linux -- Python 3.7.4, pytest-5.1.1, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/anubis/git/ratbag-emu
plugins: ordering-0.6
collected 2 items

test.py::test_b PASSED
test.py::test_a PASSED

order will work between diffrent testClass

python 3.7.1
pytest 4.4.0
pytest-ordering 0.6

there are two testClass, and I find the test will run as order which I set.

import unittest
import pytest
class TestA(unittest.TestCase):
    @pytest.mark.run(order=0)
    def test_0(self):
        pass 

    @pytest.mark.run(order=2)
    def test_2(self):
        pass 
    ...  #other test_method

class TestB(unittest.TestCase):
    @pytest.mark.run(order=1)
    def test_1(self):
        pass 
    ...  #other test_method

run the code, I find the tests run as test_0-->test_1-->test_2.

But, I want to the tests run all test_method in a TestClass and then run another TestClass.
like test_0-->test_2-->test_1, or test_1-->test_0-->test_2.

any way, the design of now is cool。
I think add a param to control the run order only work in a TestClass will be better.
Because the TestA 's setUpClass will run again after run test_1,if setUpClass is time long,will be bad feel.

All ordering tests are failing (git develop)

This is the result of the test suite on our (Arch Linux) building system:

============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /build/python-pytest-ordering/src/pytest-ordering-0.6
collected 16 items

tests/test_misc.py ..                                                    [ 12%]
tests/test_ordering.py .FFFFFFFFFFFFF                                    [100%]

=================================== FAILURES ===================================
_______________________________ test_first_mark ________________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a4cc820>

    def test_first_mark(item_names_for):
        tests_content = """
        import pytest
    
        def test_1(): pass
    
        @pytest.mark.first
        def test_2(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_2'] == ['test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:43: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_mark0
collected 0 items

============================ no tests ran in 0.00s =============================
________________________________ test_last_mark ________________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a48fca0>

    def test_last_mark(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        def test_1(): pass
    
        def test_2(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_2'] == ['test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:56: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_last_mark0
collected 0 items

============================ no tests ran in 0.00s =============================
____________________________ test_first_last_marks _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439280>

    def test_first_last_marks(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        def test_1(): pass
    
        @pytest.mark.first
        def test_2(): pass
    
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_3', 'test_1']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_2', 'test_3', 'test_1']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:72: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_last_marks0
collected 0 items

============================ no tests ran in 0.00s =============================
_______________________________ test_order_marks _______________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a4394c0>

    def test_order_marks(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=-1)
        def test_1(): pass
    
        @pytest.mark.run(order=-2)
        def test_2(): pass
    
        @pytest.mark.run(order=1)
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_3', 'test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:89: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_order_marks0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_positive _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a48ff70>

    def test_non_contiguous_positive(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=10)
        def test_1(): pass
    
        @pytest.mark.run(order=20)
        def test_2(): pass
    
        @pytest.mark.run(order=5)
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_3', 'test_1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:106: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_positive0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_negative _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439e50>

    def test_non_contiguous_negative(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=-10)
        def test_1(): pass
    
        @pytest.mark.run(order=-20)
        def test_2(): pass
    
        @pytest.mark.run(order=-5)
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_1', 'test_3']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_2', 'test_1', 'test_3']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:123: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_negative0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_inc_zero _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a34b0d0>

    def test_non_contiguous_inc_zero(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=10)
        def test_1(): pass
    
        @pytest.mark.run(order=20)
        def test_2(): pass
    
        @pytest.mark.run(order=5)
        def test_3(): pass
    
        @pytest.mark.run(order=-10)
        def test_4(): pass
    
        @pytest.mark.run(order=-20)
        def test_5(): pass
    
        @pytest.mark.run(order=-5)
        def test_6(): pass
    
        @pytest.mark.run(order=0)
        def test_7(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_7', 'test_3', 'test_1', 'test_2', 'test_5', 'test_4', 'test_6']
E       AssertionError: assert ['test_1', 't...'test_6', ...] == ['test_7', 't...'test_4', ...]
E         At index 0 diff: 'test_1' != 'test_7'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:152: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_inc_zero0
collected 0 items

============================ no tests ran in 0.00s =============================
_________________________ test_non_contiguous_inc_none _________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439ee0>

    def test_non_contiguous_inc_none(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=5)
        def test_1(): pass
    
        @pytest.mark.run(order=0)
        def test_2(): pass
    
        @pytest.mark.run(order=1)
        def test_3(): pass
    
        @pytest.mark.run(order=-1)
        def test_4(): pass
    
        @pytest.mark.run(order=-5)
        def test_5(): pass
    
        def test_6(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_2', 'test_3', 'test_1', 'test_6', 'test_5', 'test_4']
E       AssertionError: assert ['test_1', 't..._5', 'test_6'] == ['test_2', 't..._5', 'test_4']
E         At index 0 diff: 'test_1' != 'test_2'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:177: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_non_contiguous_inc_none0
collected 0 items

============================ no tests ran in 0.00s =============================
____________________________ test_first_mark_class _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a2c9940>

    def test_first_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        def test_1(): pass
    
    
        @pytest.mark.first
        class TestSuite(object):
    
            def test_3(self): pass
    
            def test_2(self): pass
    
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_2', 'test_1']
E       AssertionError: assert ['test_1', 'test_3', 'test_2'] == ['test_3', 'test_2', 'test_1']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:196: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
_____________________________ test_last_mark_class _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a34bd30>

    def test_last_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        class TestSuite(object):
    
            def test_1(self): pass
    
            def test_2(self): pass
    
    
        def test_3(): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 'test_2', 'test_3'] == ['test_3', 'test_1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:214: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_last_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
__________________________ test_first_last_mark_class __________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a2b8ee0>

    def test_first_last_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.last
        class TestLast(object):
    
            def test_1(self): pass
    
            def test_2(self): pass
    
    
        def test_3(): pass
    
    
        @pytest.mark.first
        class TestFirst(object):
    
            def test_4(self): pass
    
            def test_5(self): pass
    
        """
    
>       assert item_names_for(tests_content) == ['test_4', 'test_5', 'test_3', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 't..._4', 'test_5'] == ['test_4', 't..._1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_4'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:241: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_first_last_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
____________________________ test_order_mark_class _____________________________

item_names_for = <function item_names_for.<locals>._item_names_for at 0x7f108a439ee0>

    def test_order_mark_class(item_names_for):
        tests_content = """
        import pytest
    
        @pytest.mark.run(order=-1)
        class TestLast(object):
    
            def test_1(self): pass
    
            def test_2(self): pass
    
    
        @pytest.mark.run(order=0)
        def test_3(): pass
    
    
        @pytest.mark.run(order=-2)
        class TestFirst(object):
    
            def test_4(self): pass
    
            def test_5(self): pass
        """
    
>       assert item_names_for(tests_content) == ['test_3', 'test_4', 'test_5', 'test_1', 'test_2']
E       AssertionError: assert ['test_1', 't..._4', 'test_5'] == ['test_3', 't..._1', 'test_2']
E         At index 0 diff: 'test_1' != 'test_3'
E         Use -v to get the full diff

/build/python-pytest-ordering/src/pytest-ordering-0.6/tests/test_ordering.py:268: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/pytest-of-builduser/pytest-0/test_order_mark_class0
collected 0 items

============================ no tests ran in 0.00s =============================
___________________________ test_markers_registered ____________________________

capsys = <_pytest.capture.CaptureFixture object at 0x7f108a164c10>

    def test_markers_registered(capsys):
        pytest.main(['--markers'])
        out, err = capsys.readouterr()
>       assert '@pytest.mark.run' in out
E       AssertionError: assert '@pytest.mark.run' in '@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/latest/w...: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.\n\n'

tests/test_ordering.py:274: AssertionError
=============================== warnings summary ===============================
tests/test_ordering.py::test_first_mark
tests/test_ordering.py::test_first_last_marks
tests/test_ordering.py::test_first_mark_class
tests/test_ordering.py::test_first_last_mark_class
  /usr/lib/python3.8/site-packages/_pytest/mark/structures.py:323: PytestUnknownMarkWarning: Unknown pytest.mark.first - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

tests/test_ordering.py::test_last_mark
tests/test_ordering.py::test_first_last_marks
tests/test_ordering.py::test_last_mark_class
tests/test_ordering.py::test_first_last_mark_class
  /usr/lib/python3.8/site-packages/_pytest/mark/structures.py:323: PytestUnknownMarkWarning: Unknown pytest.mark.last - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

tests/test_ordering.py::test_order_marks
tests/test_ordering.py::test_non_contiguous_positive
tests/test_ordering.py::test_non_contiguous_negative
tests/test_ordering.py::test_non_contiguous_inc_zero
tests/test_ordering.py::test_non_contiguous_inc_none
tests/test_ordering.py::test_order_mark_class
  /usr/lib/python3.8/site-packages/_pytest/mark/structures.py:323: PytestUnknownMarkWarning: Unknown pytest.mark.run - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    warnings.warn(

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================== 13 failed, 3 passed, 14 warnings in 1.30s ===================

Seems that none of the tests were ordered…

Travis.CI results not showing up in GitHub

For PRs (and probably for master) the results of the Travis test runs do not show up anymore in GitHub (e.g. the green check mark that links to the build details). This has happened somewhere between June 2019 and now.
This does not happen in forks (at least in mine).

This may have to do with this issue - there is a solution in the issue to fix the configuration.

EDIT:
In another repo where I had the same problem I just switched the repo off and on again in the Travis configuration, and it worked again. Maybe this is all it takes... @ftobia - can you try this, please?

Unknown mark warning

When I use the following code to mark a whole test file to be run last:

pytestmark = pytest.mark.last

I always get the following warning at the end of the test run:

=========================== warnings summary ============================
c:\users\rnd10\appdata\local\programs\python\python37-32\lib\site-packages\_pytest\mark\structures.py:332
  c:\users\rnd10\appdata\local\programs\python\python37-32\lib\site-packages\_pytest\mark\structures.py:332: PytestUnknownMarkWarning: Unknown pytest.mark.last - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Note that the ordering works as expected. My version info:

platform win32 -- Python 3.7.3, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
plugins: cov-2.7.1, ordering-0.6, pycharm-0.5.0

support for ordering testcases

I want to be able to order the testcases more than the tests inside the these testcases (unittest.TestCase).

How can I do this using pytest-ordering?

Does Not follow order

Given the order the code is not executing accordingly

import pytest

@pytest.mark.run(order=2)
def test_demo2_methodA(Oneset ,set):
    print("Method a is running demo a")

@pytest.mark.run(order=4)
def test_demo2_methodB(Oneset ,set):
    print("Metod b is running demo b")

@pytest.mark.run(order=6)
def test_demo2_methodC(Oneset ,set):
    print("Method c is running demo c")

@pytest.mark.run(order=5)
def test_demo2_methodD(Oneset ,set):
    print("Metod d is running demo d")

@pytest.mark.run(order=3)
def test_demo2_methodE(Oneset ,set):
    print("Method e is running demo e")

@pytest.mark.run(order=1)
def test_demo2_methodF(Oneset ,set):
    print("Metod f is running demo f") 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.