pytest-dev / pytest-describe Goto Github PK

View Code? Open in Web Editor NEW

202.0 202.0 17.0 76 KB

Describe-style plugin for the pytest framework

License: MIT License

Python 100.00%

pytest pytest-plugin python testing

pytest-describe's People

Contributors

Stargazers

Watchers

Forkers

asmundg nitrodist miebach eliasdorneles mattcl rebeling pchomik casimiro kondratyev-nv mvpkeydown merc1031 dylancromer jacebrowning bpreskit ropdebee bluetech isabella232

pytest-describe's Issues

Marker "inheritance" not consistent with pytest behaviour

When markers are "stacked", pytest somehow keeps all markers, whereas pytest-describe throws away all but one instance of the markers. This has been reported in #23 for parametrize, but the same issue is applicable to all markers, also custom ones.

Here's an actual test case to exemplify it:

import pytest

@pytest.fixture()
def get_marks(request):
    return [mark.args[0] for mark in request.node.iter_markers(name='my_mark')]

@pytest.mark.my_mark('foo')
class TestMark:

    def test_inherited_from_parent(self, get_marks):
        assert get_marks == ['foo']

    @pytest.mark.my_mark('bar')
    @pytest.mark.my_mark('baz')
    def test_all_marks_are_chained(self, get_marks):
        assert get_marks == ['baz', 'bar', 'foo']


@pytest.mark.my_mark('foo')
def describe_marks():
    def it_is_inherited_from_describe_block(get_marks):
        assert get_marks == ['foo']

    @pytest.mark.my_mark('bar')
    @pytest.mark.my_mark('baz')
    def all_marks_are_chained(get_marks):
        assert get_marks == ['baz', 'bar', 'foo']

The final test describe_marks.all_marks_are_available fails, with the actual value for get_marks being ['bar']. They should instead be stacked.

Of course, this is a contrived example, but there's actual use cases for stacked/inherited markers, like the following:

@pytest.mark.usefixtures('fixture_with_side_effects')
def describe_something():
    @pytest.mark.usefixtures('other_fixture_with_side_effects')
    def it_does_something():
         ...

    def it_does_something_else():
         # No need for 'other_fixture_with_side_effects' here
         ...

Python version: 3.8.2
pytest version: 6.1.2
pytest-describe version: latest master

TL;DR: Marker stacking/chaining/inheritance handling isn't consistent with pytest's builtin behaviour.

Test suite requires `py` package

Now that pytest now longer depends on py, the test suite fails with its current depset. I think adding py to test dependencies would work for the time but ideally I think it should be possible to easily replace py there.

$ tox -e py310
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: prepare_metadata_for_build_wheel> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /usr/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py310: install_package_deps> python -I -m pip install 'pytest>=4.0.0'
py310: install_package> python -I -m pip install --force-reinstall --no-deps /tmp/pytest-describe/.tox/.tmp/package/1/pytest-describe-2.0.1.tar.gz
py310: commands[0]> pytest -rw
========================================================= test session starts =========================================================
platform linux -- Python 3.10.9, pytest-7.2.0, pluggy-1.0.0
cachedir: .tox/py310/.pytest_cache
rootdir: /tmp/pytest-describe, configfile: tox.ini
plugins: describe-2.0.1
collected 30 items                                                                                                                    

test/test_collect.py FF                                                                                                         [  6%]
test/test_custom_prefix.py F                                                                                                    [ 10%]
test/test_fixtures.py FFF                                                                                                       [ 20%]
test/test_marks.py FFFFFFFFFFFFF                                                                                                [ 63%]
test/test_output.py F                                                                                                           [ 66%]
test/test_shared.py FFFFFFF                                                                                                     [ 90%]
test/test_simple_execution.py FFF                                                                                               [100%]

============================================================== FAILURES ===============================================================
____________________________________________________________ test_collect _____________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-mgorny/pytest-6/test_collect0')>

    def test_collect(testdir):
        a_dir = testdir.mkpydir('a_dir')
>       a_dir.join('test_a.py').write(py.code.Source("""
            def describe_something():
                def is_foo():
                    pass
                def can_bar():
                    pass
                def _not_a_test():
                    pass
            def describe_something_else():
                def describe_nested():
                    def a_test():
                        pass
            def foo_not_collected():
                pass
            def test_something():
                pass
        """))
E       AttributeError: module 'py' has no attribute 'code'

/tmp/pytest-describe/test/test_collect.py:11: AttributeError
[...]

[enhancement] use a DocString to enhance the test report description

Hi,
I'm being using your awesome pytest-describe-it lib from a while and I love it because it allows me to have a multi-dimensional test file like in Jest 😍.
The unique thing I miss here is the possibility to tune the test report with a more descriptive title.
The actual behavior is this one:

I'd love to see instead this one:

Maybe it looks like a naive feature, but it'd add a certain degree of customization that would make your lib even more attractive.
thanks in advance.
Mike

Transfer repository to pytest-dev

As discussed in #26, it was decided to move the repository to pytest-dev.

@jacebrowning and @Cito volunteered to act as maintainers to help @ropez keep the project alive.

We have the necessary two +1 from pytest-dev developers to start the transfer.

The process is described here: Submitting Plugins to pytest-dev

I have created this issue to track the transfer and #28 and #29 to fulfill some missing requirements.

Support pytest version >= 4.5

It seems that some of the marking behaviour in pytest==4.5 has changed which breaks pytest-describe. Here's the testing output from pytest==4.4:

(env) james@basalt:~/code/git/pytest-describe$ pip install pytest==4.4
Collecting pytest==4.4
  Using cached https://files.pythonhosted.org/packages/7e/16/83b2a35c427b838df9836c9e7e4ae6dfbcbdea643db44652f693b1c57d70/pytest-4.4.0-py2.py3-none-any.whl
Requirement already satisfied: pluggy>=0.9 in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: py>=1.5.0 in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: more-itertools>=4.0.0; python_version > "2.7" in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: atomicwrites>=1.0 in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: attrs>=17.4.0 in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: setuptools in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: six>=1.10.0 in ./env/lib/python3.6/site-packages (from pytest==4.4)
Requirement already satisfied: importlib-metadata>=0.12 in ./env/lib/python3.6/site-packages (from pluggy>=0.9->pytest==4.4)
Requirement already satisfied: zipp>=0.5 in ./env/lib/python3.6/site-packages (from importlib-metadata>=0.12->pluggy>=0.9->pytest==4.4)
Installing collected packages: pytest
  Found existing installation: pytest 4.5.0
    Uninstalling pytest-4.5.0:
      Successfully uninstalled pytest-4.5.0
Successfully installed pytest-4.4.0
(env) james@basalt:~/code/git/pytest-describe$ py.test
============================================================ test session starts ============================================================
platform linux -- Python 3.6.7, pytest-4.4.0, py-1.8.0, pluggy-0.12.0
rootdir: /home/james/code/git/pytest-describe
plugins: describe-0.12.0
collected 22 items                                                                                                                          

test/test_collect.py ..                                                                                                               [  9%]
test/test_custom_prefix.py .                                                                                                          [ 13%]
test/test_fixtures.py ...                                                                                                             [ 27%]
test/test_marks.py .....                                                                                                              [ 50%]
test/test_output.py .                                                                                                                 [ 54%]
test/test_shared.py .......                                                                                                           [ 86%]
test/test_simple_execution.py ...                                                                                                     [100%]

========================================================= 22 passed in 1.14 seconds =========================================================

And 4.5:

(env) james@basalt:~/code/git/pytest-describe$ py.test -v
============================================================ test session starts ============================================================
platform linux -- Python 3.6.7, pytest-4.5.0, py-1.8.0, pluggy-0.12.0 -- /home/james/code/git/pytest-describe/env/bin/python3
cachedir: .pytest_cache
rootdir: /home/james/code/git/pytest-describe
plugins: describe-0.12.0
collected 22 items                                                                                                                          

test/test_collect.py::test_collect PASSED                                                                                             [  4%]
test/test_collect.py::test_describe_evaluated_once PASSED                                                                             [  9%]
test/test_custom_prefix.py::test_collect_custom_prefix PASSED                                                                         [ 13%]
test/test_fixtures.py::test_can_access_local_fixture PASSED                                                                           [ 18%]
test/test_fixtures.py::test_can_access_fixture_from_nested_scope PASSED                                                               [ 22%]
test/test_fixtures.py::test_local_fixture_overrides PASSED                                                                            [ 27%]
test/test_marks.py::test_special_marks PASSED                                                                                         [ 31%]
test/test_marks.py::test_keywords FAILED                                                                                              [ 36%]
test/test_marks.py::test_marks FAILED                                                                                                 [ 40%]
test/test_marks.py::test_module_marks FAILED                                                                                          [ 45%]
test/test_marks.py::test_mark_at_describe_function FAILED                                                                             [ 50%]
test/test_output.py::test_verbose_output PASSED                                                                                       [ 54%]
test/test_shared.py::test_shared_behavior PASSED                                                                                      [ 59%]
test/test_shared.py::test_multiple_shared_behaviors PASSED                                                                            [ 63%]
test/test_shared.py::test_fixture PASSED                                                                                              [ 68%]
test/test_shared.py::test_override_fixture PASSED                                                                                     [ 72%]
test/test_shared.py::test_name_mangling PASSED                                                                                        [ 77%]
test/test_shared.py::test_nested_name_mangling PASSED                                                                                 [ 81%]
test/test_shared.py::test_evaluated_once PASSED                                                                                       [ 86%]
test/test_simple_execution.py::test_can_pass PASSED                                                                                   [ 90%]
test/test_simple_execution.py::test_can_fail PASSED                                                                                   [ 95%]
test/test_simple_execution.py::test_can_fail_and_pass PASSED                                                                          [100%]

================================================================= FAILURES ==================================================================
_______________________________________________________________ test_keywords _______________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-james/pytest-10/test_keywords0')>

    def test_keywords(testdir):
        a_dir = testdir.mkpydir('a_dir')
        a_dir.join('test_a.py').write(py.code.Source("""
            import pytest
            def describe_a():
                @pytest.mark.foo
                def foo_test():
                    pass
                @pytest.mark.bar
                def bar_test():
                    pass
        """))
    
        result = testdir.runpytest('-k', 'foo')
>       assert_outcomes(result, passed=1, deselected=1)

/home/james/code/git/pytest-describe/test/test_marks.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = <RunResult ret=0 len(stdout.lines)=20 len(stderr.lines)=1 duration=0.04s>, expected = {'deselected': 1, 'passed': 1}
o = {'deselected': 1, 'passed': 1, 'warnings': 2}

    def assert_outcomes(result, **expected):
        o = result.parseoutcomes()
        del o['seconds']
    
        try:
            del o['pytest-warnings']
        except KeyError:
            pass
    
>       assert o == expected
E       AssertionError

/home/james/code/git/pytest-describe/test/util.py:10: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/pytest-of-james/pytest-10/test_keywords0
plugins: describe-0.12.0
collected 2 items / 1 deselected / 1 selected

a_dir/test_a.py .                                                        [100%]

=============================== warnings summary ===============================
/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.foo - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.bar - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============== 1 passed, 1 deselected, 2 warnings in 0.01 seconds ==============
________________________________________________________________ test_marks _________________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-james/pytest-10/test_marks0')>

    def test_marks(testdir):
        a_dir = testdir.mkpydir('a_dir')
        a_dir.join('test_a.py').write(py.code.Source("""
            import pytest
            def describe_a():
                @pytest.mark.foo
                def foo_test():
                    pass
                @pytest.mark.bar
                def bar_test():
                    pass
        """))
    
        result = testdir.runpytest('-m', 'foo')
>       assert_outcomes(result, passed=1, deselected=1)

/home/james/code/git/pytest-describe/test/test_marks.py:64: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = <RunResult ret=0 len(stdout.lines)=20 len(stderr.lines)=1 duration=0.04s>, expected = {'deselected': 1, 'passed': 1}
o = {'deselected': 1, 'passed': 1, 'warnings': 2}

    def assert_outcomes(result, **expected):
        o = result.parseoutcomes()
        del o['seconds']
    
        try:
            del o['pytest-warnings']
        except KeyError:
            pass
    
>       assert o == expected
E       AssertionError

/home/james/code/git/pytest-describe/test/util.py:10: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/pytest-of-james/pytest-10/test_marks0
plugins: describe-0.12.0
collected 2 items / 1 deselected / 1 selected

a_dir/test_a.py .                                                        [100%]

=============================== warnings summary ===============================
/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.foo - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.bar - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============== 1 passed, 1 deselected, 2 warnings in 0.01 seconds ==============
_____________________________________________________________ test_module_marks _____________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-james/pytest-10/test_module_marks0')>

    def test_module_marks(testdir):
        a_dir = testdir.mkpydir('a_dir')
        a_dir.join('test_a.py').write(py.code.Source("""
            import pytest
            pytestmark = [ pytest.mark.foo ]
            def describe_a():
                pytestmark = [ pytest.mark.bar ]
                def describe_b():
                    def a_test():
                        pass
        """))
    
        result = testdir.runpytest('-m', 'foo')
>       assert_outcomes(result, passed=1)

/home/james/code/git/pytest-describe/test/test_marks.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = <RunResult ret=0 len(stdout.lines)=20 len(stderr.lines)=1 duration=0.04s>, expected = {'passed': 1}
o = {'passed': 1, 'warnings': 2}

    def assert_outcomes(result, **expected):
        o = result.parseoutcomes()
        del o['seconds']
    
        try:
            del o['pytest-warnings']
        except KeyError:
            pass
    
>       assert o == expected
E       AssertionError

/home/james/code/git/pytest-describe/test/util.py:10: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/pytest-of-james/pytest-10/test_module_marks0
plugins: describe-0.12.0
collected 1 item

a_dir/test_a.py .                                                        [100%]

=============================== warnings summary ===============================
/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.foo - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.bar - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
===================== 1 passed, 2 warnings in 0.01 seconds =====================
______________________________________________________ test_mark_at_describe_function _______________________________________________________

testdir = <Testdir local('/tmp/pytest-of-james/pytest-10/test_mark_at_describe_function0')>

    def test_mark_at_describe_function(testdir):
        a_dir = testdir.mkpydir('a_dir')
        a_dir.join('test_a.py').write(py.code.Source("""
            import pytest
            @pytest.mark.foo
            def describe_foo():
                def describe_a():
                    def a_test():
                        pass
                @pytest.mark.bar
                def b_test():
                    pass
        """))
    
        result = testdir.runpytest('-m', 'foo')
>       assert_outcomes(result, passed=2)

/home/james/code/git/pytest-describe/test/test_marks.py:98: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = <RunResult ret=0 len(stdout.lines)=20 len(stderr.lines)=1 duration=0.04s>, expected = {'passed': 2}
o = {'passed': 2, 'warnings': 2}

    def assert_outcomes(result, **expected):
        o = result.parseoutcomes()
        del o['seconds']
    
        try:
            del o['pytest-warnings']
        except KeyError:
            pass
    
>       assert o == expected
E       AssertionError

/home/james/code/git/pytest-describe/test/util.py:10: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.5.0, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/pytest-of-james/pytest-10/test_mark_at_describe_function0
plugins: describe-0.12.0
collected 2 items

a_dir/test_a.py ..                                                       [100%]

=============================== warnings summary ===============================
/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.foo - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324
  /home/james/code/git/pytest-describe/env/lib/python3.6/site-packages/_pytest/mark/structures.py:324: PytestUnknownMarkWarning: Unknown pytest.mark.bar - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
===================== 2 passed, 2 warnings in 0.02 seconds =====================
==================================================== 4 failed, 18 passed in 1.18 seconds ====================================================

Not sure what it would take to properly fix this, aside from the easy patch:

$ git diff
diff --git a/setup.py b/setup.py
index e862a7d..c77b755 100644
--- a/setup.py
+++ b/setup.py
@@ -10,7 +10,7 @@ setup(
     author_email='[email protected]',
     license='MIT license',
     install_requires=[
-        'pytest>=2.6.0',
+        'pytest >=2.6.0, <4.5',
     ],
     entry_points={
         'pytest11': [

but I thought I'd let you know.

Are you planning to make this work for python 3?

Getting "missing 1 required positional argument:" on describes_

Hi, could you take a look to the following code?

import pytest


def describe_books():
    @pytest.fixture
    def user():
        return ...

    @pytest.fixture
    def valid_book():
        return ...

    def invalid_book():
        return ...

    def describe_create_book(user):

        def with_valid_book(valid_book):
            ... use user + valid_book fixtures ...

        def with_invalid_book(invalid_book):
            ... use user + invalid_book fixtures ...

Getting this error:

venv/lib/python3.5/site-packages/pytest_describe/plugin.py:73: in collect
    return super(DescribeBlock, self).collect()
venv/lib/python3.5/site-packages/_pytest/python.py:465: in collect
    l.sort(key=lambda item: item.reportinfo()[:2])
venv/lib/python3.5/site-packages/_pytest/python.py:465: in <lambda>
    l.sort(key=lambda item: item.reportinfo()[:2])
venv/lib/python3.5/site-packages/_pytest/python.py:387: in reportinfo
    obj = self.obj
venv/lib/python3.5/site-packages/_pytest/python.py:351: in fget
    self._obj = obj = self._getobj()
venv/lib/python3.5/site-packages/pytest_describe/plugin.py:76: in _getobj
    return self._memoizedcall('_obj', self._importtestmodule)
venv/lib/python3.5/site-packages/_pytest/main.py:315: in _memoizedcall
    res = function()
venv/lib/python3.5/site-packages/pytest_describe/plugin.py:84: in _importtestmodule
    module = make_module_from_function(self.funcobj)
venv/lib/python3.5/site-packages/pytest_describe/plugin.py:29: in make_module_from_function
    module.__dict__.update(trace_function(funcobj))
venv/lib/python3.5/site-packages/pytest_describe/plugin.py:19: in trace_function
    funcobj(*args, **kwargs)
E   TypeError: describe_create_book() missing 1 required positional argument: 'user'

describe_create_book needs to share the user fixture to the internal use cases. What I'm doing wrong?

Create a changelog

Plugins submitted to pytest-dev are required to have a changelog.

We should either create a changelog file, in addition to the README.rst, or we should tag releases and use GitHub releases to describe and list the changes in the releases, and then simply point to the releases list in the README (I prefer the latter).

multicall being deprecated in upcoming pytest release

When running pytest with this plugin, a warning now appears:

================================================ pytest-warning summary ================================================
WI1 .../pytest_describe/plugin.py:101 'pytest_pycollect_makeitem' hook uses deprecated __multicall__ argument

Describe block breaks coverage

Any code only executed by a test function inside a describe block is not reported by coverage:

def foo():
    return False


def describe_foo():
    def test_bar():
        assert foo()

produces:

 » vex tmp py.test --cov /tmp/bug.py --cov-report=term-missing /tmp/bug.py                                                               1 ↵
=============================================================================================== test session starts ================================================================================================
platform linux2 -- Python 2.7.6 -- py-1.4.25 -- pytest-2.6.3
plugins: cov, describe
collected 1 items 

../../../../tmp/bug.py F

===================================================================================================== FAILURES =====================================================================================================
______________________________________________________________________________________________ describe_foo.test_bar _______________________________________________________________________________________________

    def test_bar():
>       assert foo()
E       assert foo()

/tmp/bug.py:7: AssertionError
--------------------------------------------------------------------------------- coverage: platform linux2, python 2.7.6-final-0 ----------------------------------------------------------------------------------
Name       Stmts   Miss  Cover   Missing
----------------------------------------
/tmp/bug       5      3    40%   2, 6-7
============================================================================================= 1 failed in 0.01 seconds =============================================================================================

Nicer test output?

It'd make for a nicer BDD workflow if we could see gherkin-style red/green colourised reporting with checkmarks. To use the example provided in the docs:

$ pytest --spec-output
--------------------------------------------------------------------------------------
- path.to.spec:

List:
  Append:
    ✓ adds to end of list
  Remove:
    ✓ removes items from list
    ✗ fails

This would be even nicer with a --watch flag which observes new/changed files recursively.

Context support

Does pytest-describe support contexts?

From README:

def describe_my_function():
    def with_default_arguments():
    def with_some_other_arguments():
    def it_throws_exception():
    def it_handles_exception():

The way I'm trying to use this is:

def describe_ensure_bits_count():
    def with_bits_count_more_than_number_bit_count():
        def it_returns_unaltered_specified_number():
            assert_that(ensure_bits_count(10, 16), is_(10))

This way, the test case is not discovered.

But when I do it like this:

def describe_ensure_bits_count():
    def it_returns_unaltered_specified_number_with_bits_count_more_than_number_bit_count():
        assert_that(ensure_bits_count(10, 16), is_(10))

it works like a charm.

Am I trying to use it in a wrong way, or is this functionality just not supported?

Shared behavior should have a recursive partial override behavior

Repro

from pytest import fixture
from pytest_describe import behaves_like

class Duck(object):
    def speak(self):
        return "quack"

class Dog(object):
    def speak(self):
        return "bark"

def a_duck():
    def describe_speak():
        def it_quacks_once(subject):
            assert subject.speak() == "quack"

        def it_quacks(subject):
            assert subject.speak() == "quack"

@behaves_like(a_duck)
def describe_dog():
    @fixture
    def subject():
        return Dog()

    def describe_speak():
        def it_quacks(subject):
            assert subject.speak() == "bark"

Expected Outcome

1 failed 1 passed

Actual Outcome

1 passed

What happens in this scenario is that all tests under describe_speak within the shared behavior are overridden by the single test with the describe_speak under describe_dog.

we should be able to write additional tests within existing nested describe blocks that we get from the shared behavior. The only time a shared behavior test should be overridden is when it has the exact same name/nesting (when compared to test inheritance is very similar).

This seems a little be more tricky to implement. @ropez what do you think of this?

Roadmap for version 1 and 2

I have created this issue to discuss the next releases and versioning policy.

My suggestion is to create first a version 1.0 that will support Py 2.7 + 3.5 - 3.8 and pytest 2.x - 5.x, for those who need to test legacy software.

After 1.0 has been released, we immediately publish a version 2.0 that will require Python 3.6+ and newer pytest versions. This will allow to simplify and clean the code and reduce the maintenance burden.

If everybody agrees, I will publish such a 1.0 release this weekend and start working on 2.0.

object has no attribute '_memoizedcall' on pytest 6.x

Hi there! I started to use your library for one of my new projects and faced up with the exception like this:

tests/api/personal/test_change_password.py:None (tests/api/personal/test_change_password.py)
.venv/lib/python3.9/site-packages/pytest_describe/plugin.py:134: in _getobj
    return self._memoizedcall('_obj', self._importtestmodule)
E   AttributeError: 'DescribeBlock' object has no attribute '_memoizedcall'

I wrote a little example:

import pytest


@pytest.fixture()
def truelness():
    return True

def describe_pytest_describe():
    def should_work_on_pytest_6():
        assert True


def describe_pytest_describe_with_fixture(truelness):
    def should_work_on_pytest_6():
        assert truelness is True

output:

================================================================================================================ ERRORS ================================================================================================================
________________________________________________________________________________________________ ERROR collecting tests/test_example.py ________________________________________________________________________________________________
.venv/lib/python3.9/site-packages/pytest_describe/plugin.py:134: in _getobj
    return self._memoizedcall('_obj', self._importtestmodule)
E   AttributeError: 'DescribeBlock' object has no attribute '_memoizedcall'

During handling of the above exception, another exception occurred:
.venv/lib/python3.9/site-packages/pytest_describe/plugin.py:136: in _getobj
    return self._importtestmodule()
.venv/lib/python3.9/site-packages/pytest_describe/plugin.py:144: in _importtestmodule
    module = make_module_from_function(self.funcobj)
.venv/lib/python3.9/site-packages/pytest_describe/plugin.py:37: in make_module_from_function
    module.__dict__.update(trace_function(funcobj))
.venv/lib/python3.9/site-packages/pytest_describe/plugin.py:19: in trace_function
    funcobj(*args, **kwargs)
E   TypeError: describe_pytest_describe_with_fixture() missing 1 required positional argument: 'truelness'
======================================================================================================= short test summary info ========================================================================================================
ERROR tests/test_example.py - TypeError: describe_pytest_describe_with_fixture() missing 1 required positional argument: 'truelness'

It looks like a problem with this specific commit in pytest: pytest-dev/pytest@ceb0165

Could you please take a look?

TypeError: described_as() takes at least 2 arguments (0 given)

Don't know what to say. Here is some info:

Test session starts (platform: linux2, Python 2.7.8, pytest 2.8.2, pytest-sugar 0.5.1)
django settings: project.settings (from ini file)
rootdir: /home/l/projects/contextads, inifile: pytest.ini
plugins: describe-0.10.2, sugar-0.5.1, django-2.9.1


../../.virtualenvs/lc/local/lib/python2.7/site-packages/pytest_describe/plugin.py:76: in _getobj
    return self._memoizedcall('_obj', self._importtestmodule)
../../.virtualenvs/lc/local/lib/python2.7/site-packages/_pytest/main.py:317: in _memoizedcall
    res = function()
../../.virtualenvs/lc/local/lib/python2.7/site-packages/pytest_describe/plugin.py:84: in _importtestmodule
    module = make_module_from_function(self.funcobj)
../../.virtualenvs/lc/local/lib/python2.7/site-packages/pytest_describe/plugin.py:29: in make_module_from_function
    module.__dict__.update(trace_function(funcobj))
../../.virtualenvs/lc/local/lib/python2.7/site-packages/pytest_describe/plugin.py:19: in trace_function
    funcobj(*args, **kwargs)
E   TypeError: described_as() takes at least 2 arguments (0 given)

Tests fail with pytest 2.7

I haven't tried using it with pytest 2.7 yet. Just seen that the test suite for the plugin itself fails with the latest pytest.

consider moving under pytest-dev

Please consider moving under pytest-dev
pytest.org/latest/contributing.html#submit-a-plugin-co-develop-pytest

funcargs on describe functions

Hey there! First of all, thanks for sharing this plugin. I've been using it in some projects and I really like it. However, there's a feature I'm missing: be able to add funcargs to describe functions;

Consider the following scenario:

def describe_calculate_something_hard(calculator):
    def when_input_is_x(x, expected_result):
        assert calculator.calculate(x) == expected_result

If I do this, the calculator argument is not handled by the pytest. Maybe I'm just using it wrong.
But if this is a missing feature, I would like to have it. I could even implement it by myself, but some directions would be much appreciated.

Best regards,
casimiro.

Describe plugin does not handle parameters when names are not specified as str

I'm observing that pytest-describe apparently does not correctly handle cases when you use @pytest.mark.parametrize for a test inside a describe block, but specify argnames as a tuple or list instead of as a str. According to the pytest API reference, this should be valid.

For an MWE, consider test_bad.py:

import pytest

@pytest.mark.parametrize(
    ('foo', 'bar'),
    [
        ('apple', 'banana'),
        ('pancake', 'waffle'),
    ]
)
def test_with_multiple_params(foo, bar):
    assert foo < bar

def describe_describe():
    @pytest.mark.parametrize(
        ('foo', 'bar'),
        [
            ('apple', 'banana'),
            ('pancake', 'waffle'),
        ]
    )
    def test_with_multiple_params(foo, bar):
        assert foo < bar

If we test this, we get:

▶ /tmp/py3.5-land/bin/pytest /tmp/test/test_bad.py -vv
==================================================================================================================================================== test session starts =====================================================================================================================================================
platform linux -- Python 3.5.3, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- /tmp/py3.5-land/bin/python3.5
cachedir: .pytest_cache
rootdir: /tmp/test
plugins: describe-1.0.0
collected 0 items / 1 error                                                                                                                                                                                                                                                                                                  

=========================================================================================================================================================== ERRORS ===========================================================================================================================================================
________________________________________________________________________________________________________________________________________________ ERROR collecting test_bad.py ________________________________________________________________________________________________________________________________________________
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:134: in _getobj
    return self._memoizedcall('_obj', self._importtestmodule)
E   AttributeError: 'DescribeBlock' object has no attribute '_memoizedcall'

During handling of the above exception, another exception occurred:
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:136: in _getobj
    return self._importtestmodule()
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:145: in _importtestmodule
    copy_markinfo(module, self.funcobj)
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:63: in copy_markinfo
    merge_pytestmark(obj, funcobj)
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:86: in merge_pytestmark
    marks.update(pytestmark_dict(obj))
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:103: in pytestmark_dict
    return {pytestmark_name(mark): mark for mark in marks}
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:103: in <dictcomp>
    return {pytestmark_name(mark): mark for mark in marks}
/tmp/py3.5-land/lib/python3.5/site-packages/pytest_describe/plugin.py:94: in pytestmark_name
    name += '-' + mark.args[0]
E   TypeError: Can't convert 'tuple' object to str implicitly
================================================================================================================================================== short test summary info ===================================================================================================================================================
ERROR ../../../../../../../../tmp/test/test_bad.py - TypeError: Can't convert 'tuple' object to str implicitly
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================================================================================================================== 1 error in 0.08s ======================================================================================================================================================

However, if I instead use a string for the argnames of the parametrize inside the describe block, the test succeeds as expected.
test_good.py

import pytest                                                                                                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                                                                              
@pytest.mark.parametrize(                                                                                                                                                                                                                                                                                                     
    ('foo', 'bar'),                                                                                                                                                                                                                                                                                                           
    [                                                                                                                                                                                                                                                                                                                         
        ('apple', 'banana'),                                                                                                                                                                                                                                                                                                  
        ('pancake', 'waffle'),                                                                                                                                                                                                                                                                                                
    ]                                                                                                                                                                                                                                                                                                                         
)                                                                                                                                                                                                                                                                                                                             
def test_with_multiple_params(foo, bar):                                                                                                                                                                                                                                                                                      
    assert foo < bar                                                                                                                                                                                                                                                                                                          
                                                                                                                                                                                                                                                                                                                              
def describe_describe():                                                                                                                                                                                                                                                                                                      
    @pytest.mark.parametrize(                                                                                                                                                                                                                                                                                                 
        'foo,bar',                                                                                                                                                                                                                                                                                                            
        [                                                                                                                                                                                                                                                                                                                     
            ('apple', 'banana'),                                                                                                                                                                                                                                                                                              
            ('pancake', 'waffle'),                                                                                                                                                                                                                                                                                            
        ]                                                                                                                                                                                                                                                                                                                     
    )                                                                                                                                                                                                                                                                                                                         
    def test_with_multiple_params(foo, bar):                                                                                                                                                                                                                                                                                  
        assert foo < bar

Result:

▶ /tmp/py3.5-land/bin/pytest /tmp/test/test_good.py -vv                       
==================================================================================================================================================== test session starts =====================================================================================================================================================
platform linux -- Python 3.5.3, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- /tmp/py3.5-land/bin/python3.5
cachedir: .pytest_cache
rootdir: /tmp/test
plugins: describe-1.0.0
collected 4 items                                                                                                                                                                                                                                                                                                            

../../../../../../../../tmp/test/test_good.py::describe_describe::test_with_multiple_params[apple-banana] PASSED                                                                                                                                                                                                       [ 25%]
../../../../../../../../tmp/test/test_good.py::describe_describe::test_with_multiple_params[pancake-waffle] PASSED                                                                                                                                                                                                     [ 50%]
../../../../../../../../tmp/test/test_good.py::test_with_multiple_params[apple-banana] PASSED                                                                                                                                                                                                                          [ 75%]
../../../../../../../../tmp/test/test_good.py::test_with_multiple_params[pancake-waffle] PASSED                                                                                                                                                                                                                        [100%]

===================================================================================================================================================== 4 passed in 0.02s ======================================================================================================================================================

To be clear about versions:

▶ /tmp/py3.5-land/bin/pip freeze                       
attrs==19.3.0
importlib-metadata==1.7.0
more-itertools==8.4.0
packaging==20.4
pathlib2==2.3.5
pluggy==0.13.1
py==1.9.0
pyparsing==2.4.7
pytest==5.4.3
pytest-describe==1.0.0
six==1.15.0
wcwidth==0.2.5
zipp==1.2.0

PytestDeprecationWarning regarding DescribeBlock with pytest 5.4

With the latest pytest 5.4 I start getting the following warnings:

.tox/py36/lib/site-packages/pytest_describe/plugin.py:149: PytestDeprecationWarning:
  direct construction of DescribeBlock has been deprecated,
  please use DescribeBlock.from_parent return DescribeBlock(obj, collector)

Shared behaviours

I have a use case where I would like to use describe methods inside of a class in order to avoid test duplication by allowing me to use inheritance. Though, using classes is not strictly required. The end goal is to get to something like:

https://www.relishapp.com/rspec/rspec-core/docs/example-groups/shared-examples

proposed syntax without test class inheritance

@pytest_describe.shared
def share_a_measurable_object()

   def describe_include():
      def with_an_item_in_the_collection_returns_true(subject):
      def with_an_item_not_in_the_collection_returns_false(subject):

@pytest_describe.behaves_like('a measurable object')
def describe_my_special_array():

   @pytest.fixture
   def subject():
       return MySpecialArray()

@pytest_describe.behaves_like('a measurable object')
def describe_another_array():

   @pytest.fixture
   def subject():
         return AnotherArray()

   def it_does_something_special(subject):

Support pytest version >= 4.1

pytest-describe plugin is not working with pytest 4.0 or later.

=========================================== ERRORS ============================================
_______________ ERROR collecting test/test_samples/pytest/test/describe_test.py _______________
.tox/py27/local/lib/python2.7/site-packages/pytest_describe/plugin.py:110: in _getobj
    return self._importtestmodule()
.tox/py27/local/lib/python2.7/site-packages/pytest_describe/plugin.py:119: in _importtestmodule
    copy_markinfo(module, self.funcobj)
.tox/py27/local/lib/python2.7/site-packages/pytest_describe/plugin.py:60: in copy_markinfo
    from _pytest.mark import MarkInfo
E   ImportError: cannot import name MarkInfo
!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================== 1 error in 0.75 seconds ===================================

This error appears for the test from sample

import pytest


def describe_list():

    @pytest.fixture
    def list():
        return []

    def describe_append():

        def adds_to_end_of_list_passed(list):
            list.append('foo')
            list.append('bar')
            assert list == ['foo', 'bar']

    def describe_remove():

        @pytest.fixture
        def list():
            return ['foo', 'bar']

        def removes_item_from_list_passed(list):
            list.remove('foo')
            assert list == ['bar']

It looks like this happens due to changes in mark API since version 4.0, for example, see https://github.com/pytest-dev/pytest/blob/master/doc/en/deprecations.rst#somefunctionmarkname.

Stacking `@pytest.mark.parametrize` does not work

The pytest docs say that:

To get all combinations of multiple parametrized arguments you can stack parametrize decorators:

import pytest


@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_foo(x, y):
    pass

However, doing so using pytest-describe doesn't seem to work anymore (it did in v0.11.1).

Here's a way to reproduce the problem:

(env) james@basalt:~/code/git/pytest-describe$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   test/test_marks.py

no changes added to commit (use "git add" and/or "git commit -a")

(env) james@basalt:~/code/git/pytest-describe$ git diff
diff --git a/test/test_marks.py b/test/test_marks.py
index 5d708cc..c6a4866 100644
--- a/test/test_marks.py
+++ b/test/test_marks.py
@@ -25,6 +25,12 @@ def test_special_marks(testdir):
             @pytest.mark.parametrize('foo', (1, 2, 3))
             def isint(foo):
                 assert foo == int(foo)
+
+            @pytest.mark.parametrize('foo', (1, 2, 3))
+            @pytest.mark.parametrize('bar', ('a', 'b', 'c'))
+            def isint(foo, bar):
+                assert foo == int(foo)
+                assert bar == str(bar)
     """))
 
     result = testdir.runpytest()

(env) james@basalt:~/code/git/pytest-describe$ py.test -v -k special_marks
============================================================ test session starts ============================================================
platform linux -- Python 3.6.7, pytest-4.4.0, py-1.8.0, pluggy-0.12.0 -- /home/james/code/git/pytest-describe/env/bin/python3
cachedir: .pytest_cache
rootdir: /home/james/code/git/pytest-describe
plugins: describe-0.12.0
collected 22 items / 21 deselected / 1 selected                                                                                             

test/test_marks.py::test_special_marks FAILED                                                                                         [100%]

================================================================= FAILURES ==================================================================
____________________________________________________________ test_special_marks _____________________________________________________________

testdir = <Testdir local('/tmp/pytest-of-james/pytest-14/test_special_marks0')>

    def test_special_marks(testdir):
        a_dir = testdir.mkpydir('a_dir')
        a_dir.join('test_a.py').write(py.code.Source("""
            import pytest
    
            def describe_marks():
                @pytest.mark.xfail
                def xfails():
                    assert False
    
                @pytest.mark.xfail
                def xpasses():
                    pass
    
                @pytest.mark.skipif("0 < 1")
                def skipped():
                    pass
    
                @pytest.mark.parametrize('foo', (1, 2, 3))
                def isint(foo):
                    assert foo == int(foo)
    
                @pytest.mark.parametrize('foo', (1, 2, 3))
                @pytest.mark.parametrize('bar', ('a', 'b', 'c'))
                def isint(foo, bar):
                    assert foo == int(foo)
                    assert bar == str(bar)
        """))
    
        result = testdir.runpytest()
>       assert_outcomes(result, passed=3, xfailed=1, xpassed=1, skipped=1)

/home/james/code/git/pytest-describe/test/test_marks.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = <RunResult ret=1 len(stdout.lines)=41 len(stderr.lines)=1 duration=0.07s>
expected = {'passed': 3, 'skipped': 1, 'xfailed': 1, 'xpassed': 1}, o = {'error': 3, 'skipped': 1, 'xfailed': 1, 'xpassed': 1}

    def assert_outcomes(result, **expected):
        o = result.parseoutcomes()
        del o['seconds']
    
        try:
            del o['pytest-warnings']
        except KeyError:
            pass
    
>       assert o == expected
E       AssertionError

/home/james/code/git/pytest-describe/test/util.py:10: AssertionError
----------------------------------------------------------- Captured stdout call ------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.4.0, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/pytest-of-james/pytest-14/test_special_marks0
plugins: describe-0.12.0
collected 6 items

a_dir/test_a.py xXsEEE                                                   [100%]

==================================== ERRORS ====================================
__________________ ERROR at setup of describe_marks.isint[1] ___________________
file /tmp/pytest-of-james/pytest-14/test_special_marks0/a_dir/test_a.py, line 21
      @pytest.mark.parametrize('foo', (1, 2, 3))
      @pytest.mark.parametrize('bar', ('a', 'b', 'c'))
      def isint(foo, bar):
E       fixture 'bar' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-james/pytest-14/test_special_marks0/a_dir/test_a.py:21
__________________ ERROR at setup of describe_marks.isint[2] ___________________
file /tmp/pytest-of-james/pytest-14/test_special_marks0/a_dir/test_a.py, line 21
      @pytest.mark.parametrize('foo', (1, 2, 3))
      @pytest.mark.parametrize('bar', ('a', 'b', 'c'))
      def isint(foo, bar):
E       fixture 'bar' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-james/pytest-14/test_special_marks0/a_dir/test_a.py:21
__________________ ERROR at setup of describe_marks.isint[3] ___________________
file /tmp/pytest-of-james/pytest-14/test_special_marks0/a_dir/test_a.py, line 21
      @pytest.mark.parametrize('foo', (1, 2, 3))
      @pytest.mark.parametrize('bar', ('a', 'b', 'c'))
      def isint(foo, bar):
E       fixture 'bar' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

/tmp/pytest-of-james/pytest-14/test_special_marks0/a_dir/test_a.py:21
=========== 1 skipped, 1 xfailed, 1 xpassed, 3 error in 0.04 seconds ===========
================================================== 1 failed, 21 deselected in 0.13 seconds ==================================================

You can see that the second parametrized mark doesn't seem to get assigned to a fixture. Is this functionality intended to work, or did it just work by coincidence prior to v0.12?

Fixtures as describe arguments

This has been requested and rejected before (#9, #11, #34) but I've done some investigation and I'd like to share my findings, and reopen the discussion.

Problem setting

Consider this (seemingly simple) scenario (shamelessly pulled from #9):

import pytest


def describe_books():
    @pytest.fixture
    def user():
        return ...

    @pytest.fixture
    def valid_book():
        return ...

    @pytest.fixture
    def invalid_book():
        return ...

    def describe_create_book(user):

        def with_valid_book(valid_book):
            # use user + valid_book fixtures ...

        def with_invalid_book(invalid_book):
            # use user + invalid_book fixtures ...

Seems simple enough, we want to use the user fixture in both with_valid_book and with_invalid_book, so why not add it as a funcarg fixture on the describe_create_book block instead of repeating ourselves? Since eliminating verbosity and repetitions is one of the goals of this plugin, it would make sense for this to be possible, and indeed, as evidenced by the linked issues, people assume this to be the case (me included when I first started using this plugin). However, it isn't, and such usage fails with a type error because of missing arguments.

Let's not focus on the less-than-descriptive error message, since a fix for that would be easy enough. Let's instead focus on whether injecting fixtures as describe block funcargs would be a possibility, and that writing

def describe_foo(fixt):
   def it_barks(other_fixt):
       ...

should be functionally equivalent to writing

def describe_foo():
    def it_barks(fixt, other_fixt):
       ...

Challenges

From previous issues, two main problems have popped up:

Describe blocks are evaluated during test collection, and have no access to fixtures.
New fixture values need to somehow be injected into the test function, i.e. the describe block's locals need to be adjusted before each test is executed.

(Potential) solutions

I'd argue that problem 1 isn't a big deal, considering that code inside the describe block itself isn't actually inside of a test, and thus probably shouldn't make use of the fixtures. Instead, some sort of dummy values could be given as arguments when the describe_ function is executed, assuming (and properly documenting) that these values should not be used outside of the tests. Perhaps some meta-magic with sys.settrace or sys.setprofile could be done to remove those arguments from the outer scope and raise some error when they're accessed.

Problem 2 is somewhat bigger, since we'd need to be able to somehow update the arguments given to the describe block before executing each test within the describe block with the value of the fixture. A straightforward way would be to just rerun the describe block with the new values, but that executes all of the contained code again, which might lead to severe problems. Luckily, embedded functions carry a __closure__ attribute, which is a tuple of "cells" that carry a reference to the locals of the parent function, i.e., the describe block. Since 3.7, the contents of these cells can be mutated. To exemplify:

def describe_block(foo):
    def inner(bar):
        return foo + bar
    return inner

inner = describe_block(2)
inner(2)  # 4
inner.__closure__[0].cell_contents = 4
inner(2)  # 6

Meaning that it could be possible to actually update these values without rerunning the describe block, thus solving problem 2, and perhaps making it possible to automatically inject the fixtures as funcargs.

Notes

This wouldn't be possible in Python 3.6, since cell_contents is a read-only attribute and has only been made writeable since 3.7. I've experimented a bit, but it seems like they really didn't want you to do these sorts of things back then, as there's seemingly no way to instantiate those cell objects from within Python code either, but I may be missing something.
Although the cell type has only been reified since 3.8 (https://docs.python.org/3.8/library/types.html#types.CellType), changing cell_contents is possible since 3.7
I've successfully tested the cell_contents example above on 3.7.9, 3.8.2, and 3.9.0. I'm assuming it works on 3.10 as well, although I haven't checked.
Am I sure this will work? No, but I believe this is the most promising way to get this to work. We'll only know for sure once there's actual code that does this, or fails to do it. I'm hoping to find some time in the near future to submit a WIP PR as a proof-of-concept.
Is this some mad meta hacking? kinda sorta. But the discovery of local test functions is as well 🙂

Pycharm support

Hello all!

I'm testing out the library right now and I'm noticing that if I create describe scenarios in pycharm there doesn't seem to be a way to run single test. For normal pytest or unittest tests I can see a play button that allows running just that test

However when I try to run a pytest-describe scenario it seems I have to run the entire file to get those tests to run

Is there something I can do to have pycharm recognize this? I didn't see a specific pytest-describe plugin

DescribeBlocks not supported out of the box in VSCode Testing

VSCode seems to have an issue with describe blocks, it doesn't know what to do with them and so won't add them to the list of tests (it uses pytest's --collect-only output to discover them).

However it understands test case classes, so there might be a way to get them to work without relying on support from any IDE by wrapping any describe function as a test class ?

So that this:

def describe_me():
    def test_it_works():
       pass

Would look the same as :

class describe_me(unittest.TestCase):
    def test_it_works(self):
       pass

Persistent data among tests with django and pytest

Hello there!
I just opened this question on stack overflow.

Do you have any idea?