Comments (20)
The act of using --count 3
changes the nodeids, so with --count 3
, test_foo[10]
no longer exists.
There is a workaround, selecting parameters by keyword instead of nodeid:
(pytest-repeat) $ pytest test_param.py -v -k 'test_foo[10' --count 3
=========================== test session starts ============================
collected 6 items / 3 deselected / 3 selected
test_param.py::test_foo[10-1-3] PASSED [ 33%]
test_param.py::test_foo[10-2-3] PASSED [ 66%]
test_param.py::test_foo[10-3-3] PASSED [100%]
===================== 3 passed, 3 deselected in 0.01s ======================
from pytest-repeat.
We also cannot select individual counts using nodeid:
(pytest-repeat) $ pytest 'test_param.py::test_foo[10-2-3]'
=========================== test session starts ============================
collected 0 items
========================== no tests ran in 0.00s ===========================
ERROR: not found: /Users/okken/projects/pytest-repeat/test_param.py::test_foo[10-2-3]
(no name '/Users/okken/projects/pytest-repeat/test_param.py::test_foo[10-2-3]' in any of [<Module test_param.py>])
I don't think this is a bug, it's an affect of when the counts are added to the nodeid.
And there is the same workaround, use keywords.
(pytest-repeat) $ pytest test_param.py -v -k 'test_foo[10-2-3' --count 3
=========================== test session starts ============================
collected 6 items / 5 deselected / 1 selected
test_param.py::test_foo[10-2-3] PASSED [100%]
===================== 1 passed, 5 deselected in 0.00s ======================
from pytest-repeat.
I'd like to turn this into a feature request for pytest
from pytest-repeat.
does this happen in older pytest versions ?
from pytest-repeat.
It happens in older pytest versions as well (at least until 2.8)
I opened an issue on pytest tracker pytest-dev/pytest#4142 providing a unit test reproducing the bug.
from pytest-repeat.
I wonder if this is also related to adding/removing ::()
(pytest-dev/pytest#4127 (comment)).
from pytest-repeat.
According to pytest-dev/pytest#4142 it likely is not.
from pytest-repeat.
Any progress on this?
from pytest-repeat.
Hey, looks like this happens on the latest pytest version
Do we have any updates on this one?
from pytest-repeat.
This is happening for me with pytest 7.1.0 and pytest-repeat 0.9.1.
Is it related to the way test items are renamed and collected, in that pytest-repeat
modifies the name with extra parameters, and this affects name matching?
For example, with a parametrized test that has one parameter of value 256
, this works (runs once):
$ pytest test.py::test_something[256]
And this works (runs 1000 times but for each parameter, which may be far too many tests):
$ pytest test.py::test_something --count 1000
But to run for a single parameter value, this does not work:
$ pytest test.py::test_something[256] --count 1000
(no name 'test.py::test_something[256]' in any of [<Module test.py>])
And this runs once, but does not repeat:
$ pytest test.py::test_something[256-1-1000] --count 1000
Is there a known workaround, aside from running pytest
over and over in a shell script loop?
from pytest-repeat.
I'm curious if there's been any thinking from the contributors as to how this might work, if we could get it to work?
I come across this so often I'd be happy to help fix it, if I could get in sync with the designers' thoughts...
from pytest-repeat.
Currently no design work on this was done
I believe major internals Changes in pytest itself are necessary to enable this
from pytest-repeat.
Seems to work fine now, with pytest 7.4.2 and pytest-repeat 0.9.2
import pytest
@pytest.mark.parametrize('a', (10, 20))
def test_foo(a):
...
test run:
(pytest-repeat) $ pytest test_param.py -v --count 3
================================= test session starts =================================
collected 6 items
test_param.py::test_foo[10-1-3] PASSED [ 16%]
test_param.py::test_foo[10-2-3] PASSED [ 33%]
test_param.py::test_foo[10-3-3] PASSED [ 50%]
test_param.py::test_foo[20-1-3] PASSED [ 66%]
test_param.py::test_foo[20-2-3] PASSED [ 83%]
test_param.py::test_foo[20-3-3] PASSED [100%]
================================== 6 passed in 0.01s ==================================
from pytest-repeat.
@okken I think the issue remains and itโs that you canโt do this:
$ pytest test_param.py::test_foo[10] -v --count 3
from pytest-repeat.
This one is unfixable without a major change in pytest
from pytest-repeat.
@RonnyPfannschmidt this brings me to a question of "why leave this issue open?"
- We know desired syntax isn't realistic due to the implementation of both
pytest-repeat
and more importantly,pytest
. - So unless
pytest
changes, it's not fixable. - There is a workaround, use keywords.
So why keep this open?
from pytest-repeat.
Also just swapped "bug" for "enhancement", because there is a way to do the requested action, just not with the desired syntax.
from pytest-repeat.
I personally wouldn't know how to describe the feature request. Partly because I don't fully understand the limitations of pytest that are the problem. But I agree.
I do think, at the very least, this issue should remain open until the workaround is documented and easy to find.
from pytest-repeat.
@DavidAntliff Does the -k
workaround work for your use case?
from pytest-repeat.
@DavidAntliff Does the
-k
workaround work for your use case?
Yes, that works for me - thank you. I've been looking for a way to do this for a while.
from pytest-repeat.
Related Issues (20)
- Any way to access the current iteration count? HOT 3
- 0.9.1: pytest is failing HOT 2
- Pytest html log will disappear with pytest-repeat HOT 1
- "Fatal Python error: Cannot recover from stack overflow" When Using Intellij Debugger.
- Merged pytest-repeat and pytest-stress
- How do I get the current number of repetitions in the setup_class function HOT 4
- Organize pytest arguments into a group HOT 2
- this plugin can repeat cases after fliter by mark ? HOT 2
- document --repeat-scope in README HOT 1
- Make PyPI README prettier HOT 1
- update test matrix to Python 3.11 and pytest 7 HOT 1
- switch to pyproject.toml HOT 6
- shift CI to GH Actions HOT 1
- I'd like to help with this project HOT 4
- notes/question on release process HOT 14
- --count only executing once HOT 2
- Option to retry N times until test passes HOT 1
- Running pytest-repeat together with pytest-randomly does not give new random seed HOT 2
- Add timeout flag, to timeout test run after some number of minutes HOT 5
- repeating a test spams pytest cache HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytest-repeat.