bitwizeshift / oxtest Goto Github PK
View Code? Open in Web Editor NEW๐ง A WIP rust unit-testing library
Home Page: https://bitwizeshift.github.io/neotest
License: Apache License 2.0
๐ง A WIP rust unit-testing library
Home Page: https://bitwizeshift.github.io/neotest
License: Apache License 2.0
The definition for neotest
fixtures currently does not allow for
the argument to be of a different refness or mutability (e.g. &Fixture
is disallowed, but Fixture
is okay).
This issue is to track fixing this omission so that the calling convention
is honored based on the way the developer chooses to define their
fixture parameter.
To be able to implement/provide mocking-like functionality, we need a way to test whether expected behavior has occurred.
The conventional way of doing this is with an Expectation
-like object.
This task is to track adding expectations in some manner as a new module in neotest
.
As an extension to #11 , it's sometimes necessary to provide an expectation that certain events happen in a certain order (e.g. A before B, not vice-versa). This task is to track defining a mechanism that allows expectations to be in a forced-sequence.
Having tests return TestResult
(Result<(), Error>
objects), enables having assertions that don't panic, and also makes it possible to return from tests early.
Ideally this would be transparent to the user, so this ticket is to track the following implementation:
-> TestResult
, they must manually call Ok(())
themselves,-> TestResult
, then the test implicitly adds Ok(())
at the endThe current mechanism for subtests simply acts as a series of statements defined in a subtest! { ... }
macro. Although this functionally works, it suffers the significant drawback that it is unable to have custom attributes specified -- such as #[should_panic]or
#[ignore]`, which are needed for testing.
It's unclear what the best direction forward for this is. A couple thoughts come to mind:
[<attribute>]
syntax, e.g. subtest! { |name| [should_panic][ignore] ... }
, this is kind of gross -- but would work.#[neotest::test]
fn test_vec_index() {
let vec: Vec<u8> = Vec::default();
#[neotest::subtest]
#[should_panic]
fn out_of_bounds_panics() {
vec[20];
}
}
vec
is within fn out_of_bounds_panics()
scope, since this typically is not a true closure. The upside is that this has a clear mechanism for identifying attributes, the test ident, etc.The exact approach is yet to be determined.
As an extension to #7, it'd be nice to have "matcher" logic, which can help simplify testing that values match a given expectation in a simplified way. These matchers can be based off of the way other testing libraries are written, such as Catch2 or GTest.
This should also be usable in an assertion, which would ideally generate a custom message. These will also be useful for mocking when we want expectations on types.
The current approach to inputs works, but is a little counter-intuitive due to the combinatoric nature of the arguments. This also suffers when combined with subsections, since cohesion may drop if some subtests don't require inputs while others do.
Thus, this provides a different proposal: rather than using parameters =
, provide an inputs![ ... ]
macro that expands into an array that selects the correct input at runtime from the current __Context
.
This could be implemented in the following way:
__Context
now stores current input state (no longer just the section path).
inputs![...]
macro that expands, effectively, into [...].into_iter().nth(__context.input_index())
inputs![...]
macro being used. This is applied per subtest.This will now require a significant restructure to tests. Having subtests be able to drive parameters means that we no longer have a fixed inputs -> subtests ordering, but rather an arbitrary re-nesting of this.
Adding support for returning TestResult
objects in #6 introduced an unexpected bug: the #[should_panic]
attribute no longer functions correctly. #[should_panic]
apparently only works in functions returning ()
, which raises questions as to the future direction of testing in this library:
Should an assertion library be made that returns Error
objects as #7 suggests? Doing this would mean that #[should_panic]
tests cannot actually work with these assertions
A custom assert_panic!(...)
macro can be added that is implemented in terms of std::panic::handle_unwind
-- however this is documented to only work for panics that perform unwinding. It's unclear if this is also the behavior of #[should_panic]
, or whether #[should_panic]
can actually catch cases that abort.
A decision needs to be made in order to progress here. It's possible that supporting attributes in subtests in #14 might be sufficient for handling this, in that subtests can return ()
whereas other tests return Error
.
Parameterized testing is a staple of any unit-testing framework.
This issue is to track adding support for basing parameterized testing so that tests can operate with different inputs. In particular, the goal of this feature is:
neotest
attribute,a = [1, 2, 3]
and b = [4, 5]
will produce 6 total tests: (1, 4), (1, 5), (2, 4), (2, 5), (3, 4), (3, 5)
.neotest
parameters to be in the same order as function parametersThis is a larger task that will make use of #11 in order to generate/define mock functionality.
The exact syntax/specifics are yet to be determined, but the idea is to support something like:
trait Turtle {
fn pen_down(&mut self);
fn pen_up(&mut self);
}
#[neotest::mock]
struct MockTurtle;
impl Turtle for MockTurtle {
// Set this as a mock function
#[neotest::mock_fn]
fn pen_down(&mut self);
#[neotest::mock_fn]
fn pen_up(&mut self);
}
// ...
fn test_draw_circle() {
let mut turtle = MockTurtle::new();
let painter = Painter::new(&mut turtle);
subtest!{ |calls_pen_down|
let expectation = neotest::expect_call!( turtle.pen_down() => Times(1) );
painter.draw_circle(0, 0, 10);
assert!(expectation.satisfied());
}
}
(Syntax/names are subject to change, but this is the general idea).
Blocked by #6
The builtin assertions are lackluster for testing in that they only cover assert_eq
, assert_ne
, assert
, and assert_match
(nightly). Ideally we should have the full collection of unit test assertion helpers like mature unit test libraries have, such as GTest.
This is to track adding assertions for eq
, ne
, true
, false
, regex
, match
, etc.
These should make use of the test-return-types, which also enables adding simple succeed!()
and fail!()
macros.
As an extension to #7, it can be useful in testing to provide non-fatal assertions for tests, which can allow the code to progress after failure.
The current proposed implementation is:
__Context
to support storing assertion errors__context
for failures (this is necessary since declarative macros will fail due to __context
not being visible in the current scope)There should also be considerable code-sharing between #7 and this, since anything we want assert
s of, we should also offer expect
s.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.