Git Product home page Git Product logo

test-case's Introduction

Crates.io Crates.io Docs.rs MIT License Build Status Maintenance

Test Case

Overview

test_case crate provides procedural macro attribute that generates parametrized test instances.

Getting Started

Crate has to be added as a dependency to Cargo.toml:

[dev-dependencies]
test-case = "*"

and imported to the scope of a block where it's being called (since attribute name collides with rust's built-in custom_test_frameworks) via:

use test_case::test_case;

Example usage:

#[cfg(test)]
mod tests {
    use test_case::test_case;

    #[test_case(-2, -4 ; "when both operands are negative")]
    #[test_case(2,  4  ; "when both operands are positive")]
    #[test_case(4,  2  ; "when operands are swapped")]
    fn multiplication_tests(x: i8, y: i8) {
        let actual = (x * y).abs();

        assert_eq!(8, actual)
    }
}

Output from cargo test for this example:

$ cargo test

running 4 tests
test tests::multiplication_tests::when_both_operands_are_negative ... ok
test tests::multiplication_tests::when_both_operands_are_positive ... ok
test tests::multiplication_tests::when_operands_are_swapped ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Test Matrix

The #[test_matrix(...)] macro allows generating multiple test cases from the Cartesian product of one or more possible values for each test function argument. The number of arguments to the test_matrix macro must be the same as the number of arguments to the test function. Each macro argument can be:

1. A list in array (`[x, y, ...]`) or tuple (`(x, y, ...)`) syntax. The values can be any
   valid [expression](https://doc.rust-lang.org/reference/expressions.html).
2. A closed numeric range expression (e.g. `0..100` or `1..=99`), which will generate
   argument values for all integers in the range.
3. A single expression, which can be used to keep one argument constant while varying the
   other test function arguments using a list or range.

Example usage:

#[cfg(test)]
mod tests {
    use test_case::test_matrix;

    #[test_matrix(
        [-2, 2],
        [-4, 4]
    )]
    fn multiplication_tests(x: i8, y: i8) {
        let actual = (x * y).abs();

        assert_eq!(8, actual)
    }
}

MSRV Policy

Starting with version 3.0 and up test-case introduces policy of only supporting latest stable Rust. These changes may happen overnight, so if your stack is lagging behind current stable release, it may be best to consider locking test-case version with = in your Cargo.toml.

Documentation

Most up to date documentation is available in our wiki.

License

Licensed under of MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)

Contributing

Project roadmap is available at link. All contributions are welcome.

Recommended tools:

  • cargo readme - to regenerate README.md based on template and lib.rs comments
  • cargo insta - to review test snapshots
  • cargo edit - to add/remove dependencies
  • cargo fmt - to format code
  • cargo clippy - for all insights and tips
  • cargo fix - for fixing warnings

test-case's People

Contributors

alanbriolat avatar augustofkl avatar drwilco avatar dtolnay avatar dzejkop avatar evforde avatar frondeus avatar ijc avatar johnpeel avatar luke-biel avatar martinvonz avatar ollien avatar overhacked avatar renovate-bot avatar rpwachowski avatar rzumer avatar samueltardieu avatar strohel avatar synek317 avatar tamird avatar th0114nd avatar tomprince avatar tony84727 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

test-case's Issues

clippy warning: literal bool comparison

clippy complains if the return value is a boolean, since this results in a assert_eq!(expected_bool, actual_bool), which clippy was rather a assert!(actual_bool) or assert!(!actual_bool)

workaround is to add a #[allow(clippy::bool_assert_comparison)] to the test attributes. I think test-case should do that internally

Add support for just `panics`

#[test_case(1 => panics)]
fn foo(_x: u32) {
	panic!(":(");
}

right now, only

#[test_case(1 => panics "")]
fn foo(_x: u32) {
	panic!(":(");
}

is supported

Generate a test per file in a directory

Is it possible to generate a test-case per file in a directory? I'm working on an image generating program, and we have test images and it would be convenient to be able to say something like:

#[test_resources("images/*")]
fn image_test(path: &Path) {
  // image test code
}

This is possible with test-generator but it's less actively maintained.

Allow replacement of test-case assertion on per-fn scale

Currently test_case allows replacing it's built in assert via using and with keywords.

We'd like to be able to replace assertion on per-function and global levels. This refers to requirements specified within #31.

Initial work specifies

#[test_case(input2)]
#[test_case_use(assert_nan)]
fn tested(v: X) -> f64 {
	...
}

as possible solution for per-function replacements.

Roadmap: 2.0.0 and beyond

This is a tracking issue for test_case roadmap beyond 1.x.

Features are waiting to be implemented or are being currently implemented. 2.0.0 is coming with major codebase rewrite, thus it will be released as rc first to test stability and portability from 1.x versions.

# 2.0.0

  • removal of inconclusive within description (currently deprecated)
  • removal of hamcrest_assertions in favor of custom solution #75
  • #[test_case(args => with |x: T| assert!(...))] for inline custom assertions #31
  • #[test_case(args => using $path)] for custom assertion reuse #31
  • leave tested fn item in place where it's written #77
  • default #[test_case] with no return type matching support Result<(), E> where E: Error similar to #[test] macro #50

beyond

  • custom assertions can be applied to multiple test_cases via #[test_case_handler($path)] attribute #83
  • override test case method names #[test_case(args as function_name)], #[test_case(args => result as function_name)] and #[test_case(args => result ; "description" as function_name)] #72
  • #95
  • generate test case per file in directory #78

research needed

  • with |val: T| -> bool {body} is allowed without assertion within body
  • test case name is accessible from within test_case block #37
  • PoC how test_case could look after custom_test_frameworks are stabilized: https://github.com/luke-biel/interrogate

Make inconclusive a keyword

/u/somebodddy noticed that inconclusive could be keyword, not a part of string.

Advantages:

  • With keyword approach typo could emit a compilation error, while typo in string creates just test case with no #[ignore] attribute.
  • It gives us extendable syntax, for example today its inconclusive, tommorow could be should_panic.

Disadvantages:

  • Breaking change for already written tests - tests with "inconclusive" string will loose theirs #[ignore] attribute (which may cause failed tests or false positives or even non deterministic CI behavior)
  • Keyword approach makes "inconclusive" in fixed position, while in string you can put in in the middle of sentence.

Feel free to comment :)

Syn 1.0.58 breaks test-case

dtolnay/syn@957840e

pinning to 1.0.57 should fix, or updating the import from export to __private.

Example error in github actions:

error[E0432]: unresolved import `syn::export`
 --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/test-case-1.0.0/src/test_case.rs:3:10
  |
3 | use syn::export::TokenStream2;
  |          ^^^^^^ could not find `export` in `syn`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
error: could not compile `test-case`

I tried raising on syn but it looks like @dtolnay has made raising issues there restricted to contributors. This feels like it shouldn't have been a patch release?

Access test case name from test case body

Example code

    #[test_case(vec![(1,1),(2,2),(3,3)].iter().collect(), vec![(1,1),(2,2),(3,3)].iter().collect() ; "test")]
    fn get_changes_values<
        'a,
        'b,
        K: Eq + Ord + Clone + fmt::Debug,
        V: Eq + fmt::Debug,
    >(
        old: &'b BTreeMap<&'a K, V>,
        new: &'b BTreeMap<&'a K, V>,
    ) {
        let changes = super::get_changes_values(old, new);
        // test_case => "test"
        assert_debug_snapshot(test_case, changes);
    }

Parse '-' in numeric values used in test case

For:

#[test_case(15, 15)]
#[test_case(-15, 15)]

fn test_abs(value: f64, expected: f64) {
   assert_eq!(value.abs(), expected)
}

We get error:

error[E0428]: the name _15_15 is defined multiple times
--> src/test.rs
|
| #[test_case(15, 15)]
| ^^^^^^^^^^^^^^^^^^^^
| |
| previous definition of the value _15_15 here
| _15_15 redefined here
|
= note: _15_15 must be defined only once in the value namespace of this mod

strict float comparisons

how should one handle float comparisons with this library?

eg.

#[test_case(0 => 0.0)]
fn get_floatvalue: u128) -> f64 {
    ...
}

clippy output-

strict comparison of `f32` or `f64`
`#[warn(clippy::float_cmp)]` implied by `#[warn(clippy::pedantic)]`
`f32::EPSILON` and `f64::EPSILON` are available for the `error_margin`
for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#float_cmp

Usage with #[tokio::test]

Currently it doesn't seem to work:

    #[tokio::test]
    async fn test_case(name: &str) {
        
    }

Output:

error: the test function cannot accept arguments
   --> iml-services/iml-devices/src/db.rs:578:24
    |
578 |     async fn test_case(name: &str) {
    |                        ^^^^^^^^^^

Is there a way to provide a test_case::test_case_tokio macro to support async tests?

Declare test cases once, use them multiple times

Is there a way to declare test cases only once and use them multiple times across multiple functions in multiple files? I read the wiki but I didn't see anything like that. If there isn't, that might be a feature worth adding.

Investigate lifetime requirement on inline test_case

I'm using test_case in my prod code (outside of #[cfg(test)])

#[test_case(br#""abc""#, Ok(b"", TokenUnicodeString("abc".to_string())))]
fn parse_token_unicode_string<'a>(input: &'a [u8]) -> Result<(&'a [u8], TokenUnicodeString), Error> {

returns

error[E0261]: use of undeclared lifetime name `'a`
  --> crates/kami-schema/src/unicode_string.rs:11:64
   |
11 | fn parse_token_unicode_string<'a>(input: &'a [u8]) -> Result<(&'a [u8], TokenUnicodeString), Error> {
   |                                                                ^^ undeclared lifetime

and for

#[test_case(br#""abc""#, Ok(b"", TokenUnicodeString("abc".to_string())))]
fn parse_token_unicode_string(input: &[u8]) -> Result<(&[u8], TokenUnicodeString), Error> {

it prints out

error[E0106]: missing lifetime specifier
  --> crates/kami-schema/src/unicode_string.rs:11:56
   |
11 | fn parse_token_unicode_string(input: &[u8]) -> Result<(&[u8], TokenUnicodeString), Error> {
   |                                                        ^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but there is no value for it to be borrowed from
help: consider using the `'static` lifetime
   |
11 | fn parse_token_unicode_string(input: &[u8]) -> Result<(&'static [u8], TokenUnicodeString), Error> {
   |                                                        ~~~~~~~~

Without test_case() attribute it compiles. I'd expect this to work after changes made in 2.0.

`is` and `it` modifiers have to be first class citizens

Existence of hamcrest_assertions is usefull, but requires addition of an extra crate. It would be beneficial if user could write test_case(args => is ...) without extra work.

For first release, for all assertions provided by hamcrest2 lib we should either cover it with our own syntax or mention replacement/why it's not implemented.

Later we'll be able to expand functionality of is|it on our own.

General matchers

  • eq - already covered via arg => value is equal_to X
  • not eq
  • (not) lt is less_than X
  • (not) leq is less_or_equal_than X
  • (not) gt is greater_than X
  • (not) geq is greater_or_equal_than X
  • (not) type_of - do we need that?
  • #80

Numerical matchers

  • (not) close_to is almost_equal_to X

Filesystem matchers

  • (not) path_exists is existing_path
  • (not) file_exists is file
  • (not) dir_exists is dir

Option and Result

  • has - matches Ok(5) can be used
  • negated has - matches Ok(v) if v != 5
  • (not) ok - matches Ok(_) & matches Err(_)
  • (not) err - same
  • (not) some - same
  • (not) none - same

Collection matchers

  • (not) contains it contains X
  • (not) contains_exactly - matches vec![1, 2, 3] & matches [1, 2, 3]
  • (not) contains_in_order it contains_in_order [X, Y, Z]
  • #81
  • #82

Compound matchers

  • (not) all
  • (not) any

Misc

  • is(bool) - supported via arg => value
  • anything - useless in test_case context

Add `is (not) len ...`

As an user of test case I want to be able to test for lenght of actual value within complex assertions, eg.:

#[test_case(b"1" => is len 1)]
fn test_this(s: &[u8]) -> &[u8] {
	s
}

This feature can use simple #expr.len() and assume user uses type that supplies len method

Testing for NaN

Thanks for the awesome crate!

I was wondering if there was any way to go about testing NaN value equality? I've tried a few ideas to no success (unless I went about them wrong), such as using the hamcrest2 support.

If this isn't possible at the moment then I already have a workaround using regular tests, but maybe it could be supported somehow in the future.

Enable patterns in return syntax

Pattern matching assertions should definitely be a thing.

Can it do things like #[test_case(Some(3) => Some(_))] ?

Comment made by /u/gilescope

`matches` `if` doesn't work in tandem

    #[test_case(SimpleEnum::Var1 => matches Ok(e) if e == SimpleEnum::Var1)]
    #[test_case(SimpleEnum::Var2 => matches Err(e) if e == "var2")]
    fn extended_pattern_matching_result(e: SimpleEnum) -> Result<SimpleEnum, &'static str> {
        if e == SimpleEnum::Var1 {
            Ok(e)
        } else {
            Err("var2")
        }
    }

outputs

   Compiling basic v0.1.0 (/home/luke-biel/Workspace/frondeus/test-case/acceptance_tests/basic)
error: unexpected token
   --> src/lib.rs:153:51
    |
153 |     #[test_case(SimpleEnum::Var1 => matches Ok(e) if e == SimpleEnum::Var1)]
    |                                                   ^^

error: could not compile `basic` due to previous error

I would expect this to compile as per our wiki.

Mixing test_case with cfg?

Hi,

I am in a situation where my test case only must be compiled if some rust feature is enabled:

#[cfg(feature = "feature_type_a")]
#[test_case(TypeA)]
#[test_case(TypeB)]
fn do_stuff() {
...
}

The problem with the above code is that the cfg is set for the entire function (all cases). If feature_type_a is not enabled, the function do_stuff does not exist, but I want that it exists for TypeB. Is there any way to pass a cfg or a feature into a test_case? Something like: #[test_case(features = "feature_type_a", TypeA)]

Thank you so much!

Result::Err is not interpreted as a failure

In Rust 2018 test can return a Result type, with Result::Err being interpreted as a failure. However with parameterised tests returned errors are treated as passing. While this can be fixed by specifying an expected value, it is unintuitive.

Example:

fn should_equal_4(num: i32) -> Result<(), ()> {
    if num == 4 {
        Ok(())
    } else {
        Err(())
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use test_case::test_case;

    #[test_case(4; "Passing")]
    #[test_case(5; "Failing")]
    fn with_params(num: i32) -> Result<(), ()> {
        should_equal_4(num)?;
        Ok(())
    }

    #[test]
    fn is_4() -> Result<(), ()> {
        should_equal_4(4)?;
        Ok(())
    }

    #[test]
    fn is_5() -> Result<(), ()> {
        should_equal_4(5)?;
        Ok(())
    }
}

Expected: Both tests::with_params::failing() and is_5() fail.

Actual: Only is_5() fails.

Review possibility to add `test_case` attribute to items within `impl` blocks

As in title. Since we support using test case on free floating fn items, it would be useful to review if we can enable this functionality on impl blocks, eg.:

impl StructName {
	#[test_case("abcd" => Ok(("ab", "cd")))]
	fn parse(input: &[u8]) -> IResult<&[u8], &[u8])> {
		todo!()
	}
}

this may require extra attribute though.

`test-env-log` used with `test-case` creates test duplicates

When I use the latest test-case and test-log (new name of test-env-log) together, I get duplicate copies of each test generated.
I also tried it with what I think were the versions from when this issue was created and have the same problem.

Full example in this gist

#[cfg(test)]
mod tests {
    use test_case::test_case;

    #[test_case(0, 0 ; "the same")]
    #[test_case(0, 1 ; "different")]
    #[test_log::test]
    // #[test_env_log::test]
    fn equality(a: usize, b: usize) {
        log::info!("a: {}", a);
        log::info!("b: {}", b);
        assert_eq!(a, b)
    }
}
Output
running 4 tests
test tests::equality::the_same ... ok
test tests::equality::the_same ... ok
test tests::equality::different ... FAILED
test tests::equality::different ... FAILED

failures:

---- tests::equality::different stdout ----
thread 'tests::equality::different' panicked at 'assertion failed: `(left == right)`
  left: `0`,
 right: `1`', main.rs:18:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- tests::equality::different stdout ----
thread 'tests::equality::different' panicked at 'assertion failed: `(left == right)`
  left: `0`,
 right: `1`', main.rs:18:9


failures:
    tests::equality::different
    tests::equality::different

test result: FAILED. 2 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: test failed, to rerun pass '--bin main'

Originally posted by @newsch in #69 (comment)

Is it possible to use with pretty_assertions?

I have been playing with pretty_assertions and really like it. But when trying to use it with test-case I seem to get errors of ambiguity over assert_eq. Is it possible to use these two together?

error[E0659]: `assert_eq` is ambiguous (glob import vs any other name from outer scope during import/macro resolution)
 --> src\squared\tests.rs:7:1
  |
7 | #[test_case(5 => 25 ; "should be 25")]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ambiguous name
  |
  = note: `assert_eq` could refer to a macro from prelude
note: `assert_eq` could also refer to the macro imported here
 --> src\squared\tests.rs:7:1
  |
7 | #[test_case(5 => 25 ; "should be 25")]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  = help: consider adding an explicit import of `assert_eq` to disambiguate
  = help: or use `self::assert_eq` to refer to this macro unambiguously

warning: unused import: `test_case::test_case`
 --> src\guessing_game\tests.rs:3:5
  |
3 | use test_case::test_case;
  |     ^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

Here is the code. Uncommenting the pretty_assertions line gives the error above. Without it, it runs normally.

#[allow(unused_imports)]
use crate::squared::squared;
//use pretty_assertions::assert_eq;
#[cfg(test)]
use test_case::test_case;

#[test_case(5 => 25 ; "should be 25")]
#[test_case(2 => 4 ; "should be 4")]
fn squared_test(num: i32) -> i32 {
    squared(num)

Generate test case for each enum variant

JUnit has a really nice feature where, given an enum, you can make a test that will generate an instance of the test for each member of the enum. For instance, you could have something like this

public enum Player {
	PLAYER1, PLAYER2;
}

@ParameterizedTest
@EnumSource(Player.class)
void initializingTheGameStartsWithTheGivenPlayer(Player initialPlayer) {
	// ...
}

It would be cool if you could do something like this with test_case

enum Player {
	Player1,
	Player2,
}

#[test_case(Player)]
fn test_for_each_player(player: Player) {
	// ...
}

Now, obviously this wouldn't work if the enum had values attached to any of the variants, but at least for this specific case, it prevents you from forgetting to add a test case when you add an enum variant (since it would be auto-generated)

Bad custom test names should not fail silently

I will be borrowing the example from #19, since it has some relation to how I found this error on my own code.

So, we know from that issue that the minus will be abstracted from the automatic name creation process, BUT, we have a mechanism for custom names, and that could help the developer to understand that the problem with his code is that he is lacking a valid test name:

#[test_case(15, 15; "Abs_15_15")]
#[test_case(-15, 15; "Abs_-15_-15")]
fn test_crazy_stuff(value: i32, expected: i32) {
    assert_eq!(value.abs(), expected)
}

And this will raise a familiar error, but the interesting part is that it tried to use my custom name, since it begins with abs, and not test_crazy_stuff:

error[E0428]: the name `abs_15_15` is defined multiple times
   --> src/lib.rs:162:5
    |
162 |     #[test_case(15, 15; "Abs_15_15")]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |     |
    |     `abs_15_15` redefined here
    |     previous definition of the value `abs_15_15` here

This leads me to believe that we have a process that tries to make the name valid, since it erased the hyphen from my custom test, applied, and then we had a name collision.

I haven't tried to analyze the code to understand how the hyphen is erased in both cases, but, i believe that if a developer instantiates a custom name, it shouldn't be safeguarded, if the name is invalid it should break catastrophically warning the developer of what the problem is.

Using `impl Trait` in test_case arguments

Right now test-case do not support impl Trait in function arguments:

#[test_case("foo")]
fn test_foo(foo: impl Into<String>) { }

It returns error:

error[E0562]: impl Trait not allowed outside of function and inherent method return types
|
301 | fn test_foo(foo: impl Into) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: add #![feature(impl_trait_in_bindings)] to the rate attributes to enable

Support a string argument to "ignore" describing the reason

Since Rust 1.61.0 the argument to an ignore attribute has been printed as part of the test output. So:

#[test]
#[ignore = "Some reason"]
fn something() {...}

Will result in:

$ cargo test 
...
test something ... ignored, Some reason

I believe the syntax has been valid for some releases prior to 1.61.0 (I'm afraid I don't know since when) but was ignored.

It would be great if test_case's ignore modifier (and inconclusive I suppose) could similarly take an option reason and propagate it so that cargo test will show it.

I'm not sure that #[test_case(INPUT => ignore = "Some reason" OUTPUT)] will fit the current syntax but perhaps #[test_case(INPUT => ignore("Some reason") OUTPUT)] could work?

Regression from 1.2.1 -> 1.2.2: "return-type but no exected clause" failing to compile

Tests that were valid in 1.2.1 fail to compile with 1.2.2

an example can be seen here:

#[test_case("dtn:none" => "dtn:none" ; "when using none endpoint")]
#[test_case("dtn:n1/" => panics "" ; "when using dtn endpoint without double slash")]
#[test_case("dtn:none" => EndpointID::none().to_string() ; "when providing node eid and constructed none")]
fn from_str_tests(input_str: &str) -> String {
  EndpointID::try_from(input_str).unwrap().to_string()
}

The compiler gives the following error:

error: Test function from_str_tests has a return-type but no exected clause in the test-case. This is currently unsupported. See test-case documentation for more details.
   --> src/eid.rs:507:43
    |
507 |     fn from_str_tests(input_str: &str) -> String {
    |                                           ^^^^^^

Also, there seems to be a typo in the error message as exected should probably be expected.

Did the syntax change?
The changelog does not document any breaking changes regarding this.

Accept external test suite files to generate test cases

I don't know if this matches the plan for the crate but I though it is suitable.

The QA in the project I work on have a csv file with test cases input/output:

scenario_name,assert_http_code,assert_error_message,name
May not have linebreaks,400,error message,\n
...

I wanted to use it to generate test cases in rust, I was thinking about something like:

#[test_input] // used to generate test names
struct TestInput {
  scenario_name: String,
  assert_http_code: u16,
  assert_error_message: String,
  name: String,
}

#[generate_test_suite_from_file("path_to_file.csv")] // parsed by https://crates.io/crates/csv
fn test_suite_name(input: TestInput) {
// ... test code
}

This would generate:

mod test_suite_name {
  #[test]
  fn may_not_have_linebreaks(input: TestInput) {
    // ... test code
  }
}

Leave annotated item outside test module

I'm not sure if I'm the only one asking for this, but this feels like a no brainer, so it's kind of curious...

I want to use #[test_case] annotations on functions that are not test functions but the actual functions that implement things. They might be a part of the public API, or just used by many other functions in the same file. This would be similar to writing a "doc test", but a doc test is necessarily 3 lines and quite verbose compared to just a test_case annotation. Additionally, doc tests are not part of the normal unit test frameworks, but instead handled with a special binary, which causes all sorts of annoyance.

Currently, just adding the annotation causes there to be a module added with the function name, and the function to be moved inside that module, and then followed by all the generated test case functions. Instead, I would want the module to be named test_<ident>, the generated test cases placed inside that and the function to be left exactly as is. To be honest, that's how I assumed this to work.

This would probably need to be a flag, or a separate macro, since I guess all the current users assume that the test case function is purely for testing.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Add `is (not) empty`

As an user of test case I want to be able to test whether actual value is empty within complex assertions, eg.:

#[test_case("1" => is empty)] // fails
#[test_case("" => is empty)] // succeeds
fn test_this(s: &str) -> &str {
	s
}

We can rely on #expr.is_empty() and that user supplies a type that has such function.

Add support for empty test case

#[test_case()]
fn should_work() {}
#[test_case( ; "name")]
fn should_definitely_work() {}

Questions:

How to generate test case name if there are no arguments.

Combination with other attributes than test_case

Are there any way to combine test_case with other attributes which can be applied to test functions e.g. serial_test?

I cannot compile the following example at v1.0.0 (and serial_test = "0.5.1"),

    #[test_case(1)]
    #[test_case(2)]
    #[serial_test::serial]
    fn need_to_be_serial(x: i32) {
        assert!(x > 0);
    }

Add `it (not) matches_regex ...`

As an user of test case I want to be able to test whether actual value matches giver regular expression within complex assertions, eg.:

#[test_case("1" => it matches_regex "[0-9]")]
fn test_this(s: &str) -> &str {
	s
}

This feature has to be hidden behind feature flag (regex) and require user to manually import https://docs.rs/regex/latest/regex/ crate

Nightly clippy raises warning "unneeded unit expression"

  • test-case version: 1.1.0
  • rustc & clippy version: 1.54.0-nightly (c1e8f3a58 2021-05-30)

Clippy raises a warning to the following code.

#[cfg(test)]
mod tests {
    use test_case::test_case;

    #[test_case(1, 1)]
    fn square(x: i32, y: i32) {
        assert_eq!(x*x, y);
    }
}

Messages from clippy,

warning: unneeded unit expression
 --> src/main.rs:9:5
  |
9 |     #[test_case(1, 1)]
  |     ^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(clippy::unused_unit)]` on by default
  = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unused_unit
  = note: this warning originates in the attribute macro `test_case` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: 1 warning emitted

I think this warning should be suppressed in macro, or we should modify the macro not to use unit expressions, if possible.

Is the README correct?

The README has the following example for the base usage:

#![cfg(test)]
extern crate test_case;

use test_case::test_case;

#[test_case( 2,  4 ; "when both operands are possitive")]
#[test_case( 4,  2 ; "when operands are swapped")]
#[test_case(-2, -4 ; "when both operands are negative")]
fn multiplication_tests(x: i8, y: i8) {
    let actual = (x * y).abs();

    assert_eq!(8, actual)
}

However, I cannot get this to work. I get the following error:

error: an inner attribute is not permitted in this context
   --> src/bin/second.rs:246:3
    |
246 | #![cfg(test)]
    |   ^
    |
    = note: inner attributes, like `#![no_std]`, annotate the item enclosing them, and are usually found at the beginning of source files. Outer attributes, like `#[test]`, annotate the item following them.

When I take out the !, things work better, but not quite yet. When building regular (not test) the extern and use lines trigger errors. These go away when I do the following:

#[cfg(test)]
mod tests {
  extern crate test_case;

  use test_case::test_case;

  #[test_case( 2,  4 ; "when both operands are possitive")]
  #[test_case( 4,  2 ; "when operands are swapped")]
  #[test_case(-2, -4 ; "when both operands are negative")]
  fn multiplication_tests(x: i8, y: i8) {
    let actual = (x * y).abs();

    assert_eq!(8, actual)
  }
}

Am I doing things wrong?

Ignore/inconclusive doesn't work for void fn

E.g. this works

#[test_case(0 => ignore 0)]
fn foo(x: u8) -> u8 {
  42
}

This doesn't

#[test_case(0 => ignore)]
fn bar(x: u8) {

}

Example output:

running 2 tests
test tests::foo::_0_expects_inconclusive0 ... ignored
test tests::bar::_0_expects ... ok

test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s

Version: 2.0.2

How to use with test-env-log ?

Hello,

i tried to use test-case with test-env-log but i got the error

error[E0659]: test is ambiguous (glob import vs any other name from outer scope during import/macro resolution)

use test_case::test_case;
use test_env_log::test;

#[test_case(4,  2  ; "when operands are swapped")]
#[test_case(-2, -4 ; "when both operands are negative")]
#[test_case(2,  4  ; "when both operands are positive")]
fn multiplication_tests(x: i8, y: i8) {
	let actual = (x * y).abs();

	assert_eq!(8, actual)
}
cargo test
   Compiling git-gamble v2.1.1-alpha.0 (/home/pinage404/Project/git-gamble)
error[E0659]: `test` is ambiguous (glob import vs any other name from outer scope during import/macro resolution)
   --> tests/test_env_log_with_test_case.rs:4:1
    |
4   | #[test_case(4,  2  ; "when operands are swapped")]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ambiguous name
    |
note: `test` could refer to the attribute macro imported here
   --> tests/test_env_log_with_test_case.rs:4:1
    |
4   | #[test_case(4,  2  ; "when operands are swapped")]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = help: consider adding an explicit import of `test` to disambiguate
    = help: or use `self::test` to refer to this attribute macro unambiguously
note: `test` could also refer to the attribute macro defined here
   --> /media/exec_downloaded/rust/rustup/toolchains/1.55.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/prelude/mod.rs:119:13
    |
119 |     pub use super::v1::*;
    |             ^^^^^^^^^^^^
    = note: this error originates in the attribute macro `test_case` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0659`.
error: could not compile `git-gamble` due to previous error

How to use with test-env-log ?

related d-e-s-o/test-log#18

maybe related #38

Specify type parameters

It seems that test-case able to handle generic parameters, i.e.:

#[test_case(1i8; "xyz")]
fn my_test<T>(x: T)
where T: Debug
{
  println!("{:?}", x); 
}

but is it possible to specify a type parameter explicitly? That's in particular useful for cases, where you need to check conversion for specific trait, e.g.:

#[test_case(MyType { .. }; "xyz")]
fn my_test<T>(x: MyType)
where T: From<MyType>
{
  let _: T = x.into();
}

Add possibility to replace `assert_eq!` with custom assertion

When using => expected syntax it could be nice to set up what assertion shall we use.
It might be extra attribute:

#[test_case("foo" => "bar")]
#[test_case_assert_macro(assert_diff)]
fn test(input: &str) -> String { input.to_string() }

this could be expanded into:

mod test {
    use super::*;
    fn test(input: &str) -> String { input.to_string() }
    
    #[test]
    fn foo_expects_bar() {
       let actual = test("foo");
       assert_diff!("bar", actual);
    }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.