Git Product home page Git Product logo

ntest's Introduction

NTest

docs crates downloads build status license

Testing framework for rust which enhances the built-in library with some useful features. Inspired by the .Net unit-testing framework NUnit.

Getting Started

Some functions of NTest use procedural macros which are stable for rust edition 2018. If you use the library make sure that you are using the 2018 version of rust. Update the Cargo.toml file:

[package]
edition = "2018"
# ..

Add the NTest library to your developer dependencies in the Cargo.toml file:

[dev-dependencies]
ntest = "*"

Content

  • #[timeout()] Attribute used for timeouts in tests.
  • #[test_case()] Attribute used to define multiple test cases for a test function.
  • assert_about_equal!() Compare two floating point values or vectors for equality.
  • assert_false!() Expects false argument for test case.
  • assert_true!() Expects true argument for test case.
  • assert_panics!() Expects block to panic. Otherwise the test fails.

For more information read the documentation.

Examples

Create test cases

use ntest::test_case;

#[test_case("https://doc.rust-lang.org.html")]
#[test_case("http://www.website.php", name="important_test")]
fn test_http_link_types(link: &str) {
    test_link(link, &LinkType::HTTP);
}

Timeout for long running functions

use ntest::timeout;

#[test]
#[timeout(10)]
#[should_panic]
fn timeout() {
    loop {};
}

Combine attributes

use std::{thread, time};
use ntest::timeout;
use ntest::test_case;

#[test_case(200)]
#[timeout(100)]
#[should_panic]
#[test_case(10)]
#[timeout(100)]
fn test_function(i : u32) {
    let sleep_time = time::Duration::from_millis(i);
    thread::sleep(sleep_time);
}

ntest's People

Contributors

becheran avatar joshorndorff avatar lpenz avatar mati865 avatar poliorcetics avatar roblabla avatar serprex avatar slava-ini avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

ntest's Issues

New release

Hi, are there any plans to release new version of ntest?
Last release was in 2022, since then there was a PR that afaik prevented rust-analyzer not working with ntest + tokio::test: #23
It was merged in July but is still not part of any release :(

Rust Analyzer gives a warning (incorrect_ident_case) if `test_case` argument is an enum or struct

Description

When using struct/enum or any other pascal case type as an argument for test_case macro rust-analyzer gives a warning

Function function_Name should have snake_case name, e.g. function_name (rust-analyzer incorrect-ident-case)

ntest version: 0.9.0
Rust Analyzer: v0.3.1481
rustc: 1.67.1

To Reproduce

 #[test_case(Environment::Production)]
 fn check_env(env: Environment) {
        ...
  }

Expected behavior

Macro could compile function name into lower case variant

Screenshots

example

Desktop:

OS: Ubuntu 22.04.1 LTS

Since removing timebomb stuck tests do not timeout and fail

Describe the bug
When I originally started using ntest::timeout it was because I had tests that would infinitely loop and never complete. Back on version 0.3.4 this worked well, and I happened to never have updated in that project. When trying to use it again in a new project when diagnosing a stuck test I noticed newer versions would no longer deal with this.

To Reproduce
Create a crate containing

#[test]
#[ntest::timeout(1000)]
fn foo() {
    loop {}
}

run cargo test

Expected behavior
The test to fail after 1 second

Expected identifier error

When using two attributes such as test case and timeout they will check for the next attribute to be valid.
#[test_case(200)]
#[ntest::timeout(100)]
Will panic:
error: custom attribute panicked
--> ntest/tests/integration.rs:5:1
|
5 | #[test_case(200)]
| ^^^^^^^^^^^^^^^^^
|
= help: message: Expected identifier!

timeout macro for tests which return Result

Currently the timeout macro does not work when you add the timeout attribute to a unit test which returns a result.

e.g.

#[test]
#[timeout(100)]
fn no_timeout() -> Result<(), String> {
    let fifty_millis = time::Duration::from_millis(50);
    thread::sleep(fifty_millis);
    Ok(())
}

Would it be possible for you to enhance the macro so that it can wrap such a function and return the correct type

Run & Debug links gone in VS Code 1.74.1 with Rust Analyzer v0.3.1325

Description
VS Code has a pair of handy links for running or debugging individual tests. But when #[ntest::timeout(1000)] is placed, they are removed.

  • VS Code version: 1.74.1.
  • ntest version: 0.9.0
  • Rust Analyzer: v0.3.1325
  • rustc: 1.66.0 (69f9c33d7 2022-12-12)

To Reproduce
Put #[ntest::timeout(1000)] over the testing function.

Expected behavior
"Run" and "Debug" links should remain.

Screenshots
example

Desktop (please complete the following information):

  • OS: Ubuntu 22.04.1 LTS

Running release tests causes SIGILL

Describe the bug
I'm building ntest 0.7.3 crate for Guix System. The crate itself builds fine. However, when I run the cargo test release the process exits with the SIGILL: illegal instruction.

$ cargo test --release
       Fresh unicode-xid v0.2.1
       Fresh ntest_proc_macro_helper v0.7.3
       Fresh proc-macro2 v1.0.27
       Fresh serde v1.0.123
       Fresh quote v1.0.9
       Fresh toml v0.5.8
       Fresh syn v1.0.72
       Fresh proc-macro-crate v0.1.5
       Fresh ntest_test_cases v0.7.3
       Fresh ntest_timeout v0.7.3
       Fresh ntest v0.7.3 (/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3)
    Finished release [optimized] target(s) in 0.04s
     Running `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/ntest-2de355d2c3fb1823`

running 18 tests
test tests::about_equal_f32 ... ok
test tests::assert_false ... ok
test tests::about_equal_f64 ... ok
test tests::assert_false_fails ... ok
test tests::assert_false_trailing_comma ... ok
test tests::assert_panics_fails ... ok
test tests::assert_true ... ok
test tests::assert_panics ... ok
test tests::it_should_not_panic_if_values_are_approx_equal ... ok
test tests::assert_true_fails ... ok
test tests::compare_with_epsilon ... ok
test tests::assert_true_trailing_comma ... ok
test tests::fail_with_epsilon ... ok
test tests::assert_panics_trailing_comma ... ok
test tests::vector_trailing_comma ... ok
test tests::vector_fails ... ok
test tests::vector ... ok
test tests::vector_trailing_comma_with_epsilon ... ok

test result: ok. 18 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

     Running `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/integration-364ecb9c23d58916`

running 6 tests
error: test failed, to rerun pass '--test integration'

Caused by:
  process didn't exit successfully: `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/integration-364ecb9c23d58916` (signal: 4, SIGILL: illegal instruction)

To Reproduce
Steps to reproduce the behavior:

  1. Run guix import crate -r ntest
  2. Store definitions in <guix_repo>/guix/packages/crates-io.scm
  3. Build the package guix build -L <guix_repo> rust-ntest
  4. See error

Expected behavior
The test would pass.

Screenshots
Here is a backtrace of running in gdb

(gdb) backtrace
#0  0x00005555555703d0 in core::ops::function::Fn::call ()
#1  0x0000555555569442 in std::sys_common::backtrace::__rust_begin_short_backtrace::h014a162e230a3962 ()
#2  0x00005555555712e8 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#3  0x00005555555c2a3a in std::sys::unix::thread::Thread::new::thread_start ()
#4  0x00007ffff7facf64 in start_thread ()
   from /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libpthread.so.0
#5  0x00007ffff7ec59af in clone ()
   from /gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libc.so.6

Also as the tests run in parallel I tried to narrow down the issue with running them sequentially:

RUST_TEST_THREADS=1 cargo test --release
running 6 tests
test no_timeout ... ok
test test_function_10 ... ok
test test_function_200 ... ok
test timeout ... ok
test timeout_inf_loop ... error: test failed, to rerun pass '--test integration'

Caused by:
  process didn't exit successfully: `/tmp/guix-build-rust-ntest-0.7.3.drv-0/ntest-0.7.3/target/release/deps/integration-364ecb9c23d58916` (signal: 4, SIGILL: illegal instruction)
``

**Desktop (please complete the following information):**
 - Guix System

`timeout` attribute reports timeout even when test panics

Describe the bug
When the timeout attribute is put on a test and that test panics, the test reports that the test timed out, even though it didn't.

To Reproduce
Run this code:

#[cfg(test)]
mod tests {
    use ntest::timeout;

    #[test]
    #[timeout(20000)]
    fn panic() {
        assert!(false);
    }
}

Results in this output:

thread '<unnamed>' panicked at 'assertion failed: false', src/lib.rs:8:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tests::panic' panicked at 'timeout: the function call took 0 ms. Max time 20000 ms', src/lib.rs:6:5

The output should instead be this:

thread 'tests::panic' panicked at 'assertion failed: false', src/lib.rs:7:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Additional context
The fix is likely in ntest_proc_macro_helper, which needs to be changed to handle test panics.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.