guydunton / rust-catch Goto Github PK

View Code? Open in Web Editor NEW

3.0 2.0 0.0 35 KB

A Rust port of the Catch unit-testing framework

License: MIT License

Rust 100.00%

rust catch catch2 unit-testing unit-testing-framework

rust-catch's Introduction

Rust-Catch

✋ Disclaimer

This library is no longer being actively maintained. For reasons why read my article on the subject.

If the approach sounds interesting I recommend checking out Subcase which is also a spiritual successor to Catch2 but follows a different approach.

What is this library

This is a Rust unit-testing framework based on the excellent C++ library Catch/Catch2.

The goals of the Rust library are the same as the C++ one; write unit-test code using the patterns and idioms of the Rust language.

Example

tests! {
    test("descriptive test name") {
        assert_eq!(function_under_test(), result);
    }
}

Example with Sections

tests! {
    test("vec capacity change appropriately") {

        // Variable initialized for each section
        let mut vec = vec![1, 2, 3];

        section("push increases capacity") {
            vec.push(4);
            assert_eq!(vec.capacity(), 4);
        }

        section("pop leaves capacity same size") {
            vec.pop();
            assert_eq!(vec.capacity(), 3);
        }
    }
}

Motivation

In Rust, unit-tests are written as functions using the #[test] attribute. e.g.

#[test]
fn add_test() {
    assert_eq!(add(1, 2), 3);
}

There are some issues with this:

People write short names for functions, rather than good descriptions for the test.
Tests have to test multiple things at once or repeat setup code. There is no default support for text fixtures.

Solution

Rust-Catch aims to take the best things from default unit-testing in Rust and add facilities that make it easier to write maintainable tests.

The above test could be written using Rust-Catch as follows:

tests! {
    test("add works with positive numbers") {
        assert_eq!(add(1, 2), 3);
    }

    test("add works with negative numbers") {
        assert_eq!(add(5, -1), 4);
    }
}

// This is equivalent to the following:

#[test]
fn add_works_with_positive_numbers() {
    assert_eq!(add(1, 2), 3);
}

#[test]
fn add_works_with_negative_numbers() {
    assert_eq!(add(5, -1), 4);
}

Features

Sections

Sections are an answer to text fixtures from other testing frameworks. The setup code or teardown code in the test case becomes part of each section. This reducing the amount of repeated code in tests and reduces the amount of specialist code required for unit-tests.

tests! {
    test("Vec can be expanded and shrunk") {

        // Setup
        let mut vec = vec![1, 2, 3];

        section("Expanding a vec increases capacity and length") {
            vec.push(4);

            assert_eq!(vec.len(), 4);
            assert_eq!(vec.capacity(), 4);
        }

        section("reducing the vec decreases length but not capacity") {
            vec.pop();

            assert_eq!(vec.len(), 2);
            assert_eq!(vec.capacity(), 3);
        }
    }
}

// Is converted to:

mod Vec_can_be_expanded_and_shrunk {
    use super::*;

    #[test]
    fn Expanding_a_vec_increases_capacity_and_length() {

        // Setup
        let mut vec = vec![1, 2, 3];

        // Test code
        vec.push(4);

        assert_eq!(vec.len(), 4);
        assert_eq!(vec.capacity(), 4);
    }

    #[test]
    fn reducing_the_vec_decreases_length_but_not_capacity() {

        // Setup
        let mut vec = vec![1, 2, 3];

        // Test code
        vec.pop();

        assert_eq!(vec.len(), 2);
        assert_eq!(vec.capacity(), 3);
    }
}

Implementation Details

Rust is implemented using procedural macros.

Due to Rust parsing rules test's have to be surrounded by a tests.

tests is currently removed from the generated code but I'm not sure that's the best approach. An alternative is to wrap all test's with a mod e.g.

tests! {
    ...
}

// Becomes
#[cfg(test)] // Is this necessary?
mod test {
    ...
}

rust-catch's People

Contributors

Stargazers

Watchers

rust-catch's Issues

Change test_case to test

We just need a new version of test_case called test because it's less verbose.

Investigate fine grained warning suppression

In this issue I decided to suppress all warnings for unused variables and unused mutable.

I would like to have a more fine grained approach involving determining if a warning is valid e.g.

test_case!("test") {
    let mut a = 4;
    let name = "Michael";

    section("check a") {
        a += 1;
        assert_eq!(a, 5);
    }
    section("is a the same") {
        assert_eq!(a, 4);
    }
}

In the above example we would like to see the warning for the unused variable because that variable has not been used in either section.

A way of doing this would be to check the warnings in the code and if a warning was present in one section but not another in the same test case, we suppress the warning.

In order to do this it might help to have the proc_macro_diagnostics.

Cannot wrap sections

Currently rust_gb is struggling with it's tests because I cannot wrap braces around a section e.g.

for (opcode, register) in instructions {
    section("a logical test unit") {
    }
}

False positive of unused variables

See example below:

test_case!("test") {
    // Get a warning that the variable is unused or doesn't
    // need to be mutable.
    let mut var = 4; 

    section!("sec1") {
         // Use variable here
    }

    section!("sec2") {
        // Don't use the variable here
    }
}

This variable shouldn't be marked as unused/could be immutable.

Add CI support

Need some automated CI to ensure library maintains quality

Suppress warning for snake case test names

Currently the help for a blank name is: Hint: Try to use a descriptive name such as "Vec can be resized". But when you do this you will get a warning saying: function `Vec_can_be_resized` should have a snake case name.

This warning should be suppressed.

Change test_suite to tests

Less typing is always good

Emit warning for empty test suite

There isn’t any point having an empty test_suite so a warning should be provided explaining the syntax for a test case.

Fix mod snake case

When you have the code

test! {
    test("My Test") {
        section("my test name") {
             // Actual test
        }
    }
}

This code will be translated into:

mod My_Test {
    #[test]
    fn my_test_name() {
        // Actual test
    }
}

It appears mod names are still warning about not being snake case

test("My test") {
    section("test for one thing") {
         let result = myFunction();
    }
    section("test for a second thing") {
         let result = myFunction(withParam);
    }
    assert_eq!(result, passed);
}

Unit tests

The program needs tests.
Possible ways of doing this include hooking in something custom for panics

Cannot add #[should_panic] to test

I cannot add extra attributes to tests.

e.g. I should be able to write

#[should_panic]
test("this will panic") {
    panic!();
}

There is no documentation

There needs to be some documentation for the library to show how it should be used.

Currently when the docs are generated they only show the tests and test_suite macros.

This might also be a good opportunity to investigate rusts documentation tests as a replacement for a lot of the example code