mrzechonek / tut-framework Goto Github PK

View Code? Open in Web Editor NEW

50.0 7.0 41.0 1.32 MB

C++ Template Unit Test Framework

Home Page: http://mrzechonek.github.io/tut-framework

License: Other

C++ 95.32% Python 4.40% C 0.28%

unit-testing c-plus-plus

tut-framework's Introduction

tut-framework

TUT is a small and portable unit test framework for C++.

TUT is very portable, no matter what compiler or OS you use.
TUT consists of header files only. No libraries required, deployment has never been easier.
Custom reporter interface allows to integrate TUT with virtually any IDE or tool in the world.
Support for multi-process testing (testing deadlocks and timeouts is under way).
TUT is free and distributed under a BSD-like license.
Tests are organised into named test groups.
Regression (all tests in the application), one-group or one-test execution.
Pure C++, no macros!

TUT tests are easy to read and maintain. Here’s the simplest test file possible:

#include <tut/tut.hpp>

namespace tut
{
    struct basic{};
    typedef test_group<basic> factory;
    typedef factory::object object;
}

namespace
{
    tut::factory tf("basic test");
}

namespace tut
{
    template<>
    template<>
    void object::test<1>()
    {
        ensure_equals("2+2=?", 2+2, 4);
    }
}

tut-framework's People

Contributors

Stargazers

Watchers

Forkers

mdeilman hnkulkarni zhongxingzhi icestudent brehm orcy remileduc pgerell cjmayo lengfeld bwedding bennettgoble

tut-framework's Issues

Can you document what the two templates for test functions do?

Hello,

What do the two templates that are used with test functions do? I have not found any documentation or samples on their use. Can you please add documentation that explains what they are?

Ugly hack for visual studio output pane

Hello,

I made some ugly modifications to the tut_console_reporter.hpp file to enable sending output to Visual Studio output pane. Feel free to cleanup and incorporate into your main distribution.

There are more elegant solutions such as http://stackoverflow.com/questions/73286/capturing-cout-in-visual-studio-2005-output-window

Thanks
Rakesh

#ifndef TUT_CONSOLE_REPORTER
#define TUT_CONSOLE_REPORTER
#include <tut/tut.hpp>
#include <cassert>


#if defined( _WINDOWS )
#include <Windows.h>
#endif

/**
 * Template Unit Tests Framework for C++.
 * http://tut.dozen.ru
 *
 * @author Vladimir Dyuzhev, [email protected]
 */
namespace
{

std::ostream& operator<<(std::ostream& os, const tut::test_result& tr)
{
    switch(tr.result)
    {
        case tut::test_result::ok:
        {
            os << '.';
#if defined( _WINDOWS )
            ::OutputDebugStringA( "." );
#endif
            break;
        }
        case tut::test_result::fail:
        {
            os << '[' << tr.test << "=F]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=F]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::ex_ctor:
        {
            os << '[' << tr.test << "=C]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=C]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::ex:
        {
            os << '[' << tr.test << "=X]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=X]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::warn:
        {
            os << '[' << tr.test << "=W]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=W]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::term:
        {
            os << '[' << tr.test << "=T]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=T]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::rethrown:
        {
            os << '[' << tr.test << "=P]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=P]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::skipped:
        {
            os << '[' << tr.test << "=S]";
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << '[' << tr.test << "=S]";
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            break;
        }
        case tut::test_result::dummy:
            throw tut::tut_error("console reporter called for dummy test result");
    }

    return os;
}

} // end of namespace

namespace tut
{

/**
 * Default TUT callback handler.
 */
class console_reporter : public tut::callback
{
    std::string current_group;
    typedef std::vector<tut::test_result> not_passed_list;
    not_passed_list not_passed;
    std::ostream& os;

    console_reporter(const console_reporter &);
    console_reporter &operator=(const console_reporter &);
public:

    int ok_count;
    int exceptions_count;
    int failures_count;
    int terminations_count;
    int warnings_count;
    int skipped_count;

    console_reporter()
        : current_group(),
          not_passed(),
          os(std::cout),
          ok_count(0),
          exceptions_count(0),
          failures_count(0),
          terminations_count(0),
          warnings_count(0),
          skipped_count(0)
    {
        init();
    }

    console_reporter(std::ostream& out)
        : current_group(),
          not_passed(),
          os(out),
          ok_count(0),
          exceptions_count(0),
          failures_count(0),
          terminations_count(0),
          warnings_count(0),
          skipped_count(0)

    {
        init();
    }

    void run_started()
    {
        init();
    }

    void test_completed(const tut::test_result& tr)
    {
        if (tr.group != current_group)
        {
            os << std::endl << tr.group << ": " << std::flush;
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << std::endl << tr.group << ": " << std::flush;
            ::OutputDebugStringA( ss.str().c_str() );
#endif
            current_group = tr.group;
        }

        os << tr << std::flush;

        // update global statistics
        switch (tr.result) {
            case test_result::ok:
                ok_count++;
                break;
            case test_result::fail:
            case test_result::rethrown:
                failures_count++;
                break;
            case test_result::ex:
            case test_result::ex_ctor:
                exceptions_count++;
                break;
            case test_result::warn:
                warnings_count++;
                break;
            case test_result::term:
                terminations_count++;
                break;
            case test_result::skipped:
                skipped_count++;
                break;
            case tut::test_result::dummy:
                assert( (tr.result != tut::test_result::dummy) && "Should never be called");
        } // switch

        if ( (tr.result != tut::test_result::ok) &&
             (tr.result != tut::test_result::skipped) )
        {
            not_passed.push_back(tr);
        }
    }

    void run_completed()
    {
        os << std::endl;

        if (not_passed.size() > 0)
        {
            not_passed_list::const_iterator i = not_passed.begin();
            while (i != not_passed.end())
            {
                tut::test_result tr = *i;

                os << std::endl;

                os << "---> " << "group: " << tr.group
                << ", test: test<" << tr.test << ">"
                << (!tr.name.empty() ? (std::string(" : ") + tr.name) : std::string())
                << std::endl;

#if defined( _WINDOWS )
                std::stringstream ss;
                ss << std::endl;

                ss << "---> " << "group: " << tr.group
                << ", test: test<" << tr.test << ">"
                << (!tr.name.empty() ? (std::string(" : ") + tr.name) : std::string() )
                << std::endl;
                ::OutputDebugStringA( ss.str().c_str() );
#endif

#if defined(TUT_USE_POSIX)
                if(tr.pid != getpid())
                {
                    os << "     child pid: " << tr.pid << std::endl;
                }
#endif
                os << "     problem: ";
                switch(tr.result)
                {
                case test_result::rethrown:
                {
                    os << "assertion failed in child" << std::endl;
#if defined( _WINDOWS )
                    std::stringstream ss;
                    ss << "assertion failed in child" << std::endl;
                    ::OutputDebugStringA( ss.str().c_str() );
#endif
                    break;
                }
                case test_result::fail:
                {
                    os << "assertion failed" << std::endl;
#if defined( _WINDOWS )
                    std::stringstream ss;
                    ss << "assertion failed" << std::endl;
                    ::OutputDebugStringA( ss.str().c_str() );
#endif
                    break;
                }
                case test_result::ex:
                case test_result::ex_ctor:
                {
                    os << "unexpected exception" << std::endl;
                    if( tr.exception_typeid != "" )
                    {
                        os << "     exception typeid: "
                        << tr.exception_typeid << std::endl;
                    }
#if defined( _WINDOWS )
                    std::stringstream ss;
                    ss << "unexpected exception" << std::endl;
                    if( tr.exception_typeid != "" )
                    {
                        ss << "     exception typeid: "
                        << tr.exception_typeid << std::endl;
                    }
                    ::OutputDebugStringA( ss.str().c_str() );
#endif
                    break;
                }
                case test_result::term:
                {
                    os << "would be terminated" << std::endl;
#if defined( _WINDOWS )
                    std::stringstream ss;
                    ss << "would be terminated" << std::endl;
                    ::OutputDebugStringA( ss.str().c_str() );
#endif
                    break;
                }
                case test_result::warn:
                {
                    os << "test passed, but cleanup code (destructor) raised"
                        " an exception" << std::endl;
#if defined( _WINDOWS )
                    std::stringstream ss;
                    ss << "test passed, but cleanup code (destructor) raised"
                        " an exception" << std::endl;
                    ::OutputDebugStringA( ss.str().c_str() );
#endif
                    break;
                }
                default:
                    break;
                }

                if (!tr.message.empty())
                {
                    if (tr.result == test_result::fail)
                    {
                        os << "     failed assertion: `" << tr.message << "`"
                            << std::endl;
#if defined( _WINDOWS )
                        std::stringstream ss;
                        ss << "     failed assertion: `" << tr.message << "`"
                            << std::endl;
                        ::OutputDebugStringA( ss.str().c_str() );
#endif
                    }
                    else
                    {
                        os << "     message: `" << tr.message << "`"
                            << std::endl;
#if defined( _WINDOWS )
                        std::stringstream ss;
                        os << "     message: `" << tr.message << "`"
                            << std::endl;
                        ::OutputDebugStringA( ss.str().c_str() );
#endif
                    }
                }

                ++i;
            }
        }

        os << std::endl;

        os << "tests summary:";
#if defined( _WINDOWS )
        {
          std::stringstream ss;
          ss << "tests summary:";
          ::OutputDebugStringA( ss.str().c_str() );
        }
#endif
        if (terminations_count > 0)
        {
            os << " terminations:" << terminations_count;
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << " terminations:" << terminations_count;
            ::OutputDebugStringA( ss.str().c_str() );
#endif
        }
        if (exceptions_count > 0)
        {
            os << " exceptions:" << exceptions_count;
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << " exceptions:" << exceptions_count;
            ::OutputDebugStringA( ss.str().c_str() );
#endif
        }
        if (failures_count > 0)
        {
            os << " failures:" << failures_count;
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << " failures:" << failures_count;
            ::OutputDebugStringA( ss.str().c_str() );
#endif
        }
        if (warnings_count > 0)
        {
            os << " warnings:" << warnings_count;
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << " warnings:" << warnings_count;
            ::OutputDebugStringA( ss.str().c_str() );
#endif
        }

        os << " ok:" << ok_count;
#if defined( _WINDOWS )
        {
          std::stringstream ss;
          ss << " ok:" << ok_count;
          ::OutputDebugStringA( ss.str().c_str() );
        }
#endif

        if(skipped_count > 0)
        {
            os << " skipped:" << skipped_count;
#if defined( _WINDOWS )
            std::stringstream ss;
            ss << " skipped:" << skipped_count;
            ::OutputDebugStringA( ss.str().c_str() );
#endif
        }
        os << std::endl;
#if defined( _WINDOWS )
        {
          std::stringstream ss;
          ss << std::endl;
          ::OutputDebugStringA( ss.str().c_str() );
        }
#endif
    }

    virtual bool all_ok() const
    {
        return not_passed.empty();
    }

private:

    void init()
    {
        ok_count = 0;
        exceptions_count = 0;
        failures_count = 0;
        terminations_count = 0;
        warnings_count = 0;
        skipped_count = 0;
        not_passed.clear();
    }
};

}

#endif

Need a function to check if an exception has been raised

It would be nice to have a function like ensure_throw that asserts that an error has been thrown.

If you are interested, I can try to make one.

Why use global include ?

Hi, thanks for the library, but I have one question when I try to integrate it into android java \ cpp app, I can't build it with cmake by default, because of some global includes inside the library:
#include <tut/tut_config.hpp> for example.

Yes, I fixed this by specifying cmake work with such <> includes inside library module, but whether or not there is some background about such include style?

thanks in advance,

Files including tut header don't compile

err: In file included from /usr/local/include/tut/tut_exception.hpp:5:0,
                 from /usr/local/include/tut/tut.hpp:14,
                 from ../test.cpp:1:
/usr/local/include/tut/tut_result.hpp: In constructor ‘tut::test_result_posix::test_result_posix()’:
/usr/local/include/tut/tut_result.hpp:44:22: error: ‘getpid’ was not declared in this scope
         : pid(getpid())

Tag a new release

Last release is from 2013, can you tag a new one?

Installation fails with Python 3.7

Installation with Python 3.7 results in the following error:

Traceback (most recent call last):
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 114, in waf_entry_point
    run_commands()
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 171, in run_commands
    parse_options()
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 144, in parse_options
    Context.create_context('options').execute()
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Options.py", line 146, in execute
    super(OptionsContext,self).execute()
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Context.py", line 93, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Context.py", line 134, in recurse
    user_function(self)
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/wscript", line 11, in options
    opt.load('compiler_cxx')
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Context.py", line 90, in load
    fun(self)
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Tools/compiler_cxx.py", line 36, in options
    opt.load_special_tools('cxx_*.py')
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Context.py", line 321, in load_special_tools
    lst=self.root.find_node(waf_dir).find_node('waflib/extras').ant_glob(var)
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Node.py", line 361, in ant_glob
    ret=[x for x in self.ant_iter(accept=accept,pats=[to_pat(incl),to_pat(excl)],maxdepth=kw.get('maxdepth',25),dir=dir,src=src,remove=kw.get('remove',True))]
  File "/private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-nxope4ko/spack-src/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Node.py", line 361, in <listcomp>
    ret=[x for x in self.ant_iter(accept=accept,pats=[to_pat(incl),to_pat(excl)],maxdepth=kw.get('maxdepth',25),dir=dir,src=src,remove=kw.get('remove',True))]
RuntimeError: generator raised StopIteration

This is due to a bug in waf that was fixed about a year ago.

Can you update to the latest version of waf so tut can be built with the latest version of python?

documentation out of sync: introductory example

In index.html#creating_1 the callback class refers to 'test_pos'. This member does not seem to be present in tut_result.hpp and gcc suggests 'test'. Looking at tut_result.hpp, gcc appears to be right.

Build fails with -Werror=terminate

03:28:27 runner system command -> ['x86_64-pc-linux-gnu-g++', '-Werror=terminate', '-march=native', '-O2', '-pipe', '-O2', '-Wall', '-Wextra', '-Weffc++', '-ftemplate-depth-100', '-Idefault/include', '-I../include', '../selftest/teardown_ex.cpp', '-c', '-o', 'default/selftest/teardown_ex_2.o']
../selftest/teardown_ex.cpp: In destructor ‘virtual tut::teardown_ex::dummy::~dummy()’:
../selftest/teardown_ex.cpp:30:55: error: throw will always call terminate() [-Werror=terminate]
                 throw runtime_error("ex in destructor");
                                                       ^
../selftest/teardown_ex.cpp:30:55: note: in C++11 destructors default to noexcept

Details here: https://bugs.gentoo.org/show_bug.cgi?id=601962

A group per source file

I'd like to create one test group per source file, however, tests do not run when the runner is statically linked.

Minimal example:

m1.C

#include <iostream>
#include <tut/tut.hpp>

namespace tut 
{
    struct group1{};
    
    typedef test_group<group1> tg;
    tg test_group_1("module 1: group 1");
    typedef tg::object testobject;
    
    template<> template<>
    void testobject::test<1>()
    {
        set_test_name("module 1, group 1, test 1");
        std::cout << "module 1, group 1, test 1\n";
    }
}

m2.C

#include <iostream>
#include "tut/tut.hpp"

namespace tut 
{ 
    struct group2{};
    
    typedef test_group<group2> tg;
    tg test_group_2("module 2: group 2");
    typedef tg::object testobject;
    
    template<> template<>
    void testobject::test<1>()
    {
        set_test_name("module 2, group 2, test 1");
        std::cout << "module 2, group 2, test 1\n";
    }
}

main.C

#include <tut/tut.hpp>

namespace tut
{
    test_runner_singleton runner;
}

int main()
{
    tut::runner.get().run_tests();
    return 0;
}

Build shared works:

$ gcc -I. -c m1.C
$ gcc -I. -c m2.C
$ g++ -I. -c main.C
$ g++ -o test main.o m1.o m2.o
$ file ./test
./test: ELF 64-bit LSB pie executable x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=d620007974a67f95c4b486b862718c9433670b06, not stripped
$ ./test
module 1, group 1, test 1
module 2, group 2, test 1

Build static runs no tests:

$ gcc -I. -c m1.C
$ gcc -I. -c m2.C
$ g++ -I. -c main.C
$ ar rcs libm1.a m1.o
$ ar rcs libm2.a m2.o
$ g++ -static main.C -I. -L. -lm1 -lm2 -o test-static
$ file ./test-static 
./test-static: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=cf4782d823ae547ecdccd3ce39f20e5bc1a7edff, not stripped
$ ./test-static
$

As shown above, linking statically does not run the tests. This is probably not a TUT issue, and I'm probably not using this correctly. Does this have something to do with the static test_runner_singleton? I tried pulling in the runner from main.C into m1.C and m2.C as extern but without luck. How do I ensure that tests defined in different compilation units register with the same runner that's (potentially) defined in another file? (And why does it register with the correct runner linking dynamically, but not statically?)

port wscript to newer waf

The bundled waf is very old (1.5.12) while the current version is 1.8.8. The included wscript doesn't work with waf version 1.8.8:
$ waf --help
Waf: The wscript in '/builddir/build/BUILD/tut-framework-2013-12-18' is unreadable
Traceback (most recent call last):
File "/usr/share/waf/waflib/Scripting.py", line 92, in waf_entry_point
set_main_module(os.path.join(Context.run_dir,Context.WSCRIPT_FILE))
File "/usr/share/waf/waflib/Scripting.py", line 117, in set_main_module
Context.g_module=Context.load_module(file_path)
File "/usr/share/waf/waflib/Context.py", line 328, in load_module
exec(compile(code,path,'exec'),module.dict)
File "/builddir/build/BUILD/tut-framework-2013-12-18/wscript", line 6, in
import Options
ImportError: No module named Options