Having a comma in the CESTER_COMMENT macro causes the pre-processor to treat the comment value as a parameter to the cester_comment() function.

#include <exotic/cester.h>

CESTER_COMMENT(
    Because of this comma, the pre-processor sends as \n
    multiple parameters to the internal cester_comment(0 function
);

E.g

CESTER_TEST(test_1, inst,
      cester_assert_true(0, "This fails for it zero");
)

Result

AssertionError test_assert_true_false.c:6: in 'test_1' expr => 'This fails for it zero'

From master branch.

Command ./tests --cester-verbose --cester-minimal --cester-output=junitxml > out.xml produces

�[0m<?xml version="1.0" encoding="UTF-8" ?>
�[0m�[0m<testsuite tests="�[0m�[0m1�[0m�[0m" failures="�[0m�[0m0�[0m�[0m" name="�[0m�[0mtests�[0m�[0m" errors="0" skipped="�[0m�[0m0�[0m�[0m" time="�[0m�[0m0.00�[0m�[0m">
�[0m�[0m    <testcase classname="�[0m�[0mtests�[0m�[0m" name="�[0m�[0mtest_add_should_sum�[0m�[0m" time="�[0m�[0m0.00�[0m�[0m"�[0m�[0m/>
�[0m�[0m</testsuite>
�[0m

Output on terminal looks good.

If the test case executes beyond the expected time it should fail with ExecutionBeyoundTimeError or something.
There is no need to introduce a thread it should work in multiprocess so the main process can time how long it starts running and if it beyond the time it kills the test case process and reports it,

There should also be an option to allow it to execute complete and report the time the execution completes and the time it expected to complete so it fails on the differences.

Test compilation

Add magic and an awesome feature where you can test whether a test case compiles or not if it does not output the error message. The compiled source can be any language,
This can be achieved by setting the compilation executable, options for that test case then cester try to execute that with exec or similar function, then capture the output. This should be optional and can be disabled with #define CESTER_IGNORE_COMPILATION_TEST

Sample format

CESTER_COMPILE_TEST(does_this_so_code_compiles, inst, "gcc", "-ansi --pedantic-errors", {
    #include <stdio>
    int main() {
        printf("yahoo\n");
    }
})

CESTER_COMPILE_TEST(does_this_python_code_runs, inst, "python", "", {
    import io
    print("hello python")
})

The first argument is the test case, the second is the test instance, the third is the compiler executable, the fourth is the code to compile (this is converted to string so there should be a way to remove the enclosing { and };

Test execution

cester should be able to test whether a program runs and validates the output is what is expected similar to CESTER_COMPILE_TEST

CESTER_EXECUTE_TEST(execute_program, inst, "program.exe", "--op", {
    cester_assert_program_output_eq(execute_program, "Unknow option --op");
})

the first argument is the test case name, the second is the test instance, the third is the program or full path to the program o execute, the fourth is the argument to send to the program, the last is the test case body

Allow the ability to execute the test interactively.

• Can execute the previous test
• Execute a particular test by name or index
• Change text cases parameters
• Can see the time, memory took e.t.c to execute a test case
• E.t.c.

On linux with C++ compiler if a test case crash and was recovered by signal, the second time another test case crash signal is not able to recover and intercept the segfault, this bug is only present on Linux with C++ compiler, clang, and gcc.

The solution is to simply skip other test cases if one crash occur, but that would not be the default behavior because compiling the test as C or C++ on other platforms the issue is not present, instead accept a cli flag --cester-skiponfirstcrash and CESTER_SKIP_ON_FIRST_CRASH() this way the option can be supplied to faulty binaries only

Add a new test case macro that allows selectively execution features, can accept options for the compiler, platform, and architecture.
Should execute test if it matched the + selection and ignore if matches the - selection. Using bitflag for a more seamless selection. Should be optional for compilation.

Proposed format

CESTER_TEST_SELECTIVE(test_name, arg, {
    // test body
    cester_assert_nothing();
}, 
// fourth argument is flags of selected platforms, compilers, and architectures 
CESTER_SELECTIVE_PLATFORM_ALL | CESTER_SELECTIVE_COMPILER_ALL | CESTER_SELECTIVE_ARCH_ALL
,
// fifth argument is flags of platforms, compilers, and architectures to ignore
CESTER_SELECTIVE_RASPBERRY_PI
)

Possible CESTER_SELECTIVE_* bit flags

• CESTER_SELECTIVE_PLATFORM_ALL
• CESTER_SELECTIVE_COMPILER_ALL
• CESTER_SELECTIVE_ARCH_ALL
• CESTER_SELECTIVE_NONE
• CESTER_SELECTIVE_PLATFORM_WINDOWS
• CESTER_SELECTIVE_PLATFORM_LINUX
• CESTER_SELECTIVE_PLATFORM_MACOS
• CESTER_SELECTIVE_PLATFORM_UNIX
• CESTER_SELECTIVE_PLATFORM_RASPBERRY_PI
• CESTER_SELECTIVE_PLATFORM_ANDRUINO
• CESTER_SELECTIVE_PLATFORM_HAIKU
• CESTER_SELECTIVE_COMPILER_CLANG
• CESTER_SELECTIVE_COMPILER_GCC
• CESTER_SELECTIVE_COMPILER_TCC
• CESTER_SELECTIVE_COMPILER_MSVC
• CESTER_SELECTIVE_ARCH_ARM
• CESTER_SELECTIVE_ARCH_X86
• CESTER_SELECTIVE_ARCH_X64
• CESTER_SELECTIVE_ARCH_X86_64

Discussed in #54

Please add support for for multiple C/C++ files containing assertions/tests. Ideally it should be possible to build the tests with a single command, and then run the tests with a single command.

Not just download the header file into the current folder, create the folder include/exotic/ in the folder and install it to it.

The doc says that everything in the macro is printed out at the beginning of the test. However, in my environment of mingw64 in msys2 on windows, the following snippet doesn't print the message.

#include <exotic/cester.h>

CESTER_COMMENT(
    This is a multiline comment in \n
    the source code this text is printed out \n
    before any test cases is executed.
)

CESTER_TEST(foo_bar, test_instance,
	cester_assert_cmp_msg(10,>,2,"is 10 greater than 2");
)
	
CESTER_TEST(foo_zoo, test_instance,
	cester_assert_int_gt(1, 2);
)

After compiling it with gcc test.c -I. -o test, the exe only prints the test results without the comment at the beginning. Am I missing anything?

Add new cli flags to specify test cases to skip as alternate to the flag that specified the tests to run. Also add flag to list all the tests in the executable.

Also add flag to run todo or skep tests (CEATER_TODO_TEST, CESTER_SKIP_TEST).

If neither _WIN32 nor unix are defined, the following compilation error occurs:

libcester/include/exotic/cester.h:3199:13: error: label 'end_sub_process' used but not defined
 3199 |             goto end_sub_process;
      |             ^~~~

The label end_sub_process is only defined for the _WIN32 or unix case, but not in the 3rd case fallback.

I guess this block of code needs to be disabled in this situation.

Allow multiple definition of supplement macros like CESTER_BODY, CESTER_OPTIONS, CESTER_COMMENT, CESTER_BEFORE_ALL, CESTER_BEFORE_EACH, CESTER_AFTER_ALL and CESTER_AFTER_EACH

I have an issue with this code

CESTER_CAPTURE_STDERR();
__wrap_infrsprin_(&n,rp,ip,c,&torq1);
cester_assert_stderr_stream_content_contain("constraint");
CESTER_RELEASE_STDERR();

that seems to not capture stderr stream content

The cester.ps1 powershel script should auto detect the Visual Studio C++ header folder and install into it.

Support for multiple functions for a single macro function below:

CESTER_OPTIONS
CESTER_BEFORE_ALL
CESTER_BEFORE_EACH
CESTER_AFTER_ALL
CESTER_AFTER_ALL

# and in future

CESTER_SETUP
CESTER_TEARDOWN

The method is to concatenate the line number with the name of the function like:

#define CESTER_OPTIONS(x) void cester_options_before_main_##__LINE__();

In the file that contains my tests, I would like to define a structure that holds all of my test configurations and is then used to initialize the arg pointer in CESTER_BEFORE_ALL. So, something like this:

struct MyTestSetup
{
   ...
};

CESTER_BEFORE_ALL(inst,
    class MyTestSetup setup;
    inst->arg = &setup;
)

However, because the mechanism include the file multiple times, I can't do this because it will fail compilation. I also cannot put the structure definition into the CESTER_BEFORE_ALL definition because then the individual tests won't see it.

So, I can make it work by doing

#ifndef DO_ONCE
#define DO_ONCE
struct MyTestSetup
{
   ...
};
#endif

Not horrible, but not pretty.

Is there a macro that I did not see that allows me to do this more cleanly?

If not, its not a high priority but I would suggest that there be a CESTER_FILE_GLOBAL() section, or something that does the equivalent of the DO_ONCE hack and allows you to put file global type definitions and variables.

Check for memory leak in CESTER_BEFORE_ALL in CESTER_AFTER_ALL, So allocated byte in CESTER_BEFORE_ALL is deallocated in CESTER_AFTER_ALL. Same application to CESTER_BEFORE_EACH and CESTER_AFTER_EACH

Add a new flag --cester-detail that prints out the detail of a test executable, this flag should works with the --cester-ouput=* flag so the output format can be based on which is specified

• Text
• JunitXML
• Tap
• Tap Version 13
• JSON
• Key Value

Make the assertion macro print more information in the output, such as the expression that gave a particular output.

For the test case

CESTER_TEST(test_xarray_init, inst, 
    XArray* array;
    enum xtypes_stat status = xarray_new(&array);
    cester_assert_uint_eq(status, X_OK);
)

Compile and run test with the verbose option on :

Currently

Passed test_xarray.c:18: in 'test_xarray_init' expr => 'expected 0, received 0'

Proposed

Passed test_xarray.c:18: in 'test_xarray_init' expr => 'expected status=0, received L_OK=0'

The proposed should be the output with verbose on and the currently is when no verbose is on

The behavior of the following macros was not verified currently. This issue is a reminder.

CESTER_BEFORE_ALL
CESTER_BEFORE_EACH
CESTER_AFTER_EACH
CESTER_AFTER_ALL

Expecting different behaviour on Unix base system and windows

• Make failing line print red instead of just grey and red marker
• add s or m as in seconds or minute to the time displayed before the test name
• can register for a test to segfault
• can register for a test to fail
• can register for a test to be prematurely terminated
• expected timeout for a test case. can extend the timeout for a test case
• test whether a test case output to console, stderr and stdout
• make the library compatible with C 89 and ANSI
• add assertion macros to test for types range

see https://www.decodejava.com/data-types-in-c.htm https://en.wikipedia.org/wiki/C_data_types

See https://github.com/testing-cabal/subunit

cester.h appears to have \r\n at the end of each line.

Normally, source code in git has just \n at the end, and git automatically transforms these to \r\n pairs if a checkout is done on Windows and transforms them back to \r\n when the commit is done.

 % curl -s https://raw.githubusercontent.com/exoticlibraries/libcester/master/include/exotic/cester.h | cat -v |head
^M
/**^M
    \copyright MIT License Copyright (c) 2020, Adewale Azeez ^M
    \author Adewale Azeez <[email protected]>^M
    \date 10 April 2020^M
    \file cester.h^M
^M
    Cester is a header only unit testing framework for C. The header ^M
    file can be downloaded and placed in a project folder or can be ^M
    used as part of libopen library by including it in the projects ^M
%

This is generally considered incorrect.

For example, when I updated my cester.h to the new version, I got this warning:

 % git commit -m 'now follows CESTER specs' -a
warning: CRLF will be replaced by LF in src/tests/cester.h.
The file will have its original line endings in your working directory
[slg-dev fbfbd8a] now follows CESTER specs
 3 files changed, 69 insertions(+), 79 deletions(-)
 create mode 100644 src/tests/Makefile
%

You may wish to review these articles and update it accordingly:

• https://docs.github.com/en/github/using-git/configuring-git-to-handle-line-endings
• https://stackoverflow.com/questions/10418975/how-to-change-line-ending-settings
• https://www.edwardthomson.com/blog/git_for_windows_line_endings.html

Manage and test for memory leak with memory allocated with calloc similar to current malloc management.

Create the install script in the website repo that accepts parameters to specify what exotic library to download, which should work for any header-only library.

OK, the current test reporter is garbage and not easy to manage at all. I am looking at portion the library to systems with very small memory and interfaces hence there might be a need to remote failures or errors in a non-textual way, e.g. on andruino failure can be reported with red light and success can be reported with a green light.

e.g.
To report the test execution on andruino using a red and green led

CESTER_REPORTER_FINAL(_, exit_code, {
     if (exit_code == EXIT_SUCCESS) {
         digitalWrite(GREEN_LED_BUILTIN, HIGH);
         return;
    }
    digitalWrite(RED_LED_BUILTIN, HIGH);
});

In the current implementation it possible but that will be a crazy task to achieve, with a proper reporter interface that will be easily achievable and more test report format like subunit, key-value can be easily integrated.

Also, the new report should provide an interface to intercept or/and allow the creation of custom reports.

CESTER_REPORTER_EXPECT_ACTUAL(cr1, pass, test_case, expect, actual, {
     if (pass == 1) {
         printf("The test in test case %s passes --- expected %s received %s\n", test_case->name, expect, actual);
    }
});

Text

  + (0.00s) test assert equal
  + (0.00s) test assert not equal

Passed test_assert_test_verbose_levels.c:6: in 'test_assert_equal' => expected 'NULL=0000000000000000', received '((void*)0)=0000000000000000'
Passed test_assert_test_verbose_levels.c:10: in 'test_assert_not_equal' => not expecting 'NULL', received '"NotEqual"'
Passed test_assert_test_verbose_levels.c:11: in 'test_assert_not_equal' expr => '(1)'

Ran 2 test(s) in 0.00 Seconds
Synthesis: SUCCESS Tests: 2 | Passing: 2 | Failing: 0

key-value

total_test = 2
passing = 2
failing = 0
file_name = test_assert_test_verbose_levels

[test_assert_equal]
total_test = 1
passing = 1
failing = 0
time_taken = 0.00s
output = Passed test_assert_test_verbose_levels.c:6: in 'test_assert_equal' => expected 'NULL=0000000000000000', received '((void*)0)=0000000000000000'

[test_assert_not_equal]
total_test = 2
passing = 2
failing = 0
time_taken = 0.00s
output = Passed test_assert_test_verbose_levels.c:10: in 'test_assert_not_equal' => not expecting 'NULL', received '"NotEqual"' \
                           Passed test_assert_test_verbose_levels.c:11: in 'test_assert_not_equal' expr => '(1)'

It would be useful to be able to define a test that uses C++ templates to test template functions. For example, if I have a function like this:

template <typename T>
inline bool isSubnormal(const T & v)
{
    return std::isfinite(v) && !std::isnormal(v) && v != T(0.0);
}

Currently, my tests look like this:

CESTER_TEST(isSubnormalFloatWithSubnormalInput, inst,
    float p1 = std::numeric_limits<float>::min() / float(2.0);
    cester_assert_true(isSubnormal(p1));
)
CESTER_TEST(isSubnormalDoubleWithSubnormalInput, inst,
    double p1 = std::numeric_limits<double>::min() / double(2.0);
    cester_assert_true(isSubnormal(p1));
)

It would be great to do something like:

CESTER_TEMPLATE_TEST(isSubnormalWithSubnormalInput, inst, T, (float, double, long double),
    T p1 = std::numeric_limits<T>::min() / T(2.0);
    cester_assert_true(isSubnormal(p1));
)

Basically - add parameters specifying the template variable and the list of types to instantiate.

It would have to generate names for each of the tests that include an indication what T was for that test.

I have not really thought through the ideal syntax, because if you have more than one template variable, my suggested syntax is not very practical, but you get the idea.....

Create macros to invoke the before and after function

CESTER_BEFORE_ALL()
CESTER_BEFORE_EACH()
CESTER_AFTER_ALL()
CESTER_AFTER_EACH()

Attache to superTestInstance.registered_test_cases

cester_fatal_assert_not_null(user);
cester_assert_true(1);

If user is null the second assertion will not run, the test case will terminate immediately

Firstly, thanks for the awesome testing framework.

I've used libcester on termux, but failed to compile the given example. The errors are as follows:

In file included from ./test.c:2:
/data/data/com.termux/files/usr/local/include/exotic/cester.h:283:10: error: field has incomplete type 'FILE' (aka 'struct __sFILE')
    FILE original_stream;                   /**< The actual address of the captured stream. For internal use only.*/
         ^
/data/data/com.termux/files/usr/include/stdio.h:61:8: note: forward declaration of 'struct __sFILE'
struct __sFILE;
       ^
In file included from ./test.c:2:
/data/data/com.termux/files/usr/local/include/exotic/cester.h:407:10: error: field has incomplete type 'FILE' (aka 'struct __sFILE')
    FILE output_stream_address;                         /**< Output stream address. incase the output stream was captured in test it state can be reset. For internal use only. */
         ^
/data/data/com.termux/files/usr/include/stdio.h:61:8: note: forward declaration of 'struct __sFILE'
struct __sFILE;
       ^
In file included from ./././test.c:2:
/data/data/com.termux/files/usr/local/include/exotic/cester.h:3880:13: error: incomplete type 'FILE' (aka 'struct __sFILE') is not assignable
    *stream = *(captured_stream->replaced_stream_handle);
    ~~~~~~~ ^
/data/data/com.termux/files/usr/include/stdio.h:61:8: note: forward declaration of 'struct __sFILE'
struct __sFILE;
       ^
In file included from ./././test.c:2:
/data/data/com.termux/files/usr/local/include/exotic/cester.h:3959:33: error: incomplete type 'FILE' (aka 'struct __sFILE') is not assignable
                        *stream = *(captured_stream->replaced_stream_handle);
                        ~~~~~~~ ^
/data/data/com.termux/files/usr/local/include/exotic/cester.h:367:43: note: expanded from macro 'CESTER_ARRAY_FOREACH'
                                          z\
                                          ^
/data/data/com.termux/files/usr/include/stdio.h:61:8: note: forward declaration of 'struct __sFILE'
struct __sFILE;
       ^
4 errors generated.

My compiler is clang of version 15 on termux. Then I check it on windows with clang of the same version, and no error happens.

Maybe this error is related to the termux platform, but I'm not sure. If you are free, please check it. Thanks!

I used something like this:

CESTER_BEFORE_ALL(inst,  
    std::vector setup;
    inst->arg = & setup;
)

This did not work because the destructor for 'setup' was called before any of the tests were actually called.

This should be a big Warning in the documentation for CESTER_BEFORE_ALL. I know the example uses dynamic allocation which gets around the problem, but it does not explain why. I think it needs a clear statement that you must use data that is not automatically deallocated.

In hindsight I understand why this is, but it took me a while to figure out why I was crashing and I think others will make the same mistake.

I've added cester to a system and I'm getting this warning, which is perplexing:

./cester.h:3277:17: warning: Isolated tests not supported in this environment. The tests will be run on the main process [-W#pragma-messages]
        #pragma message("Isolated tests not supported in this environment. The tests will be run on the main process")
                ^
1 warning generated.

It still works, but I'd like to know what the warning means. It's not in the documentation.

Use the iterator for a more clean recovery method instead of jmp_buf.
The way this will work is a super instance index will increase per the test case index so when a test case crashes the run_test_iterator will be invoked with the next test index instead of starting over again, or as of today using jmp_buf to jump to the point of recovery.

I am new beginner for this framework.
I found some annoying warning message during my tutorial.

How to reproduce

// test.c
#include <exotic/cester.h>
 
CESTER_TEST(test_one, inst,
    cester_assert_nothing();    
)

$ gcc -I./ test.c -o test.out
In file included from test.c:1:
./exotic/cester.h:481:5: warning: initialization of ‘int’ from ‘void *’ makes integer from pointer without a cast [-Wint-conversion]
  481 |     CESTER_NULL,                                     /* output_stream_address */
      |     ^~~~~~~~~~~
./exotic/cester.h:481:5: note: (near initialization for ‘superTestInstance.output_stream_address._flags’)

my environment

$ uname -a
Linux foo 6.5.0-27-generic #28~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 15 10:51:06 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

temporary solution

I managed to prevent the warning message from appearing by using -Wno-int-conversion option.

Please help newbie. Thanks :)

all cester cester_assert_ptr_not_equal, cester_assert_str_equal
is causing segfault in a pointer to pointer scenerio, fix in libcester

char **ptr;

Add generator to generate values between ranges,
To generate assertions for ranges of options

CESTER_TEST(generators, inst, {
    CESTER_GENERATE_INT_RANGE(x,1, 30, {
        cester_assert_int_eq(x, y);
    })
})

or better format

• add logic to send a kill signal to a test case or running program case or compilation case after some time or at a specific time, so the user termination can be tested
• fix libcester tap format printing multiple #
• check whether malloc is actually called for a malloc with a different name
• add libcester to madeinnigeria repo
• add JSON output format
• add key-value output format

In file included from /path/tests/tests.c:1:0:
/path/_deps/libcester-src/include/./exotic/cester.h: In function ‘cester_run_test’:
/path/build/_deps/libcester-src/include/./exotic/cester.h:3259:9: error: unknown type name ‘pid_t’
         pid_t pid;

I think the problem occurs because older POSIX (so older Linux API) defines pid_t only in <sys/types.h>. POSIX.1-2001 defines it also in <unistd.h> and cester.h includes only the last one.
I would recommend to add additional <sys/types.h> maybe in some #ifdef statement checking available POSIX implementation.

I found this issue on CentOS 7.4 with 3.10.0-957 kernel.

Bug Reproduction

#include <stdio.h>
#include <exotic/cester.h>

CESTER_BODY(
int global_size = 1;
int size() {
    printf("Getting size %d\n", global_size);
    global_size++;
    return global_size;
}
)

CESTER_TEST(test_init_set, _, {
    cester_assert_int_eq(size(), 1);
})

CESTER_OPTIONS(
    CESTER_VERBOSE_LEVEL(2);
)

Notice it prints out "Getting size X" more than once