silentbicycle / greatest Goto Github PK

I am trying to compile with gcc 4.8.2 and the Makefile from git:

$ LANG=C make
cc -Wall -Werror -Wextra -pedantic -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations -c -o example-suite.o example-suite.c
example-suite.c:9:6: error: function declaration isn't a prototype [-Werror=strict-prototypes]
TEST blah() {
^
cc1: all warnings being treated as errors
: recipe for target 'example-suite.o' failed
make: *** [example-suite.o] Error 1

This can be fixed by changing the offending line to
TEST blah(void) {
The same change has to be made at 8 other positions in example.c.

After that some more errors pop up:
$ LANG=C make
cc -o example example.c example-suite.o -Wall -Werror -Wextra -pedantic -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations
example.c: In function 'suite':
example.c:94:5: error: passing argument 1 of 'GREATEST_SET_SETUP_CB' from incompatible pointer type [-Werror]
GREATEST_SET_SETUP_CB(trace_setup, NULL);
^
In file included from example.c:5:0:
greatest.h:180:6: note: expected 'void ()(void *)' but argument is of type 'void ()(void)'
void GREATEST_SET_SETUP_CB(greatest_setup_cb cb, void *udata);
^
example.c:95:5: error: passing argument 1 of 'GREATEST_SET_TEARDOWN_CB' from incompatible pointer type [-Werror]
GREATEST_SET_TEARDOWN_CB(trace_teardown, NULL);
^
In file included from example.c:5:0:
greatest.h:181:6: note: expected 'void ()(void )' but argument is of type 'void ()(void)'
void GREATEST_SET_TEARDOWN_CB(greatest_teardown_cb cb, void *udata);
^
cc1: all warnings being treated as errors
Makefile:12: recipe for target 'example' failed
make: ** [example] Error 1

This can be resolved by changing trace_setup() and trace_teardown() to require a void* argument and using it.

It'll be nice to have color in the output. Red for failed tests, Green for passed tests, and so on.

After a little search I found this simple way of doing it.

Is there any aspects to consider in order to implement this? Or is it as straight forward as it seems?

Thanks!
Fran

greatest could get a flag for running long tests, a flag for only running quick tests, or some way to filter them out.

It's not unusual for tests to include both quick-running tests, which are run frequently during active development, and a few tests which take much longer to run due to generating large data sets, extensive fuzzing or property-based testing, etc. This distinction tends to cross-cut other filtering methods like suite grouping or name filters.

This is mostly an API design question -- the implementation should be pretty simple. Is it better to run long tests by default or not? Running them may be a better default, to make CI system config a bit simpler.

This could be implemented as:

• a long-tests flag (and a -L flag in the CLI runner)
• a quick-tests-only flag (and a -q flag in the CLI runner)
• a negative name filter (e.g. -T _slow_, to not run tests with _slow_ in the name), since the test filtering logic is already isolated to greatest_name_match.

The negative name filter may be preferable, because it's a more generally useful mechanism.

Would you be open to add automatic test detection to the next major version ?

Similarly to what Catch C++ test library does
https://github.com/philsquared/Catch/blob/master/docs/tutorial.md

I like the simplicity of your library but not having to modify several files to add tests suites and tests would be a nice feature IMO

All double quotation marks' positions followed by macros got messy when I formated the codes from eclipse and vs2017. They look like this:

/* A test runs various assertions, then calls PASS(), FAIL(), or SKIP(). */
TEST x_should_equal_1(void)
{
    int x = 1;
    /* Compare, with an automatic "1 != x" failure message */
    ASSERT_EQ(1, x)
    ;

    /* Compare, with a custom failure message */
    ASSERT_EQm("Yikes, x doesn't equal 1", 1, x)
    ;

    /* Compare, and if they differ, print both values,
     * formatted like `printf("Expected: %d\nGot: %d\n", 1, x);` */
    ASSERT_EQ_FMT(1, x, "%d")
    ;
    PASS()
    ;
}

How to resolve it?
Thank you.

While the test name (-t) and suite name (-s) filter flags' default substring match is convenient, sometimes only one test should run. Sometimes exclude (-x) covers this, but particularly when automating running a list of tests by name, being able to require exact match would be better.

Add a flag (-e, perhaps) and a corresponding flag setter function (greatest_exact_name_match()), and update the test and suite name logic to account for it. (When on, it should apply to both -- it's probably not worth having distinct exact-match flags for the suite name and for the test name.)

Since this is an interface change (new feature), it'll need to wait until 1.5.0.

There should be a webpage for greatest, and an address in the header for the up-to-date version. Since greatest is typically copied into projects, it's easy to stick with a very old version.

Passing the FILE* in would be better, since GREATEST_STDOUT is user-configurable.

In your blog, you said:

Of course, most of the actual testing code is in the test cases. Every test is a function with zero or more assertions, then a call to PASS(), FAIL(), or SKIP(). (There are variants that return a custom message, PASSm(msg), FAILm(msg), and SKIPm(msg).)

you explained clear about how to use PASS(), FAIL(), or SKIP() but did not say it clear why we need them?

For example, if one assertion failed, PASS() here makes no senses on me. Assertion failing is enough to say something. So why we need PASS(), FAIL(), or SKIP()?

Thanks,
regards.

One handy feature (I find) of cmocka is that it raises a SIGABRT on a test failure. Paired with gdb, this will directly drop you in the failing line.

Would you accept a patch that introduce an optional flag -a (would imply -f) which raises an abort signal on the first encountered failure? Maybe the feature is already there and my quick glancing was not enough.

While integrating the upstream longjmp-based assert implementation, I've just realised something:

Isn't making greatest_save_context a regular function and calling setjmp inside it invalid? (undefined behaviour) It's my understanding that you can't safely setjmp(), destroy the stack frame it was called from, and then longjmp back into it? So greatest_save_context really ought to be a macro, or possibly a forced-inline function.

The code as it stands seems to actually work in the cases I've thrown at it so far, probably because the return pointer in the test function's stack frame is in the same place as in greatest_save_context() in my case, and the return value (rax) is used as the same variable in either case.

        greatest_test_res res = greatest_save_context();
        if (res == GREATEST_TEST_RES_PASS) {
            res = TEST(); // <- return pointer actually causes greatest_save_context() to return here after longjmp?
        }

I haven't verified this theory at the assembly/register level, but I'm fairly sure the current use of setjmp/longjmp isn't actually legal, despite it appearing to work OK on my system. (OSX/x86-64)

I'll put together a suggested fix.

So failed test can report exact numbers that differ, not just that they do.

While RE search isn't likely (due to dynamic allocation), greatest_pre_test and greatest_run_suite could be changed to use substring matching instead of full strcmp.

Same usage patterns as ASSERT_STR_EQ, but use memcmp instead of strcmp.

The elapsed time printouts are based on clock, but could use gettimeofday, clock_gettime, etc. after checking whether the appropriate symbols are defined. This would give more accurate elapsed wall clock time.

A few of the assert macros use internal variables to capture the macro arguments. The internal variables names are short, though (e.g. "exp", "got"), and if they match the names in user code, they can expand to to lines like const char *fmt = fmt;. These names should be prefixed, to reduce the likelihood of collisions.

Hi,

I have some tests which share the same function, and that function is passed the item to test as its void *ENV argument. I added this, so I can pass a useful name for each of those tests:

+/* adapted from GREATEST_RUN_TEST1() */
+#define GREATEST_RUN_TEST1_NAME(NAME, TEST, ENV) \
+    do {                                                                \
+        if (greatest_test_pre((NAME)) == 1) {                            \
+            enum greatest_test_res res = GREATEST_SAVE_CONTEXT();       \
+            if (res == GREATEST_TEST_RES_PASS) {                        \
+                res = TEST((ENV)) ;                        \
+            }                                                           \
+            greatest_test_post((NAME), res);                             \
+        }                                                               \
+    } while (0)

where NAME replaces #TEST, which was the same for each test.

I don't like making the API bigger just for this, because greatest's appeal is its balance of size and usefulness.

Example of the kind of array I have: https://github.com/katef/jvst/blob/master/tests/unit/test_ir.c#L137
I'll probably make them independent functions, and live with the duplication of setup and teardown inside each. I wanted to share the situation I have, in case it's interesting.

Minimal demonstration case:

#include "greatest.h"

TEST x_should_equal_1(void) {
  int x = 1;
  ASSERT_EQ_FMT(1, x, "%d");
  PASS();
}

SUITE(the_suite) {
  RUN_TEST(x_should_equal_1);
}

GREATEST_MAIN_DEFS();

int main (int argc, char** argv) {
  GREATEST_MAIN_BEGIN();
  RUN_SUITE(the_suite);
  GREATEST_MAIN_END();
}

When I try to compile this (as main.c) using clang -std=c11 -Weverything main.c, I get the following warning spew:

main.c:5:3: warning: format string is not a string literal [-Wformat-nonliteral]
  ASSERT_EQ_FMT(1, x, "%d");
  ^~~~~~~~~~~~~~~~~~~~~~~~~
./greatest.h:992:24: note: expanded from macro 'ASSERT_EQ_FMT'
#define ASSERT_EQ_FMT  GREATEST_ASSERT_EQ_FMT
                       ^
./greatest.h:389:5: note: expanded from macro 'GREATEST_ASSERT_EQ_FMT'
    GREATEST_ASSERT_EQ_FMTm(#EXP " != " #GOT, EXP, GOT, FMT)
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./greatest.h:442:38: note: expanded from macro 'GREATEST_ASSERT_EQ_FMTm'
            fprintf(GREATEST_STDOUT, greatest_FMT, EXP);                \
                                     ^~~~~~~~~~~~
main.c:5:3: warning: format string is not a string literal [-Wformat-nonliteral]
  ASSERT_EQ_FMT(1, x, "%d");
  ^~~~~~~~~~~~~~~~~~~~~~~~~
./greatest.h:992:24: note: expanded from macro 'ASSERT_EQ_FMT'
#define ASSERT_EQ_FMT  GREATEST_ASSERT_EQ_FMT
                       ^
./greatest.h:389:5: note: expanded from macro 'GREATEST_ASSERT_EQ_FMT'
    GREATEST_ASSERT_EQ_FMTm(#EXP " != " #GOT, EXP, GOT, FMT)
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./greatest.h:444:38: note: expanded from macro 'GREATEST_ASSERT_EQ_FMTm'
            fprintf(GREATEST_STDOUT, greatest_FMT, GOT);                \
                                     ^~~~~~~~~~~~
./greatest.h:208:18: warning: padding struct 'struct greatest_run_info' with 2 bytes to align 'tests_run' [-Wpadded]
    unsigned int tests_run;     /* total test count */
                 ^
./greatest.h:222:17: warning: padding struct 'struct greatest_run_info' with 4 bytes to align 'msg' [-Wpadded]
    const char *msg;
                ^
main.c:13:1: warning: cast from 'const void *' to 'unsigned char *' drops const qualifier [-Wcast-qual]
GREATEST_MAIN_DEFS();
^
./greatest.h:893:43: note: expanded from macro 'GREATEST_MAIN_DEFS'
    unsigned char *buf = (unsigned char *)t, diff_mark = ' ';           \
                                          ^
5 warnings generated.

I know that these can be suppressed with a few pragmas, but it'd be nice if the library didn't emit them in the first place.

Information of consequence: I'm on Arch and am using Clang 4.0.0, and my version of greatest.h is the latest git (as of the time of this issue).

Another RUN_TEST macro could be added to allow parametric testing without VA_ARGS:

RUN_TEST1(test_name, &struct_with_args_passed_in_as_void_pointer);

Not quite as nice as RUN_TESTp, but still portable.

As a convenience, the API could add an assertion ASSERT_EQ_ENUM(expected, got, enum_name_fun) that compares expected with got, and uses the enum_name_fun (typedef char const *enum_name_fun(int enum_value)) to print e.g. "Expected MODE_SLEEP, got MODE_SHUTDOWN" rather than "Expected 5, got 3".

Next step: Check the standards about whether a function can take a generic int type for an enum arg without warnings. (IIRC it's different in C++.) The ASSERT_EQ_ENUM macro could cast them, though.

There seems to be a typo on line 351 where void NAME(void) is duplicated:
#define GREATEST_SUITE(NAME) void NAME(void); void NAME(void)
No big deal, but caused me some warnings in Eclipse. No effect on the build though.

I feel greatest needs a method for avoiding duplicate test names. I personally have 2 scenarios where this would be helpful.

1. Visually getting a overview of results is difficult when many tests are named the same and only a few of them are failing.

2. I use TeamCity for continuous integration. I was looking to expanded the CI builds to run some basic regression testing. All of my tests use greatest. greatest support is not integrated into TeamCity but they have a method for reporting test results on the fly. I created a powershell script which interprets greatest print statements and creates the necessary TeamCity messages. Usage looks like mytests.exe -v | greatest2teamcity.ps1. One of the notes in the TeamCity documentation is that it's highly recommend to have unique suite+test names. Currently I solved this problem by keeping a counter of duplicate test names and appending that count amount to the end of the test name when duplicates are detected.

In my use of the greatest test framework I regularly use the following programming paradigms.

RUN_TEST1(MyFirstTest, AN_ENUM_TYPE_A);
RUN_TEST1(MyFirstTest, AN_ENUM_TYPE_B);
RUN_TEST1(MyFirstTest, AN_ENUM_TYPE_C);
RUN_TEST1(MyFirstTest, AN_ENUM_TYPE_D);
RUN_TEST1(MyFirstTest, AN_ENUM_TYPE_E);
RUN_TEST1(MyFirstTest, AN_ENUM_TYPE_F);

--OR--

TEST ASpecialTest(U8BIT u8Resolution)
{
    /*
    Do great things with u8Resolution
    */

    PASS();
}

void ASpecialTest_iterator()
{
    U8BIT u8Resolution;

    for (u8Resolution = 8; u8Resolution < 98; u8Resolution++)
    {
        RUN_TEST1(ASpecialTest, u8Resolution);
    }
}

SUITE(my_suite)
{
    ASpecialTest_iterator();
}

Both of these generate a long list of outputs that look something like

PASS MyFirstTest:  (1 ticks, 0.000 sec)
PASS MyFirstTest:  (1 ticks, 0.000 sec)
FAIL MyFirstTest:  (0 ticks, 0.000 sec)
PASS MyFirstTest:  (1 ticks, 0.000 sec)
PASS MyFirstTest:  (0 ticks, 0.000 sec)
PASS MyFirstTest:  (1 ticks, 0.000 sec)
PASS MyFirstTest:  (0 ticks, 0.000 sec)

One possible solution would be for greatest to have some built in functionality for looping through a range of variables and possible appending the variables value to the end of the test name. This still may not solve this issue for me as some of my "iterator functions" have up to 4 nested for loops.

Another possible solution would be to create additional marcos to the already existing GREATEST_RUN_TEST, GREATEST_IGNORE_TEST, and GREATEST_RUN_TEST1. Three new macros would take in a string argument for an extended name.

For example I have the following in a working example

/* Run a test in the current suite with one void * argument,
 * which can be a pointer to a struct with multiple arguments. 
 * Defined with an extended test name */
#define GREATEST_RUN_TEST1_EXT_NAME(TEST, ENV, EXT_NAME)                \
    do {                                                                \
        if (greatest_test_pre(#TEST) == 1) {                            \
            enum greatest_test_res res = GREATEST_SAVE_CONTEXT();       \
            if (res == GREATEST_TEST_RES_PASS) {                        \
                res = TEST(ENV);                                        \
            }                                                           \
            greatest_test_post(#TEST, EXT_NAME, res);                   \
        }                                                               \
    } while (0)

/* Run a test in the current suite with one void * argument,
 * which can be a pointer to a struct with multiple arguments. */
#define GREATEST_RUN_TEST1(TEST, ENV)                                   \
    GREATEST_RUN_TEST1_EXT_NAME(TEST, ENV, "")

It's a good tool!
Is there any plan to support mock and BDD in future?

I have a project where I would like to supply a command-line options to control the unit test. I could of course just omit GREATEST_MAIN_BEGIN() and parse the greatest options myself, but I wonder if it would be worth adding some way to handle this?

One option would be to stop the greatest parsing loop on encountering -- (pretty standard), and then my code can pick up any arguments that come after.

I am not sure if this can be classified as an issue, but it can make the tests fail silently so it's not ideal either. I had this issue from a user contribution on one of my repos

If by inadvertance a user-defined function has the same name (but not necessarily the same signature) than a TEST function, it turns out the compilation doesn't fail, and during testing the test function calls itself recursively instead of calling the function once.

int update_float32(float f)
{
    // some dummy code
   return (int)f;
}

TEST update_float32()
{
  ASSERT_EQ(update_float32(0.2),0);
  PASS();
}

I do agree this is an unlikely scenario, but it did happen at least and took me a bit of time to figure out what was going on.

I am getting a warning from Clang 3.8 when compiling with greatest (jibsen/brieflz#3). I tracked it back to the comparison at line 651.

Is the comparison to stdout necessary, or could that line just be fflush(GREATEST_STDOUT);?

I'm evaluating Greatest for usage on 8-bit atmega MCUs. So far it is excellent, I'm running the same test suits on PC (for comfortable debugging) and on avr (to test on final platform). It is so much better than minunit, which I've used before.

However, I edited greatest.h to get rid of all time-related stuff. Sometimes on embedded devices there is no clock functionality, so it cannot track time. In these cases dependency on time.h is just waste of space and calculations on garbage times are waste of CPU cycles.

I think that the best solution would be to have define such as this (maybe with better name) :

#define USE_CLOCK 1

Unsetting it would strip out all time.h dependencies and time calculations.
What do you think about that?

The README.md documentation talks about ASSERT_FALSE(cond) and ASSERT_FAILm(msg, cond). However, the actual macro is ASSERT_FALSEm(msg,cond).

Took me about 1 minute to fix after reading the code when it didn't build, but seems like a trivial fix :)

I'm just starting to use your lib for some c++ pet project of mine. I'm quite rusty when it comes to compiling C code - so I'm not sure if the following occurs due to my exotic compiler/flags combination.
I've pasted the basic usage example in my main. I'm using a Apple LLVM version 5.1 (clang-503.0.40).

I'm compiling with -std=c++11 -stdlib=libc++ and

linking with -std=c++11 -stdlib=libc++ -framework OpenGL -framework Cocoa -framework IOKit -lglew

The error I get is

src/main.cpp:253:1: error: redefinition of 'greatest_type_info_string'
GREATEST_MAIN_DEFS();
^
src/../include/greatest/greatest.h:561:20: note: expanded from macro 'GREATEST_MAIN_DEFS'
greatest_type_info greatest_type_info_string = {                        \
                   ^
src/../include/greatest/greatest.h:141:20: note: previous definition is here
greatest_type_info greatest_type_info_string;

If I remove the first definition - it runs fine.

Thanks for the lib, hth
Axel

After the change in 9d22d53, it is now possible for the setup & teardown callbacks to not get cleared when starting a new suite. To reproduce this, run a specific test (using -t) in a suite which is run after another suite sets its setup/teardown callbacks. The previous suite's callback can still trigger before/after the test.

This was introduced by: 9d22d53#diff-cb2c0eaf1e241d4d474321b3aa28dd7dR620

http://ccodearchive.net/

Any interest in making a version for C++?

Functions that used to be macros and have names in ALL_CAPS should be renamed in lowercase.

(This will be an API change.)

The README should mention using GREATEST_INIT() and greatest_get_report(&report) rather than GREATEST_MAIN_BEGIN() and GREATEST_MAIN_END() when using greatest built into another system, rather than running directly from the command line.

A few of the ASSERT macros will evaluate their argument multiple times. (GREATEST_ASSERT_EQ_FMTm, GREATEST_ASSERT_ENUM_EQm) Where possible, they should only evaluate them one, and save the value. When this is difficult to reconcile with the type system, the documentation should call attention to it.

I believe the %d should be %u in this line:

https://github.com/silentbicycle/greatest/blob/master/greatest.h#L669

I am getting a warning when compiling with MSVC.

1.4.0 will have greatest_set_test_suffix() (#63). It would probably be quite useful to include this test suffix when matching using test filters and exclude filters.

The current implementation uses "%s%s" with two NULL checks on the suffix that print "" or "_" and "" or the suffix, respectively, to avoid buffering/string concatenation.

The greatest_info struct could include a statically allocated buffer with a configurable size for the current test name and optional suffix, but it would expand the memory requirement further. The current approach can use strings in ROM, or otherwise allows the caller to decide how much buffer space is appropriate, without double buffering.

If a buffer is used, there also needs to be a test name + suffix size limit added to the API. (Also, keep in mind that snprintf probably isn't an option for -std=c89.)

The greatest.h file has a license, but it would be good to also have a LICENSE to follow typical project layouts.

The examples in the documentation build a test runner to execute from the command line, but it could easily be a run_tests() function instead. This should be documented, and could also be made a bit clearer by making the initialization vs. command line argument parsing more distinct.

Using them in combination can lead to an infinite loop calling the PRNG, rather than exiting.

• OS: Windows 10, cygwin64, latest as of Aug 1st, 2017
• gcc version 5.4.0 targetting x86_64 target

Repro steps:

• Add -DGREATEST_USE_LONGJMP to compiler command line
• Run a test suite with complex code (deep function calls, dynamically allocated buffers), of which several tests are run with RUN_TESTp.
• Hard to repro with trivial tests, as there is no state to clobber.

When GREATEST_USE_LONGJMP is set, setjmp() (through GREATEST_SAVE_CONTEXT()) is only called in the RUN_TEST case and never in the RUN_TEST1/RUN_TESTp cases.

There isn't much point in having a single suite ("main" or whatever) for a few tests. Update the internals so that calling RUN_TEST outside of any suite defaults to one.

This will also need some updates in SET_SETUP, SET_TEARDOWN, and GREATEST_REPORT.

When building a test suite using greatest.h, it's possible to get a shadowing warning about exp's use. This is the name of a function available in math.h as well.

I've been unable to produce this on GCC versions other than 4.6.3.

Greatest Version: 1.4.0

GCC output:

$ gcc -Wshadow greatest_exp_repro.c -o ger
greatest_exp_repro.c: In function 'greatest_do_assert_equal_t':
greatest_exp_repro.c:14:1: warning: declaration of 'exp' shadows a global declaration [-Wshadow]
greatest_exp_repro.c: In function 'greatest_string_equal_cb':
greatest_exp_repro.c:14:1: warning: declaration of 'exp' shadows a global declaration [-Wshadow]
greatest_exp_repro.c: In function 'greatest_memory_equal_cb':
greatest_exp_repro.c:14:1: warning: declaration of 'exp' shadows a global declaration [-Wshadow]

Example File:

/* greatest_exp_repro.c */

#include "math.h"
#include "greatest.h"

TEST test_exp_repro()
{
	PASS();
}

SUITE(test_exp_suite)
{
	RUN_TEST(test_exp_repro);
}

GREATEST_MAIN_DEFS();

int main(int argc, char *argv[])
{
	GREATEST_MAIN_BEGIN();
	RUN_SUITE(test_exp_suite);
	GREATEST_MAIN_END();
}

System Information:

$ uname -a
Linux c02e4fba5a58 4.13.0-39-generic #44~16.04.1-Ubuntu SMP Thu Apr 5 16:43:10 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

GCC version information:

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.6.3-1ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)

I am having trouble using greenest script, it is not recognized by the command line
I tried placing the file in the current folder the console is pointing to (along with .exe file) but it doesn't change a thing
I am using gradle as build system
Any hints on using that script ?

It would probably be worth adding an assertion to do a raw memory compare (with memcmp), with a hexdump that emphasizes the different portions.

This could be built on the GREATEST_ASSERT_EQUAL_T functionality, much like GREATEST_ASSERT_STR_EQ.

The FIRST_FAIL flag (which corresponds to the -f command line test runner option) means it should not start new suites if there have already been any failures. Currently, it only prevents tests from running if there have been failures in the current suite.

Hi there,

I use the framework a lot at work to test my embedded code and I like the simplicity.

At my current project I have a lot of unit tests (currently ~1000) within different libraries. Unfortunately, due to the fact how the different libraries are integrated into the whole project, this leads to some problems at the execution order of the unit tests.

For Example:

• LibA calls LibB and LibC to initialze them
• I want to unit test all three different libs, but the dependencies result in problems for example when each lib will be initialized
• When I init LibB or LibC within the unit test, this results in a problem on execution in LibA, because the lib expects LibB and LibC as not initialized at startup

So, for the moment I am implementing functions only for unit tests to reset internal information to avoid such problem, which works for the moment but only because of very limited dependencies. I expect more such problems for the future when the count and complexity of the libs increase.

Is there or are there any plans for feature within the GREATEST framework to support different test runner (exe) to be executed? This would be the ideal solution for me, to differ the unit tests of the different libraries into separated test runner!?

Another benefit of such a feature is that it will be possible to test the same lib within different test runner to simulate different execution paths, which is necessary when internal static variabled must be reset before the next test...

Please let me know whether or how I can help (testing the framework, ...)...

Greetings,
Ronald

There should be RUN_SUITE1/RUN_SUITEp macros for suites that take argument(s), such as a random number seed, verbosity setting, or iteration limit.

Because the existing syntax for defining/calling suites is SUITE(suitename), this will be a breaking API change.

/* previous style */
SUITE(suite);

/* new style */
SUITE suite(void);
SUITE suite1(void *some_arg);
SUITE suite_multi(size_t apple, void *banana, int carrot);

SUITE suite(void) {}

RUN_SUITE(suite);   /* no change */

RUN_SUITE1(suite1, arg);  /* suite with 1 argument */

/* suite with multiple arguments (C99 and later standards only) */
RUN_SUITEp(suite_multi, 1, NULL, 2);

Is greatest compatible with CI services like appveyor ?

I belive it would be compliant as long as test errors are outputed on stderr instead of stdout. Looking at the code, this is not the case for now isn't it ?