Comments (5)
When the test is executed in a child process, it should sort of work already because the parent process should see strange exit code. (Maybe it leads to a sub-optimal message because currently the parent process assumes that calling abort()
went from the new TEST_ASSERT
. But the message could be reword to cover the assert
as well.)
When in the same process, it's completely different story. I can't see any other potential way then the SIGABRT
handler and longjmp
if we want to support that.
However right now I am somewhat reserved because I see it as a quite crazy thing a process can do and I have some doubts about it:
-
Is it safe to make
longjmp
from a signal handler? AFAIK, at least on some systems/architectures, kernel implants a different stack into the thread for handling the signal. Is it supported by POSIX standard so we can use it for other POSIX systems as well? Or, is it guaranteed to work on some important system so that it would be worth to add such support conditionally just on that system? -
What if some people install their own signal handlers as part of their unit tests? I think we should allow them.
-
Last but not least, I am afraid how big impact it would have for ease of debugging of such failing test. It should not harm the usability of Acutest with gdb or valgrind and I am not sure how well those tools can handle processes which do such crazy things at all or how big nuisance it would be for people any such tool in order to find the culprit what's going wrong in their test.
To conclude, feel free to go ahead and implement it, but be prepared that those questions will have to be answered in some reasonable way before anything like that gets merged.
from acutest.
Oops. Sorry. Ignore the previous post. I missed the point you want to make your test succeed when it does abort()
.
from acutest.
I think my doubt (2) could be solved by installing/uninstalling the handler just for the time of the macro.
But my doubts (1) and (3) still hold.
Also, if we want a new macro like TEST_THAT_THIS_ABORTS(my_func())
, I think we would have to support it everywhere, including Windows. I don't think we can have it as an optional feature like being able to start a subprocess or colorize the output. It would be part of the Acutest API and so the macro should have the same semantics everywhere. When people port their code to a new platform they should not need to rewrite the unit tests to avoid the macro.
To be honest, I'm somewhat skeptical in that regard but feel free to argue or show a code proving I'm wrong ;-)
from acutest.
Also see #18. Maybe some of the arguments used there apply here too.
from acutest.
Thinking about this a little bit more, I now believe it is a bad idea. Let me react to some statements in the original post.
It can be viewed as the poor cousin of
TEST_EXCEPTION
for C.
I disagree. In C++, exceptions are more or less normal error handling mechanism and, for example, part of various library APIs. I.e. it is important that for a given known and documented error condition (e.g. an invalid parameter passed to a function) some API signals the error to the caller in some expected way (here, some particular exception type) and that this can be tested.
On the other hand, abort()
is a last resort when a program does not know any better then to kill itself in order to prevent a potential bigger damage. It's not a normal error handling mechanism. Calling abort()
generally means the program itself says "Ohh. I am broken. Come and debug me."
From this perspective, C++ exceptions are rather a cousin of some special return values like -1
or NULL
which are used as some error indicators in C.
Moreover, such tests are useful for reaching 100% branch coverage, otherwise one branch of the
assert
is never hit.
I agree good test coverage is generally a nice thing to have. But it is not the goal of the testing per se. It's only some helper metrics. Value of that metrics has its limits. This question of abort()
is in my opinion one such limit.
I argue that in a correct program, if there are code paths leading to abort()
, they are actually unreachable (at least for any input in some supported domain) and this property of such a correct program prevents you to really reach the 100% code test coverage of it.
If you are able to write a test causing an assertion in the tested code, you should likely rather go and fix the code so it does not have a reason to raise the assertion in the first place (and hence change it to the program which cannot have 100% code coverage in principle as stated above).
To conclude, the very fact that it asserts means the program has some problem. Providing a mechanism saying "hey, the test passes" in such case imho makes no real sense.
from acutest.
Related Issues (20)
- reserved identifier violation HOT 2
- Completion of error handling HOT 2
- #define ⇒ enum?
- Windows outputs "Debug Error - Abort() was called" on every failed test HOT 2
- Q: constructor/destructor functions? HOT 4
- TEST_STRING_EQUAL method HOT 2
- Compile works but program does not finish and takes 100% CPU ?! HOT 3
- Memory leak in children HOT 2
- Call cleanup handler after assert fail HOT 4
- travisci C std 17 HOT 1
- How can I compile multiple units to one executable? HOT 1
- Any way to test for a non-zero exit(status) condition? HOT 3
- flush stdout and stderr before aborting
- Allow TEST_CHECK for non-int conditions
- Can acutest add support like doctest's DOCTEST_CONFIG_DISABLE?
- Provide TEST_SKIP(message) macro
- Warning `warning: variable length array folded to constant array as an extension` on Clang 10 HOT 2
- Is this repo abandoned? HOT 2
- Any way to figure out code coverage or lines not covered? HOT 1
- Can exception handling be disabled for use in testing embedded C++ systems? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from acutest.