rra / c-tap-harness Goto Github PK

The diag() in c-tap-harness currently prints to stdout, but the comments mention stderr:

/*
 * Report a diagnostic to stderr.  Always returns 1 to allow embedding in
 * compound statements.
 */

(and libtap.sh is similar)

In contrast, the diag() from perl's Test::More et al does actually print to stderr and I assume has for a long time. This difference can be pretty confusing, since runtests from c-tap-harness discards stderr unless we run with -o.

So if you're running tests through runtests -v, you get diagnostic info if the test was written in C using c-tap-harness, but you don't if it's written in perl's Test::More. If a test fails, you don't know why:

$ cat foo-t
#!/usr/bin/env perl

use strict;
use warnings;

use Test::More;

plan tests => 1;

is(1, 0, "example");

$ runtests -v foo
foo

1..1
not ok 1 - example
FAILED 1 (exit status 1)
[...]

The behavior of c-tap-harness makes more sense to me than perl's, but I'm not familiar with the historical context here. Searching for some info finds some discussion implying this can be a contentious issue (such as https://www.nntp.perl.org/group/perl.qa/2007/03/msg8289.html ), suggesting that there may be no good fix for this.

If we just don't ignore stderr for runtests -v (or have a new option to let stderr through), that would let the diagnostics be visible. They'd probably be interleaved weirdly, but at least the data is there.

Or if things are left alone, it could be helpful to mention this as a slight incompatibility with other TAP producers like perl's. The user can change any perl tests to show diagnostic info on stdout in the TAP stream; having a note would help explain why you're not seeing any diagnostics and what you need to do.

release/4.7-8-ge169247

make install fails complaining that some manpage is already present when linking the file (older version installed). A fix is to add -f to ln's flags in the Makefile:

--- Makefile~	2022-02-04 00:23:25.915145798 -0300
+++ Makefile	2022-02-04 00:24:12.997853946 -0300
@@ -468,7 +468,7 @@
 LDFLAGS = 
 LIBOBJS = 
 LIBS = 
-LN_S = ln -s
+LN_S = ln -sf
 LTLIBOBJS = 
 MAINT = #
 MAKEINFO = ${SHELL} /home/vonbrand/tmp/c-tap-harness/build-aux/missing makeinfo

Sorry, no real understanding of the autoconf mess to know how to really fix this.

The readme features the following build instructions:

You can build C TAP Harness and run its internal test suite with:

 ./configure
 make
 make check

This does not work because there is no configure file in the project. If I use the build steps of:

autoreconf -vif
./configure
make

Then the build fails with the following error message:

make: *** No rule to make target docs/api/bail.3', needed by all-am'. Stop.

I have forked this project and added a dockerfile which demonstrates this issue. You can find that here. Building the docker image will fail and the error will be displayed.

I created a single test within my TESTS list file and commented it out:

# example

This causes the program to crash. I fixed this in the code by modifying read_test_list(...) to return NULL when no testset structures have been populated.