Tree view console report for Maven Surefire JUnit5 plugin. :^)

License: Apache License 2.0

Java 100.00%

maven-surefire-junit5-tree-reporter's Introduction

Maven Surefire JUnit5 TreeView Extension

If you are a Maven Surefire user and ever wanted a fancy tree output for your tests instead of a bunch of logs, you absolutely should try this.

This is a dependency for maven-surefire-plugin, it adds a tree view for the unit tests executed using JUnit5.

Installation

The Maven Repository can be found here.

Configure your POM like the following

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.1.0</version>
    <dependencies>
        <dependency>
            <groupId>me.fabriciorby</groupId>
            <artifactId>maven-surefire-junit5-tree-reporter</artifactId>
            <version>1.2.1</version>
        </dependency>
    </dependencies>
    <configuration>
        <reportFormat>plain</reportFormat>
        <consoleOutputReporter>
            <disable>true</disable>
        </consoleOutputReporter>
        <statelessTestsetInfoReporter
                implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
        </statelessTestsetInfoReporter>
    </configuration>
</plugin>

Output Theme

The output can be printed using two Themes: UNICODE and ASCII (by default).

UNICODE

<statelessTestsetInfoReporter
        implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
    <theme>UNICODE</theme>
</statelessTestsetInfoReporter>

ASCII

<statelessTestsetInfoReporter
        implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
    <theme>ASCII</theme>
</statelessTestsetInfoReporter>

Failure details

By default, <consoleOutputReporter><disable>true</disable></consoleOutputReporter> disables all console output. To debug test failures, it may be useful to see the console output and stack traces when a test fails. To do so, you can configure this extension like this:

<statelessTestsetInfoReporter
        implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
    <printStacktraceOnError>true</printStacktraceOnError>
    <printStacktraceOnFailure>true</printStacktraceOnFailure>
    <printStdoutOnError>true</printStdoutOnError>
    <printStdoutOnFailure>true</printStdoutOnFailure>
    <printStdoutOnSuccess>false</printStdoutOnSuccess>
    <printStderrOnError>true</printStderrOnError>
    <printStderrOnFailure>true</printStderrOnFailure>
    <printStderrOnSuccess>false</printStderrOnSuccess>
</statelessTestsetInfoReporter>

Contribute

You are welcome to contribute to the project, for this just open an issue or issue + PR to develop branch.

If you want to create your own output based on the Theme Enum, feel free to open a PR.

Debugging

If you ever want to debug the code, please use the following command

mvnDebug test

Then attach a remote JVM debugger on port 8000

maven-surefire-junit5-tree-reporter's People

Contributors

Stargazers

Watchers

maven-surefire-junit5-tree-reporter's Issues

report doesn't show @Before and @After times

All @before an @after annotations are not displayed as part of the report

in the original report you can see the time elapsed for the entire class, but in the tree report you just see the elapsed time per method.

e,g:
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s - in org.example.MyTest

if for example, you have a @before method in your class which takes few seconds, you will not be able to capture that

Nested tests not logged

I have Test with 2 @nested class in same level.
After run test only one nested test class logged in result.

OOME when there are many tests failures

We have a project with 700+ tests and on our CI we noticed multiple times OOME errors (sadly we weren't able to get a dump) when our tests are failing.

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "surefire-forkedjvm-command-thread"

failsafe / surefire are configured with

<!-- Custom reports -->
<reportFormat>plain</reportFormat>
<consoleOutputReporter>
    <disable>true</disable>
</consoleOutputReporter>
<statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter"/>

and JUnit with

junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.mode.default=same_thread
junit.jupiter.execution.parallel.mode.classes.default=concurrent

To display the tests results as tree I suppose the reporting is keeping in memory many informations due to the execution of tests in parallel (I see several statics fileds in https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter/blob/master/src/main/java/org/apache/maven/plugin/surefire/report/TestReportHandler.java#L12-L15)

If there are several failures and the memory is limited like in a CI system a surefire JVM can die with an OOME because of the TestReportHandler size. We could probably increase the tests JVMs size but it's a bit annoying to do it just for the tests reporter.

Unfortunately does not work with maven-surefire-plugin:3.0.0-M6

Upgrade to maven-surefire-plugin:3.0.0-M8

Initialization error corrupts output

Hey,

in case of following error:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.004 s <<< FAILURE! - in com.faceit.hubs.model.entities.competition.SomeTest
[ERROR] initializationError  Time elapsed: 0.004 s  <<< ERROR!
org.junit.runners.model.InvalidTestClassError:
Invalid test class 'com.faceit.hubs.model.entities.competition.SomeTest':
  1. Method shouldBeOk should have no parameters

the output looks as follows:

[INFO] ├─ initializationError - 0.003s
[INFO] │  └─ ✘ null - 0.001s

Shows error name instead of test name.

[Not An Issue] Just want to see I like this extension.

Thanks for sharing.
It's awesome!
I've been looking for such a solution for a long time.

printStdout and printStderr must ignore empty output

If you enable success logs by:

<printStdoutOnSuccess>true</printStdoutOnSuccess>
<printStderrOnSuccess>true</printStderrOnSuccess>

It will show empty logs, but it must be ignored (show only test name):

[reportFormat] Question about formats

What different report formats exist and what does it influence?

buffer output by plugin execution

I can confirm the plugin is now (v1.0.0) buffering the output by class.
Would it be possible to buffer the output by plugin execution?
Or is this something Maven should be handling?

For example, currently a log might have a line

[INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ wwgcommon ---

followed by

[INFO] |  +-- [OK] testUnpaidImport - 0.179s
[INFO] |  '-- [OK] testReplyImport - 0.002s

Using mockito-bom 4.11.0

IllegalStateException thrown when running in parallel

Running in parallel appears to trigger an IllegalStateException when cloning the reporter object.

Maven 3.8.6
Java 11
Surefire 3.0.0-M7
Reporter plugin 1.0.0
Using the following in-POM configuration:

<runOrder>random</runOrder>
<reportFormat>plain</reportFormat>
<consoleOutputReporter>
    <disable>true</disable>
</consoleOutputReporter>
<statelessTestsetInfoReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode">
</statelessTestsetInfoReporter>

Running Maven with the following flags:

./mvnw --debug -T4C clean package -DforkCount=4 -Dparallel=all 2>&1 | tee output.log

Debug output seems to indicate the following is the root of the error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project my-project: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test failed: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test (default-test) on project my-project: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test failed: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7:test failed: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)
Caused by: java.lang.IllegalStateException: org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporterUnicode
    at org.apache.maven.plugin.surefire.extensions.SurefireStatelessTestsetInfoReporter.clone (SurefireStatelessTestsetInfoReporter.java:78)
    at org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoReporter.clone (JUnit5StatelessTestsetInfoReporter.java:120)
    at org.apache.maven.plugin.surefire.CommonReflector.createStartupReportConfiguration (CommonReflector.java:96)
    at org.apache.maven.plugin.surefire.CommonReflector.createReportingReporterFactory (CommonReflector.java:73)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.getSuitesIterator (ForkStarter.java:746)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkOnceMultiple (ForkStarter.java:351)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run (ForkStarter.java:326)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run (ForkStarter.java:269)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider (AbstractSurefireMojo.java:1334)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:1167)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:931)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:370)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:210)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:195)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :my-project

Is there a workaround for this? If any other info is needed, please let me know! :-)

IndexOutOfBoundsException being dumped

Just had a look in my Surefire reports output directory and spotted the following:

target/surefire-reports/2022-12-30T14-01-27_616-jvmRun1.dumpstream

# Created at 2022-12-30T14:01:38.848
ForkStarter IOException: Index: 1, Size: 1.
org.apache.maven.plugin.surefire.booterclient.output.MultipleFailureException: Index: 1, Size: 1
	at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.<init>(ThreadedStreamConsumer.java:64)
	at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer.<init>(ThreadedStreamConsumer.java:122)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:600)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:311)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:268)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1334)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1167)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:931)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:370)
	at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:210)
	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:195)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1589)
	Suppressed: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
		at java.base/java.util.Collections$SingletonList.get(Collections.java:4959)
		at org.apache.maven.plugin.surefire.report.TreePrinter$TestPrinter.printClass(TreePrinter.java:162)
		at org.apache.maven.plugin.surefire.report.TreePrinter$TestPrinter.printTest(TreePrinter.java:96)
		at org.apache.maven.plugin.surefire.report.TreePrinter$TestPrinter.access$500(TreePrinter.java:85)
		at org.apache.maven.plugin.surefire.report.TreePrinter.lambda$printTests$1(TreePrinter.java:82)
		at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
		at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
		at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
		at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
		at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
		at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
		at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
		at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
		at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
		at org.apache.maven.plugin.surefire.report.TreePrinter.printTests(TreePrinter.java:82)
		at org.apache.maven.plugin.surefire.report.TestReportHandler.printTests(TestReportHandler.java:111)
		at org.apache.maven.plugin.surefire.report.TestReportHandler.print(TestReportHandler.java:44)
		at org.apache.maven.plugin.surefire.report.ConsoleTreeReporterBase.testSetCompleted(ConsoleTreeReporterBase.java:26)
		at org.apache.maven.plugin.surefire.report.ConsoleTreeReporterAscii.testSetCompleted(ConsoleTreeReporterAscii.java:32)
		at org.apache.maven.plugin.surefire.report.ConsoleTreeReporterBase.testSetCompleted(ConsoleTreeReporterBase.java:8)
		at org.apache.maven.plugin.surefire.report.TestSetRunListener.testSetCompleted(TestSetRunListener.java:213)
		at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$TestSetCompletedListener.handle(ForkClient.java:147)
		at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$TestSetCompletedListener.handle(ForkClient.java:135)
		at org.apache.maven.plugin.surefire.booterclient.output.ForkedProcessEventNotifier.notifyEvent(ForkedProcessEventNotifier.java:233)
		at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.handleEvent(ForkClient.java:353)
		at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.handleEvent(ForkClient.java:59)
		at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:96)
		... 1 more

Tests appear to still be running, though. I don't know how serious this is or whether it is anything to worry about or not.

Other info:

Invocation: $ ./mvnw clean verify -T8

$ uname -a
Linux z170x-fedora 6.0.12-300.fc37.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Dec 8 16:58:47 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ java -version
java -version
openjdk version "19.0.1" 2022-10-18
OpenJDK Runtime Environment Corretto-19.0.1.10.1 (build 19.0.1+10-FR)
OpenJDK 64-Bit Server VM Corretto-19.0.1.10.1 (build 19.0.1+10-FR, mixed mode, sharing)

$ ./mvnw --version 
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /home/ashley/.m2/wrapper/dists/apache-maven-3.8.6-bin/1ks0nkde5v1pk9vtc31i9d0lcd/apache-maven-3.8.6
Java version: 19.0.1, vendor: Amazon.com Inc., runtime: /home/ashley/.sdkman/candidates/java/19-amzn
Default locale: en_GB, platform encoding: UTF-8
OS name: "linux", version: "6.0.12-300.fc37.x86_64", arch: "amd64", family: "unix"

<properties>
  <argLine>
    -Dorg.slf4j.simpleLogger.log=INFO
    -Xshare:off
  </argLine>

  <maven-surefire-plugin.version>3.0.0-M7</maven-surefire-plugin.version>
  <maven-surefire-junit5-tree-reporter.version>1.1.0</maven-surefire-junit5-tree-reporter.version>
</properties>

<plugin>
  <!-- Unit testing config -->
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${maven-surefire-plugin.version}</version>

  <configuration>
    <excludes>
      <exclude>*IntegrationTest</exclude>
    </excludes>

    <failIfNoTests>true</failIfNoTests>
    <runOrder>random</runOrder>
    <tempDir>./tmp/unit</tempDir>
    <useModulePath>true</useModulePath>

    <!--
      This block is needed to show @DisplayName and @ParameterizedTest
      in reports with the provided names.
    -->
    <statelessTestsetReporter
      implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
      <disable>false</disable>
      <version>3.0</version>
      <usePhrasedFileName>false</usePhrasedFileName>
      <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
      <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
      <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
    </statelessTestsetReporter>
    <consoleOutputReporter>
      <disable>${hide-test-logs-in-console}</disable>
    </consoleOutputReporter>
    <statelessTestsetInfoReporter
      implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5StatelessTestsetInfoTreeReporter">
      <disable>false</disable>
      <usePhrasedFileName>false</usePhrasedFileName>
      <usePhrasedClassNameInRunning>true</usePhrasedClassNameInRunning>
      <usePhrasedClassNameInTestCaseSummary>true</usePhrasedClassNameInTestCaseSummary>
    </statelessTestsetInfoReporter>
  </configuration>

  <dependencies>
    <dependency>
      <groupId>me.fabriciorby</groupId>
      <artifactId>maven-surefire-junit5-tree-reporter</artifactId>
      <version>${maven-surefire-junit5-tree-reporter.version}</version>
    </dependency>
  </dependencies>
</plugin>

src/test/resources/junit-platform.properties

junit.jupiter.execution.parallel.enabled=true

Parameterized tests looks strange

Hey, great project!

I have a parameterized test as follows:

@RunWith(JUnitParamsRunner.class)
public class SomeTest {
    @Test
    @Parameters(method = "values")
    public void shouldBeOk(List<Integer> leaderboards, Integer count) {
        assertThat(leaderboards.size(), is(count));
    }
    private Object[] values() {
        return new Object[]{
                new Object[]{List.of(), 0},
                new Object[]{List.of(1), 1},
        };
    }
}

which looks like:

[INFO] ├─ shouldBeOk([], 0) [0] - 0.001s
[INFO] │  ├─ ✔ null - 0s
[INFO] │  └─ ✔ null - 0.001s

To compare:

Get elapsed time reported correctly for @Nested tests

BTW, the time reported is still not correct because of a reverse order of events the surefire plugin is sending to the ReportConsole

here's an example:

public class NestedWaitingTest {
@Test
public void outside() throws InterruptedException {
    Thread.sleep(1000);
}
@Nested
class Inside1Test {
    @Test
    public void inside1() throws InterruptedException {
        System.out.println("Waiting for 3 seconds");
        Thread.sleep(3000);
    }
    @Test
    public void inside1fast() {
    }

    @Nested
    class Inside2Test {
        @Test
        public void inside2fast() {
        }
    }
}
}

you get this report (without plugin)
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 s - in NestedWaitingTest$Inside1Test$Inside2Test
[INFO] NestedWaitingTest.outside Time elapsed: 1.019 s
[INFO] NestedWaitingTest$Inside1Test.inside1fast Time elapsed: 0.001 s
[INFO] NestedWaitingTest$Inside1Test.inside1 Time elapsed: 3.007 s
[INFO] NestedWaitingTest$Inside1Test$Inside2Test.inside2fast Time elapsed: 0.001 s
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.021 s - in NestedWaitingTest$Inside1Test
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.053 s - in NestedWaitingTest

this translate into 3 events coming into ConsoleReporter
1st one you use today to print the entire data (includes 4 entires in the TreeSetStats) - but uses the elapsed time of the inner*2 test class
2nd and 3rd event are coming without any entries in the TreeSetStats but with correct elapsed time information

so if you want to be consistent with the elapsed time, you need to implement a different approach

Originally posted by @avishayh in #21 (comment)

Using JUnit Jupiter BOM 5.9.2

Nested tests are not being printing correctly

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] |
[INFO] +-- Nested Sample
[INFO] |
[INFO] +-- Inner Test
[INFO] |
[INFO] +-- Inner Inner Test
[INFO] |
[INFO] +-- Inner Inner Inner Test
[INFO] | +-- [OK] Should pass - 0.036s
[INFO] | +-- [OK] Should pass2 - 0.002s
[INFO] | +-- [OK] Inner test should pass - 0.002s
[INFO] | +-- [OK] Inner test should pass2 - 0.001s
[INFO] | +-- [OK] Inner Inner Test should pass - 0.002s
[INFO] | +-- [OK] Inner Inner Inner Test should pass - 0.002s

Disabled tests not logged as warning

Although the tree reporter indicates which tests are skipped by colour and the text "is @disabled" the log line is still at INFO level.

This is fine though, as the surefire summary line, e.g.
[WARNING] Tests run: 22, Failures: 0, Errors: 0, Skipped: 4
is enough to indicate a problem (where log output is set to filter out INFO logs)

Ticket made as a reference for future.

Upgrade to maven-surefire-plugin:3.0.0-M7

Newer Surefire – elapsed time output format wrong

Since Surefire 3.1.1 the formatting of the elapsed time comes already with the s appended.

However, the TreePrinter appends another s, leading to output like this:

…
[INFO] │  ├─ ✔ validDataUrlIsParsed(String)[1] image/png - 0.006 ss
[INFO] │  ├─ ✔ validDataUrlIsParsed(String)[2] image/jpeg - 0.001 ss
[INFO] │  ├─ ✔ validDataUrlIsParsed(String)[3] image/jpg - 0.001 ss
…

Minimum version of Java 8 or 11

In pom.xml we see references to Java 8: https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter/blob/master/pom.xml#L18

        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>

But in github workflow, we see a reference to Java 11: https://github.com/fabriciorby/maven-surefire-junit5-tree-reporter/blob/master/.github/workflows/maven-publish.yml#L16

    - name: Set up JDK 11
      uses: actions/setup-java@v3
      with:
        distribution: 'temurin'
        java-version: '11'

I suggest aligning these versions.

Add support to show the "TestAbortedException" message in the output

I am working with the release 1.2.1.

It works fine for the @DisplayName outputs, but not for a TestAbortedException message, it when an assumption fails. I am not sure if is a missing or incomplete configuration, for more details see:

correct configuration of maven-surefire-plugin 3 to show @DisplayName and TestAbortedException of JUnit 5

I read the maven-surefire-junit5-tree-reporter page, but it seems is not possible reach this goal because this support does not exist yet

Thank You

(Some) nested tests are not printed at all

Reproducer: https://github.com/rkraneis/tree-reporter

Context: https://github.com/rkraneis/tree-reporter/blob/2a656ebb9e4c5c39371b5fd4e849c326641437fb/README.md

Actual:

├─ io.github.rkraneis.treereporter.OuterTest - 0.036s
│  └─ ✔ test - 0.011s
├─ ── InnerSub2 - 0.001s
│     ├─ ✔ testFromBase - 0s
│     └─ ✔ testInnerSub2 - 0.001s

Expected:

├─ io.github.rkraneis.treereporter.OuterTest - 0.036s
│  └─ ✔ test - 0.011s
├─ ── Inner1 - 0.001s
│     └─ ✔ testInner1 - 0.001s
├─ ── Inner2 - 0.001s
│     └─ ✔ testInner2 - 0.001s
├─ ── InnerSub1 - 0.001s
│     ├─ ✔ testFromBase - 0s
│     └─ ✔ testInnerSub1 - 0.001s
├─ ── InnerSub2 - 0.001s
│     ├─ ✔ testFromBase - 0s
│     └─ ✔ testInnerSub2 - 0.001s

The tests themselves are executed (Tests run: 7, Failures: 0, Errors: 0, Skipped: 0).

N.B.: I started my own tree view printer a while ago (basically just as a copy of what the junit jupiter console launcher does), but didn't like the messy setup (test execution listener). Your POM based version would be much better, setup-wise.

Option to buffer report

When building multithreaded the logs come out piecemeal and it doesn't read easily. Is it possible to buffer the report and send it all in one go?

Nested test tree not formatted correctly

Reproducer: https://github.com/tmarback/tree-reporter-issue

When using nested tests, the "bars" that draw the test tree are incorrect, with the horizontal lines for the test classes originating a layer above where they should:

(note how for the 1st-level nested classes the horizontal line crosses 2 different vertical lines, while on the 2nd-level nested classes the horizonal line goes past the vertical line)

PS <snip>\tree-reporter> mvn test
[INFO] Scanning for projects...
[INFO] 
[INFO] -------------------------< test:tree-reporter >-------------------------
[INFO] Building tree-reporter 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
<snip>
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] +--Outer - 0.056s
[INFO] |  +-- [OK] testOuter1 - 0.016s
[INFO] |  +-- [OK] testOuter2 - 0s
[INFO] |  '-- [OK] testOuter3 - 0s
[INFO] +--.--Outer Inner 2 - 0.005s
[INFO] |  |  +-- [OK] testInner1 - 0.001s
[INFO] |  |  +-- [OK] testInner2 - 0.001s
[INFO] |  |  '-- [OK] testInner3 - 0.001s
[INFO] +--.--Outer Inner 3 - 0.005s
[INFO] |  |  +-- [OK] testInner1 - 0s
[INFO] |  |  +-- [OK] testInner2 - 0.001s
[INFO] |  |  '-- [OK] testInner3 - 0s
[INFO] |  '--.--Outer Inner 3 Inner-er 2 - 0.013s
[INFO] |     |  +-- [OK] testInnerer1 - 0.001s
[INFO] |     |  +-- [OK] testInnerer2 - 0s
[INFO] |     |  '-- [OK] testInnerer3 - 0s
[INFO] |  '--.--Outer Inner 3 Inner-er 3 - 0.013s
[INFO] |     |  +-- [OK] testInnerer1 - 0s
[INFO] |     |  +-- [OK] testInnerer2 - 0s
[INFO] |     |  '-- [OK] testInnerer3 - 0.001s
[INFO] |  '--.--Outer Inner 3 Inner-er 1 - 0.013s
[INFO] |     |  +-- [OK] testInnerer1 - 0s
[INFO] |     |  +-- [OK] testInnerer2 - 0.001s
[INFO] |     |  '-- [OK] testInnerer3 - 0s
[INFO] +-----Outer Inner 1 - 0.005s
[INFO] |     +-- [OK] testInner1 - 0s
[INFO] |     +-- [OK] testInner2 - 0.001s
[INFO] |     '-- [OK] testInner3 - 0s
[INFO] 
[INFO] Results:
[INFO]
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.787 s
[INFO] Finished at: 2023-05-18T03:39:04-07:00
[INFO] ------------------------------------------------------------------------

Also the @DisplayName names seem to be getting appended at each level, which from the examples doesn't seem to be the expected.

Warning with JUnit parallel Tests

Just checked this thing out and love the output, but when I use a parallel test setup I get a warning.

Steps to reproduce:
Configure your maven-surefire-plugin with

                  <properties>
                        <configurationParameters>
                            # PARALELIZATION FOR SUREFIRE ONLY, NOT FOR FAILSAFE
                            junit.jupiter.execution.parallel.enabled=true
                            junit.jupiter.execution.parallel.config.strategy=dynamic
                            junit.jupiter.execution.parallel.config.dynamic.factor=1
                            junit.jupiter.execution.parallel.mode.default = same_thread
                            junit.jupiter.execution.parallel.mode.classes.default = concurrent
                        </configurationParameters>
                    </properties>

Run your tests

Expected: Just the nice output

Result: The nice output and

WARNING] ForkStarter IOException: Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 2, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 3, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 1, Size: 1
Index: 3, Size: 1. See the dump file ***/target/surefire-reports/2022-10-24T22-19-57_613-jvmRun1.dumpstream

Dump file attached.
2022-10-24T22-16-54_224-jvmRun1.dumpstream.zip

fabriciorby / maven-surefire-junit5-tree-reporter Goto Github PK