tdomzal / junit-docker-rule Goto Github PK

Sometimes it is required to specify concrete host port which we want container port be specified to

In cases when one must wait for multiple messages in specified sequence it would be handy to use something like

...
.imageName(...)
.waitForMessageSequence("Listening on")
.nextMessage("Service started")
.waitDone()
...

In command line version it is not necessary for container Dockerfile to specify EXPOSE to make part mapping possible when run, for example:

docker run -p 1234:1234 ... 

will make port 1234 on host mapped even if image Dockerfile does not use EXPOSE.

In docker rule this behaves different - when EXPOSE is not used it is not possible to manually expose it with expose(..) option like:

@ClassRule
public static DockerRule testee = DockerRule.builder()//
        .imageName("someimage")//
        .expose("1234", "1234")//
        //...
        .build();

will not map port on host.

    @ClassRule
    public static DockerRule postgresRule = DockerRule.builder()
            .imageName("postgres:9.5.13")
            .waitFor(WaitFor.logMessageSequence(
                    "waiting for server to shut down",
                    "database system is ready to accept connections."
            ))
            .build();

causes

java.lang.NoClassDefFoundError: org/glassfish/hk2/api/MultiException

	at pl.domzal.junit.docker.rule.DockerRule.<init>(DockerRule.java:81)
	at pl.domzal.junit.docker.rule.DockerRuleBuilder.build(DockerRuleBuilder.java:51)
	at my.company.tests.it.BaseIntegrationTest.<clinit>(BaseIntegrationTestTemplate.java:67)
	at sun.misc.Unsafe.ensureClassInitialized(Native Method)
	at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
	at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
	at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
	at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
	at java.lang.reflect.Field.get(Field.java:393)
	at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73)
	at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230)
	at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255)
	at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244)
	at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:362)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.ClassNotFoundException: org.glassfish.hk2.api.MultiException
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 21 more

For example, this command is perfectly valid:

docker run -d -p "80" -p "80" ...

But this test case will fail:

@Rule
public static DockerRule testee = DockerRule.builder()
        .imageName("nginx")
        .expose("80")
        .expose("80")
        .build();

@Test
public void shouldExposeSpecifiedPort() throws InterruptedException, IOException {
    String homepage = "http://"+testee.getDockerHost()+":"+testee.getExposedContainerPort("80")+"/";
}

with exception:

java.lang.IllegalStateException: binding list for 80/tcp is longer than 1
    at pl.domzal.junit.docker.rule.DockerRule.getExposedContainerPort(DockerRule.java:319)
    at pl.domzal.junit.docker.rule.DockerRuleExposePortStaticTest.shouldExposeSpecifiedPort(DockerRuleExposePortStaticTest.java:22)

Docker rule should allow publishing same port multiple times and getExposedContainerPort should return first external port it mapped to (if it is published multiple times - using not first one it makes any difference?).

Could we have an 0.3 release on Maven Central?

Right now our Jenkins is running in a container and thus the waitForTcpPort conditions which try to connect to ports exposed on the Docker host fail when running the tests in the CI.

The new StartCondition + TcpPortChecker classes from the SNAPSHOT version would come handy (I could use them to connect directly to the container port).

Unfortunately my company's Nexus is not mirroring the Sonatype repo, so I can not use the snapshot from there.

BTW, thank you for this great library!

Csaba

Version 0.4.
Test case:

package pl.domzal.junit.docker.rule.regression;

import org.junit.Rule;
import org.junit.Test;

import pl.domzal.junit.docker.rule.DockerRule;

public class Bug42 {

    @Rule
    public final DockerRule toxicContainer = DockerRule.builder()
            .imageName("shopify/toxiproxy")
            // commenting out this line will make it work as expected
            //.entrypoint("/go/bin/toxiproxy")
            .build();

    @Test
    public void shouldStart() {
    }

}

ends with error:

java.lang.IllegalStateException: {"message":"oci runtime error: container_linux.go:265: starting container process caused \"exec: \\\"-host=0.0.0.0\\\": executable file not found in $PATH\"\n"}
	at pl.domzal.junit.docker.rule.DockerRule.before(DockerRule.java:156)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
	...
Caused by: com.spotify.docker.client.exceptions.DockerRequestException: Request error: POST https://192.168.99.100:2376/containers/55aeec9f3c2b3b92ee89e0636fdd125040fe1f39100c73c96f2a22eab2b09f9b/start: 400, body: {"message":"oci runtime error: container_linux.go:265: starting container process caused \"exec: \\\"-host=0.0.0.0\\\": executable file not found in $PATH\"\n"}
	at com.spotify.docker.client.DefaultDockerClient.propagate(DefaultDockerClient.java:2691)
	at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:2652)
	...
Caused by: javax.ws.rs.BadRequestException: HTTP 400 Bad Request
	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:999)

In case of docker daemon request exceptions message is nor displayed or propagated in exception thrown

Hello,

I just find a little bug.

If an image don't have a repo tags, there's a null pointer and the docker container is not runned.

Here's the produced stack :

java.lang.ExceptionInInitializerError: null
at pl.domzal.junit.docker.rule.DockerRule.imageAvaliable(DockerRule.java:152)
at pl.domzal.junit.docker.rule.DockerRule.(DockerRule.java:72)
at pl.domzal.junit.docker.rule.DockerRuleBuilder.build(DockerRuleBuilder.java:37)

The code : (DockerRule line 152)

List listImages = dockerClient.listImages(ListImagesParam.danglingImages(false));
for (Image image : listImages) {
// IMAGE.REPOTAGS() IS NULL HERE
if (image.repoTags().contains(imageNameWithTag)) {
.....
}
}

ListImagesParam.danglingImages(false), returns an image with a null repoTags, don't know why.
In this case, your code crash, leave the loop and docker container is not launched.

Thank you, regards

Cédric

It would be very handy to just stream everything from container output to on of: jvm console / slf4j log or specified log file.
Looks like spotify client allows that.

Address GHSA-2qrg-x229-3v8q

Log4j is used in test scope so not exported along with junit-docker-rule - this makes it not crucial but it will be good to just retire quite old log4j library.

Only 'latest" tag should be pulled.

Equivalent to:
docker run .. --label "name=value"

Add between container

It seems default dependency type should be rather non-shaded.
Shaded should be used only in cases when:

• when you want to distribute product as uber-jar
• when there is a problem with some common third party dependency.

So I'm going to switch to docker client dependency to shaded and eventually produce both versions when it will be necessary.

It may be handy for rule client to have access to started container config information as returned by spotify docker client.

There are multiple methods to specify when container is considered as started, called in short "startup conditions" (all are currently available via builder waitFor... methods).

There is no possibility to define own, custom startup conditions but it would be quite handy in specific cases.

There are cases when staring / stopping container instances is required inside test scenario.
For such cases exposing before() and after() methods may be handy.

Try out this example:

    @Rule
    public DockerRule dbRule = DockerRule.builder()
            .imageName("postgres:9.6")
            .name("db-test")
            .restartPolicy(RestartPolicy.always())
            .expose("5432")
            .env("POSTGRES_USER", "testdb")
            .env("POSTGRES_PASSWORD", "testdb")
            .waitFor(WaitFor.logMessageSequence("PostgreSQL init process complete", "database system is ready to accept connections"))
            .waitFor(WaitFor.tcpPort(5432))
            .build();

First message arrives by stdout, second via stderr. They are shown in consistent order when viewing via docker logs, but the rule above often fails to match the condition.

hello,
from what I read so far @Rule and TestRule won't work anymore,
instead one should use @ExtendWith(SomeExtension.class) and AfterEachCallback & BeforeEachCallback
are you planning on supporting this ?
best regards,
peter

From time to time these test cases fails, especially under travis:
ExamplePortExposeStaticTest

Rule uses log4j api for logging.
Should use generic slf4j api and leave decision what logging framework will be used for target environment.

hello,
I've been using version 0.3 for quite some time -which worked like a charm-, but after migrating to spring boot 2.0.2 it won't work anymore, see exception:

java.lang.NoClassDefFoundError: org/glassfish/hk2/api/MultiException
at pl.domzal.junit.docker.rule.DockerRule.<init>(DockerRule.java:81)
at pl.domzal.junit.docker.rule.DockerRuleBuilder.build(DockerRuleBuilder.java:51)

upgrading to version 0.4 didn't fixed that.
have you experienced the same issue?

Linking requires that container link points to is statically names which is problematic in any CI environment.

expose(...) should behave similarly to command line client and allow mapping udp socket ports:

... -p 1234:1234/udp ...

Example stack:

com.spotify.docker.client.ContainerNotFoundException: Container not found: 442debb3feb694708fc70ec4e8f0da3b92a3c6df8f491430fa937be69f8cab60
at com.spotify.docker.client.DefaultDockerClient.containerAction(DefaultDockerClient.java:453)
at com.spotify.docker.client.DefaultDockerClient.startContainer(DefaultDockerClient.java:441)
at pl.domzal.junit.docker.rule.DockerRule.before(DockerRule.java:95)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: com.spotify.docker.client.DockerRequestException: Request error: POST https://192.168.99.100:2376/containers/442debb3feb694708fc70ec4e8f0da3b92a3c6df8f491430fa937be69f8cab60/start: 404
at com.spotify.docker.client.DefaultDockerClient.propagate(DefaultDockerClient.java:1218)
at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:1189)
at com.spotify.docker.client.DefaultDockerClient.containerAction(DefaultDockerClient.java:449)
... 11 more
Caused by: com.spotify.docker.client.shaded.javax.ws.rs.NotFoundException: HTTP 404 Not Found
at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:990)
at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:799)
at org.glassfish.jersey.client.JerseyInvocation.access$500(JerseyInvocation.java:91)
at org.glassfish.jersey.client.JerseyInvocation$5.completed(JerseyInvocation.java:760)
at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:197)
at org.glassfish.jersey.client.ClientRuntime.access$300(ClientRuntime.java:78)
at org.glassfish.jersey.client.ClientRuntime$2.run(ClientRuntime.java:179)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:340)
at org.glassfish.jersey.client.ClientRuntime$3.run(ClientRuntime.java:209)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

There are cases when rule is used in not so common scenarios and exposing internal DockerClient instancje may be handy.
F.e.: I have need monitor container cpu usage during test.

Currently there is getDockerClient() method on rule with package access - I can make it public.

Encountering an odd issue with this rule on our project.
For the MQLight container ( https://developer.ibm.com/messaging/2015/02/11/introducting-docker-mqlight/ ), I see the container shut down before the test case has completed.

If you get a chance the issue can be reproduced by running:
miniature-queue\mqlight\src\test\java\com\github\mrchris2000\queue\mqlight\example\FanoutTest.java

The repository is : https://github.com/mrchris2000/miniature-queue

This should be self contained enough for a simple reproduction.
Any clues as to the problem are greatly appreciated.

Like in command line:

docker run -e VAR=value ...