testcontainers / dind-drone-plugin Goto Github PK

View Code? Open in Web Editor NEW

28.0 28.0 19.0 53 KB

Plugin for Drone CI v0.8+ to enable use of Testcontainers using Docker-in-Docker

License: Apache License 2.0

Dockerfile 6.44% Shell 93.56%

dind-drone-plugin's People

Contributors

Stargazers

Watchers

Forkers

jopecko smeb lucasventurasc legart johnny850807 gaybro8777 isabella232 igorrybalka reddit simoneborgio georgecodes kalrover timbze devvault petenattress danielt998

dind-drone-plugin's Issues

Dependency caching for test execution

Test phase is really slow using the plugin, mainly because I'm unable to cache Maven dependencies.

I'm currently working on this pipeline:

kind: pipeline
type: docker
name: default

steps:
  - name: test
    image: quay.io/testcontainers/dind-drone-plugin
    environment:
      CI_WORKSPACE: "/drone/src"
    settings:
      # This image will run the cmd with your build steps
      build_image: maven:3-openjdk-17-slim
      # This specifies the command that should be executed to perform build, test and
      #  integration tests. Not to be confused with Drone's `command`:
      cmd: sleep 5 && mvn clean test
      prefetch_images:
        - "maven:3-openjdk-17-slim"
#      volumes:
#        - name: maven-repo
#          path: /root/.m2
    volumes:
      - name: dockersock
        path: /var/run

  - name: build
    image: maven:3-openjdk-17-slim
    commands:
      - mvn clean package -DskipTests
    volumes:
      - name: maven-repo
        path: /root/.m2

# Specify docker:dind as a service
services:
  - name: docker
    image: docker:dind
    privileged: true
    volumes:
      - name: dockersock
        path: /var/run

volumes:
  - name: dockersock
    temp: { }
  - name: maven-repo
    host:
      path: /home/administrator/drone/cache/maven-repo

maven-repo volume is used as cache for Maven dependencies. It works fine in the build step, but AFAIK it is not possible to use the same approach in test (see the commented lines below prefetch_images).
Moreover, test step downloads the maven:3-openjdk-17-slim image every time it is executed.
The result is test step is taking 11 minutes, while the actual build step only requires 22 seconds.

Is it possible to cache dependencies and use them in a test step with dind or is there another best-practice to improve this performance?

Do this approach works on AWS Fargate?

We have our CI server running in AWS Fargate and Fargate currently dont allow docker in docker. I want to check if this approach can be implemented in the AWS Fargate running container and will it work ?

Do we require elevated permission for test containers in a docker as Fargate dont allow it .

Update the documentation with support for newer drone versions

Hi guys, first of all thank you very much for sharing this plugin with the community it was very useful for us.

After try a lot of times we finally were able to use it once we realize that the documentation was maybe a bit outdated and having a little update on it could help others to use it faster and maybe not give up on it.

If you guys don't mind I will submit a PR with an example of the way it worked for us, maybe regarding to the drone version or something but, unfortunately, the current documentation as is did not worked for us.

Support platform linux/arm64

if I'm not mistaken, there is currently no arm64 image available of the plugin.
Could you please build and push the current version with --platform linux/arm64?

Thanks!

Repo not maintained?

It seems like this repo is not maintained?

I have forked and made my own docker image where the docker:dind image is updated and also implemented #19

If this repo is going to be maintained I would be more than happy to make a PR here. I'm not looking for more stuff to maintain 😄

https://github.com/timbze/dind-drone-plugin
https://hub.docker.com/repository/docker/timbze/dind-drone-plugin/general

How do I mount a volume from drone runner to the dind container?

Testcontainers v1.16.0 - Could not find a valid Docker environment

The same configuration works fine on v1.15.2 and v1.15.3, however, it fails on v1.16.0. The drone.yml (Drone v2.0.4) is:

kind: pipeline
type: docker
name: verify

steps:
- name: test
  image: quay.io/testcontainers/dind-drone-plugin
  environment:
    CI_WORKSPACE: "/drone/src"
    EXTRA_DOCKER_OPTIONS: "--add-host=private.repo.local:192.168.11.19"
  settings:
    cmd: sleep 10 && mkdir -p /root/.m2 && cp ./drone/settings.xml /root/.m2/settings.xml && ./mvnw clean verify -Pwith-DockerTests -ntp
    build_image: adoptopenjdk/openjdk11:debian-slim
    prefetch_images:
    - "quay.io/keycloak/keycloak:14.0.0"
  volumes:
  - name: dockersock
    path: /var/run
  - name: cache
    path: /root/.m2

services:
- name: docker
  image: docker:dind
  privileged: true
  volumes:
  - name: dockersock
    path: /var/run

volumes:
- name: dockersock
  temp: {}
- name: cache
  temp: {}

Output Errors:

[ERROR] microservices.quarkus.json.customexceptions.CustomExceptionMappersDockerTest.testResourceWithExceptionMapper  Time elapsed: 0.007 s  <<< ERROR!
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
Caused by: java.lang.reflect.InvocationTargetException
Caused by: java.lang.RuntimeException: Error waiting for test resource future to finish.
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Unable to start Quarkus test resource microservices.quarkus.json.KeycloakServer@3a5739d2
Caused by: java.lang.RuntimeException: Unable to start Quarkus test resource microservices.quarkus.json.KeycloakServer@3a5739d2
Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration

[INFO] Running microservices.quarkus.json.CodeFlowDockerTest
2021-08-11 20:27:00,203 WARN  [org.tes.uti.TestcontainersConfiguration] (pool-4-thread-1) Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
2021-08-11 20:27:00,211 INFO  [org.tes.doc.DockerClientProviderStrategy] (pool-4-thread-1) Loaded org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy from ~/.testcontainers.properties, will try it first
2021-08-11 20:27:00,220 INFO  [org.tes.doc.DockerMachineClientProviderStrategy] (pool-4-thread-1) docker-machine executable was not found on PATH ([/opt/java/openjdk/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin])
2021-08-11 20:27:03,046 ERROR [org.tes.doc.DockerClientProviderStrategy] (pool-4-thread-1) Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
2021-08-11 20:27:03,046 ERROR [org.tes.doc.DockerClientProviderStrategy] (pool-4-thread-1)     EnvironmentAndSystemPropertyClientProviderStrategy: failed with exception NullPointerException (Host name must not be null)
2021-08-11 20:27:03,046 ERROR [org.tes.doc.DockerClientProviderStrategy] (pool-4-thread-1)     EnvironmentAndSystemPropertyClientProviderStrategy: failed with exception NullPointerException (Host name must not be null)
2021-08-11 20:27:03,046 ERROR [org.tes.doc.DockerClientProviderStrategy] (pool-4-thread-1)     UnixSocketClientProviderStrategy: failed with exception RuntimeException (java.lang.NoClassDefFoundError: com/github/dockerjava/api/model/DockerObject). Root cause ClassNotFoundException (com.github.dockerjava.api.model.DockerObject)
2021-08-11 20:27:03,046 ERROR [org.tes.doc.DockerClientProviderStrategy] (pool-4-thread-1) As no valid configuration was found, execution cannot continue
[ERROR] Tests run: 3, Failures: 0, Errors: 1, Skipped: 2, Time elapsed: 2.299 s <<< FAILURE! - in microservices.quarkus.json.CodeFlowDockerTest
[ERROR] microservices.quarkus.json.CodeFlowDockerTest.testTokenTimeoutLogout  Time elapsed: 0 s  <<< ERROR!
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
Caused by: java.lang.reflect.InvocationTargetException
Caused by: java.lang.RuntimeException: Error waiting for test resource future to finish.
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Unable to start Quarkus test resource microservices.quarkus.json.KeycloakServer@4807285e
Caused by: java.lang.RuntimeException: Unable to start Quarkus test resource microservices.quarkus.json.KeycloakServer@4807285e
Caused by: java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration

Is this issue more likely related to a change in the API of testcontainers rather than this dind-drone-plugin?

Looking forward to any hint of how to debug this error.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.