Jib 3.4.2 build fails about jib HOT 22 OPEN

Moving the jib plugin version definition to the root of my multi-module project with apply false and using it at the subproject solved the issue for me as well.

from jib.

Here is a non multi-module project buildEnvironment (also spring-boot but not using the spring-boot gradle plugin), our compress comes from Jib and it also broke our jibBuildTar step after 3.4.2

+--- com.google.cloud.tools.jib:com.google.cloud.tools.jib.gradle.plugin:3.4.2
|    \--- com.google.cloud.tools:jib-gradle-plugin:3.4.2
|         +--- org.apache.commons:commons-compress:1.26.0

This is on Gradle (wrapper) 8.6, Java 21, and Linux (Fedora 38, but also failing in GitHub CI on ubuntu-latest).

It seems to be the same issue as reported above:

> Task :jibBuildTar FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jibBuildTar'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: 'void org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(org.apache.commons.compress.archivers.tar.TarArchiveEntry)'

Stacktrace:

Caused by: java.lang.NoSuchMethodError: 'void org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(org.apache.commons.compress.archivers.tar.TarArchiveEntry)'
        at com.google.cloud.tools.jib.tar.TarStreamBuilder.writeAsTarArchiveTo(TarStreamBuilder.java:52)
        at com.google.cloud.tools.jib.hash.Digests.computeDigest(Digests.java:104)
        at com.google.cloud.tools.jib.blob.WritableContentsBlob.writeTo(WritableContentsBlob.java:37)
        at com.google.cloud.tools.jib.cache.CacheStorageWriter.writeUncompressedLayerBlobToDirectory(CacheStorageWriter.java:430)
        at com.google.cloud.tools.jib.cache.CacheStorageWriter.writeUncompressed(CacheStorageWriter.java:282)
        at com.google.cloud.tools.jib.cache.Cache.writeUncompressedLayer(Cache.java:145)
        at com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep.call(BuildAndCacheApplicationLayerStep.java:108)
        at com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep.call(BuildAndCacheApplicationLayerStep.java:38)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)

from jib.

@chanseokoh Aligning the versions in de root project plugins block fixes the problem in our multi-module project. Good to know about this obscure Gradle issue/bug, thanks!

from jib.

you can take any base spring boot project on 3.2.4

from jib.

I had some free time to look into why my build fails and in my case buildEnvironment wasn't telling me the truth, because it did not include the dependencies from buildSrc (I have a custom plugin for jOOQ that spins up a testcontainer, to build the DB and generate the DSL).

demo/buildSrc$ ../gradlew -q dependencies

Revealed this:

\--- org.testcontainers:postgresql -> 1.19.7
     \--- org.testcontainers:jdbc:1.19.7
          \--- org.testcontainers:database-commons:1.19.7
               \--- org.testcontainers:testcontainers:1.19.7
                    +--- junit:junit:4.13.2
                    |    \--- org.hamcrest:hamcrest-core:1.3 -> 2.2
                    |         \--- org.hamcrest:hamcrest:2.2
                    +--- org.slf4j:slf4j-api:1.7.36 -> 2.0.12
                    +--- org.apache.commons:commons-compress:1.24.0

While running ./gradlew buildEnvironment was telling me that nothing uses commons-compress except jib and it is on version 1.26.0 which wasn't the case.

And in this particular case testcontainers already tracks the update, so it's just a matter of time until the order is restored again... testcontainers/testcontainers-java#8338

The fix was also simple once I figured out what was the problem, I've added the following into my buildSrc/build.gradle where I had testcontainers defined:

// buildSrc/build.gradle
dependencies {
  // other plugin deps
  implementation 'org.testcontainers:postgresql:1.19.7'
  implementation 'org.apache.commons:commons-compress:1.26.1'
}

And now both, testcontainers and jib, seem to be working for me.

from jib.

@mihalyr-prospect there you go! As you saw, the cause is that Gradle pulls in an old version of some library in your build environment for one reason or another where ./gradlew buildEnv lies, which is a known Gradle issue. All you have to do is to resolve your diamond dependency conflict in your project in one way or another. Glad you were able to fix it.

from jib.

Facing the same issue also! I hope that will be fixed soon

from jib.

Hey @artemptushkin, thanks for filing the issue and appreciate you sharing the workaround! As we try to learn more, would it be possible to see how your project's dependency tree looks without the explicit inclusion of org.apache.commons:commons-compress:1.26.0? Particularly curious if the dependency is being transitively brought in from another place.

from jib.

Are there any updates?

from jib.

We have the same issue in the jib-gradle-plugin (without the workaround). A gradlew buildEnvironment shows org.apache.commons:commons-compress:1.26.0 is requested by jib-gradle-plugin itself:

\--- com.google.cloud.tools:jib-gradle-plugin:3.4.2
     +--- com.google.http-client:google-http-client:1.42.2
     |    ...
     +--- com.google.http-client:google-http-client-apache-v2:1.42.2
     |    ...
     +--- com.google.auth:google-auth-library-oauth2-http:1.10.0
     |    ...
     +--- org.apache.commons:commons-compress:1.26.0

Other transitive dependencies request 1.21 but Gradle resolves the conflict with the newer version requested by jib-gradle-plugin. But the stacktrace shows it is the jib-gradle-plugin itself which breaks on this dependency, as can be seen in the first post.

from jib.

@adenhartog thanks for the trace

My

|         |    +--- org.apache.commons:commons-compress:1.21

comes from

+--- org.springframework.boot:org.springframework.boot.gradle.plugin:3.2.4

Though no "commons-codec" in the trace for ./gradlew buildEnvironment

from jib.

For comparison, we also have the spring-boot-gradle plugin but in our project. It requests 1.21 but gradle resolves it to 1.26.0 in our project:

          |    \--- org.springframework.boot:spring-boot-gradle-plugin:3.2.4
          |         +--- org.springframework.boot:spring-boot-buildpack-platform:3.2.4
          |         |    ...
          |         |    +--- org.apache.commons:commons-compress:1.21 -> 1.26.0 (*)

from jib.

Other transitive dependencies request 1.21 but Gradle resolves the conflict with the newer version requested by jib-gradle-plugin.

For comparison, we also have the spring-boot-gradle plugin but in our project. It requests 1.21 but gradle resolves it to 1.26.0 in our project:

If your project is a multi-module one, then it's pretty common that ./gradlew buildEnvironment lies about how it resolved build environment dependencies; it's a long-standing Gradle bug where build dependencies declared in the parent project shamelessly and silently overrides dependencies of sub-projects, as opposed to what you see from ./gradlew buildEnv.

So, if you haven't done so, I recommend first that you strictly follow these instructions for setting up your multi-module project. That is, declare all your plugins in the project root while not applying them and then selective apply plugins in sub-projects.

UPDATE: another case of Gradle lying with ./gradlew buildEnv was #4235 (comment) where an old version of the Apache library was silently pulled in from buildSrc/build.gradle.

from jib.

Another thing, I saw the workaround is to enforce commons-codec version 1.16, in our project is at version 1.11 currently, which is also pulled in by jib

+--- com.google.cloud.tools.jib:com.google.cloud.tools.jib.gradle.plugin:3.4.2
|    \--- com.google.cloud.tools:jib-gradle-plugin:3.4.2
|         +--- com.google.http-client:google-http-client:1.42.2
|         |    +--- org.apache.httpcomponents:httpclient:4.5.13
|         |    |    +--- org.apache.httpcomponents:httpcore:4.4.13 -> 4.4.15
|         |    |    +--- commons-logging:commons-logging:1.2
|         |    |    \--- commons-codec:commons-codec:1.11

from jib.

@adenhartog glad you fixed your project.

@mihalyr-prospect I think it must be that for one reason or another, your project is set up to "downgrade" the Apache commons compress 1.26.0, which is the version Jib requires. You should ensure that your build environment doesn't downgrade it. If you have to force it, do so. I can take a look if you have a reproducible sample, but I don't think Jib is doing anything wrong about it.

from jib.

@chanseokoh I shared above our buildEnvironment that shows that we use commons compress 1.26.0 that is only pulled in by jib and nothing should depend on it or "downgrade" it as far as I can see. Unfortunately, I don't have a sample as it is a private project and right now this isn't that important for us to debug this further. For now I just skip this update, it was supposed to be only a patch bump that broke our CI, we don't really need anything from the new version right now.

from jib.

Is there a fix in the next release? or we should force the usage of commons-compress & commons-codec?
Anyway at the moment, we downgraded also to the previous version.

from jib.

FWIW, there is nothing to fix in Jib. Jib asks to use the latest version of the libraries, but somehow your project is pulling an old version. Basically, it is your responsibility to correctly set up your project so that it doesn't happen. If your project is a multi-module one, make sure you follow #4235 (comment). Sometimes, you will have to manually resolve this kind of diamond dependency conflicts.

from jib.

@chanseokoh no, that's not fully right, there is no "commons-codec:commons-codec" at all as you see I have to define it by myself. There are plenty of comments here, you could take a look if you can fix it, usually it's a case that we have to enforce a libraries version in the build classpath - it doesn't happen for other plugins

from jib.

If anyone can provide a reproducible sample, I can help.

from jib.

I am not the maintainer of this repo. I am contributing my own time to help the community. Please provide a reproducible sample if it is that easy to create.

from jib.

Okay, two dependencies:

• commons-codec:commons-codec - no dependency at all, your trace confirms
• commons-compress - wrong version, can we enforce or anything? gradle resolves the lower one it happens I know

from jib.