Support artifact checksum pinning about rules_jvm_external HOT 15 CLOSED

One benefit of a lock file not yet discussed here is that the artifacts are pre-resolved to an artifact URL and shasum. This can save a lot of time every time you would normally need to re-run the coursier resolver (several minutes in our repository). Doing the resolution once and storing that information to a lock file saves us a ton of time in our CI builds.

from rules_jvm_external.

I agree. This is something that I'll discuss with @aehlig, who implemented the Bazel-wide lock file for external repositories (--experimental_workspace_resolved_file).

from rules_jvm_external.

Work in progress to support artifact checksum pinning in a file read and exported by maven_install: #169

from rules_jvm_external.

I'm probably doing this incorrectly, but the --checksum flag doesn't seem to be doing what I think it should do:

$ sha1sum /usr/local/google/home/jingwen/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar
88e3c593e9b3586e1c6177f89267da6fc6986f0c  /usr/local/google/home/jingwen/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar

$ cat /usr/local/google/home/jingwen/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar.sha1 
88e3c593e9b3586e1c6177f89267da6fc6986f0c

$ coursier fetch com.google.errorprone:error_prone_annotations:2.2.0 --checksum 88e3c593e9b3586e1c6177f89267da6fc6986f0c
mirrorConfFiles=List(MirrorConfFile(/usr/local/google/home/jingwen/.config/coursier/mirror.properties, true))
allMirrors0=List()
Resolution error: Error downloading com.google.errorprone:error_prone_annotations:2.2.0
  not found: /usr/local/google/home/jingwen/.ivy2/local/com.google.errorprone/error_prone_annotations/2.2.0/ivys/ivy.xml
  checksum not found: 

from rules_jvm_external.

Looks like Coursier CLI doesn't support this yet, according to the referenced issue. We may have to do the checksum verification out of band, after downloading the jars and possibly while generating the BUILD targets.

from rules_jvm_external.

Would this issue also cover "locking" versions of all transitive dependencies? (Like Cargo.lock etc.)

from rules_jvm_external.

@arjantop That's #107

from rules_jvm_external.

@jin #107, does not seem to be about generating or using a .lock file.

from rules_jvm_external.

My understanding is that locking and pinning are similar to each other. #107 talks about locking top level artifacts to specific versions, but doesn't cover transitive dependencies. Let's continue discussions there?

from rules_jvm_external.

@jin If I understand #107 it is about overriding/pinning specific transitive dependencies to some version, .lock file would be generated automatically, pinning all versions to the resolved ones

from rules_jvm_external.

Yes, and I imagine lock file generation will involve Bazel WORKSPACE resolve files to a large extent.

from rules_jvm_external.

@jin Maybe this issue should be renamed "Support artifact checksum pinning" or something to differentiate it from #134 which is more actually verification IMHO

from rules_jvm_external.

@jin any thought on the benefit of a lock file that @JaredNeil mentioned?

from rules_jvm_external.

@jin just a follow-up. Any update on the lock file?

from rules_jvm_external.

Sorry, no updates on this yet.

from rules_jvm_external.