Git Product home page Git Product logo

rules_jvm_external's Introduction

rules_jvm_external

Transitive Maven artifact resolution and publishing rules for Bazel.

  • Main build: Build Status
  • Examples build: Build status

Table of Contents

Features

  • MODULE.bazel bzlmod configuration (Bazel 7 and above)
  • WORKSPACE configuration
  • Artifact version resolution with Coursier or Maven
  • Import downloaded JAR, AAR, source JARs
  • Export built JARs to Maven repositories
  • Pin resolved artifacts with their SHA-256 checksums into a version-controllable JSON file
  • Custom Maven repositories
  • Private Maven repositories using netrc files
  • Integration with Bazel's downloader and caching mechanisms for sharing artifacts across Bazel workspaces
  • Versionless target labels for simpler dependency management
  • Ability to declare multiple sets of versioned artifacts
  • Supported on Windows, macOS, Linux

Get the latest release here.

Examples

You can find examples in the examples/ directory.

Projects using rules_jvm_external

Find other GitHub projects using rules_jvm_external with this search query.

Prerequisites

  • Bazel 5.4.1 and above.

Support for Bazel versions between 4.x and 5.4 is only available on rules_jvm_external releases 5.x.

Support for Bazel versions before 4.0.0 is only available on rules_jvm_external releases 4.2 or earlier.

Usage

With bzlmod (Bazel 7 and above)

If you are starting a new project, or your project is already using Bazel 7 and above, we recommend using bzlmod to manage your external dependencies, including Maven dependencies with rules_jvm_external. It address several shortcomings of the WORKSPACE mechanism. If you are unable to use bzlmod, rules_jvm_external also supports the WORKSPACE mechanism (see below).

See bzlmod.md for the usage instructions. bzlmod is on-by-default in Bazel 7.0.

With WORKSPACE file (legacy)

List the top-level Maven artifacts and servers in the WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "4.5"
RULES_JVM_EXTERNAL_SHA = "b17d7388feb9bfa7f2fa09031b32707df529f26c91ab9e5d909eb1676badd9a6"

http_archive(
    name = "rules_jvm_external",
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    sha256 = RULES_JVM_EXTERNAL_SHA,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps")

rules_jvm_external_deps()

load("@rules_jvm_external//:setup.bzl", "rules_jvm_external_setup")

rules_jvm_external_setup()

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "junit:junit:4.12",
        "androidx.test.espresso:espresso-core:3.1.1",
        "org.hamcrest:hamcrest-library:1.3",
    ],
    repositories = [
        # Private repositories are supported through HTTP Basic auth
        "http://username:password@localhost:8081/artifactory/my-repository",
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
    ],
)

Credentials for private repositories can also be specified using a property file or environment variables. See the Coursier documentation for more information.

rules_jvm_external_deps uses a default list of maven repositories to download rules_jvm_external's own dependencies from. Should you wish to change this, use the repositories parameter, and also set the path to the lock file:

rules_jvm_external_deps(
   repositories = ["https://mycorp.com/artifacts"],
   deps_lock_file = "@//:rules_jvm_external_deps_install.json")
rules_jvm_external_setup()

If you are using bzlmod, define an install tag in your root MODULE.bazel which overrides the values:

maven.install(
    name = "rules_jvm_external_deps",
    repositories = ["https://mycorp.com/artifacts"],
    lock_file = "@//:rules_jvm_external_deps_install.json",
)

Once these changes have been made, repin using REPIN=1 bazel run @rules_jvm_external_deps//:pin and commit the file to your version control system (note that at this point you will need to maintain your customized rules_jvm_external_deps_install.json):

Next, reference the artifacts in the BUILD file with their versionless label:

java_library(
    name = "java_test_deps",
    exports = [
        "@maven//:junit_junit",
        "@maven//:org_hamcrest_hamcrest_library",
    ],
)

android_library(
    name = "android_test_deps",
    exports = [
        "@maven//:junit_junit",
        "@maven//:androidx_test_espresso_espresso_core",
    ],
)

The default label syntax for an artifact foo.bar:baz-qux:1.2.3 is @maven//:foo_bar_baz_qux. That is,

  • All non-alphanumeric characters are substituted with underscores.
  • Only the group and artifact IDs are required.
  • The target is located in the @maven top level package (@maven//).

API Reference

You can find the complete API reference at docs/api.md.

Pinning artifacts and integration with Bazel's downloader

rules_jvm_external supports pinning artifacts and their SHA-256 checksums into a maven_install.json file that can be checked into your repository.

Without artifact pinning, in a clean checkout of your project, rules_jvm_external executes the full artifact resolution and fetching steps (which can take a bit of time) and does not verify the integrity of the artifacts against their checksums. The downloaded artifacts also cannot be shared across Bazel workspaces.

By pinning artifact versions, you can get improved artifact resolution and build times, since using maven_install.json enables rules_jvm_external to integrate with Bazel's downloader that caches files on their sha256 checksums. It also improves resiliency and integrity by tracking the sha256 checksums and original artifact urls in the JSON file.

Since all artifacts are persisted locally in Bazel's cache, it means that fully offline builds are possible after the initial bazel fetch @maven//.... The artifacts are downloaded with http_file which supports netrc for authentication. Your ~/.netrc will be included automatically. To pass machine login credentials in the ~/.netrc file to coursier, specify use_credentials_from_home_netrc_file = True in your maven_install rule. For additional credentials, add them in the repository URLs passed to maven_install (so they will be included in the generated JSON). Alternatively, pass an array of additional_netrc_lines to maven_install for authentication with credentials from outside the workspace.

To get started with pinning artifacts, run the following command to generate the initial maven_install.json at the root of your Bazel workspace:

$ bazel run @maven//:pin

Then, specify maven_install_json in maven_install and load pinned_maven_install from @maven//:defs.bzl:

maven_install(
    # artifacts, repositories, ...
    maven_install_json = "//:maven_install.json",
)

load("@maven//:defs.bzl", "pinned_maven_install")
pinned_maven_install()

Note: The //:maven_install.json label assumes you have a BUILD file in your project's root directory. If you do not have one, create an empty BUILD file to fix issues you may see. See #242

Note: If you're using an older version of rules_jvm_external and haven't repinned your dependencies, you may see a warning that you lock file "does not contain a signature of the required artifacts" then don't worry: either ignore the warning or repin the dependencies.

Updating maven_install.json

Whenever you make a change to the list of artifacts or repositories and want to update maven_install.json, run this command to re-pin the unpinned @maven repository:

$ REPIN=1 bazel run @maven//:pin

Without re-pinning, maven_install will not pick up the changes made to the WORKSPACE, as maven_install.json is now the source of truth.

Requiring lock file repinning when the list of artifacts changes

It can be easy to forget to update the maven_install.json lock file when updating artifacts in a maven_install. Normally, rules_jvm_external will print a warning to the console and continue the build when this happens, but by setting the fail_if_repin_required attribute to True, this will be treated as a build error, causing the build to fail. When this attribute is set, it is possible to update the maven_install.json file using:

# To repin everything:
REPIN=1 bazel run @maven//:pin

# To only repin rules_jvm_external:
RULES_JVM_EXTERNAL_REPIN=1 bazel run @maven//:pin

Alternatively, it is also possible to modify the fail_if_repin_required attribute in your WORKSPACE file, run bazel run @maven//:pin and then reset the fail_if_repin_required attribute.

Custom location for maven_install.json

You can specify a custom location for maven_install.json by changing the maven_install_json attribute value to point to the new file label. For example:

maven_install(
    name = "maven_install_in_custom_location",
    artifacts = ["com.google.guava:guava:27.0-jre"],
    repositories = ["https://repo1.maven.org/maven2"],
    maven_install_json = "@rules_jvm_external//tests/custom_maven_install:maven_install.json",
)

load("@maven_install_in_custom_location//:defs.bzl", "pinned_maven_install")
pinned_maven_install()

Future artifact pinning updates to maven_install.json will overwrite the file at the specified path instead of creating a new one at the default root directory location.

Multiple maven_install.json files

If you have multiple maven_install declarations, you have to alias pinned_maven_install to another name to prevent redefinitions:

maven_install(
    name = "foo",
    maven_install_json = "//:foo_maven_install.json",
    # ...
)

load("@foo//:defs.bzl", foo_pinned_maven_install = "pinned_maven_install")
foo_pinned_maven_install()

maven_install(
    name = "bar",
    maven_install_json = "//:bar_maven_install.json",
    # ...
)

load("@bar//:defs.bzl", bar_pinned_maven_install = "pinned_maven_install")
bar_pinned_maven_install()

(Experimental) Support for Maven BOM files

Support for Maven BOMs can be enabled by switching the resolver used by maven_install to one that supports Maven BOMs. This can be done by setting the resolver attribute to maven. The new resolver will likely result in different resolutions than the existing resolver, so it is advised to re-run your dependencies pin.

The new resolver requires you to use a maven_install_json file, though if you have not yet pinned your dependencies, this can simply be an empty file.

As an example:

maven.install(
    # Resolution using BOMs is supported by using the `maven` resolver
    resolver = "maven",
    boms = [
        "org.seleniumhq.selenium:selenium-bom:4.18.1",
    ],
    artifacts = [
        # This dependency is included in the `selenium-bom`, so we can omit the version number
        "org.seleniumhq.selenium:selenium-java",
    ],
    # The `maven` resolver requires a lock file, though this can be an empty file before pinning
    lock_file = "@//:manifest_install.json",
)    

Generated targets

For the junit:junit example, using bazel query @maven//:all --output=build, we can see that the rule generated these targets:

alias(
  name = "junit_junit_4_12",
  actual = "@maven//:junit_junit",
)

jvm_import(
  name = "junit_junit",
  jars = ["@maven//:https/repo1.maven.org/maven2/junit/junit/4.12/junit-4.12.jar"],
  srcjar = "@maven//:https/repo1.maven.org/maven2/junit/junit/4.12/junit-4.12-sources.jar",
  deps = ["@maven//:org_hamcrest_hamcrest_core"],
  tags = ["maven_coordinates=junit:junit:4.12"],
)

jvm_import(
  name = "org_hamcrest_hamcrest_core",
  jars = ["@maven//:https/repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar"],
  srcjar = "@maven//:https/repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3-sources.jar",
  deps = [],
  tags = ["maven_coordinates=org.hamcrest:hamcrest.library:1.3"],
)

These targets can be referenced by:

  • @maven//:junit_junit
  • @maven//:org_hamcrest_hamcrest_core

Transitive classes: To use a class from hamcrest-core in your test, it's not sufficient to just depend on @maven//:junit_junit even though JUnit depends on Hamcrest. The compile classes are not exported transitively, so your test should also depend on @maven//:org_hamcrest_hamcrest_core.

Original coordinates: The generated tags attribute value also contains the original coordinates of the artifact, which integrates with rules like bazel-common's pom_file for generating POM files. See the pom_file_generation example for more information.

Outdated artifacts

To check for updates of artifacts, run the following command at the root of your Bazel workspace:

$ bazel run @maven//:outdated

Advanced usage

Fetch source JARs

To download the source JAR alongside the main artifact JAR, set fetch_sources = True in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    fetch_sources = True,
)

Checksum verification

Artifact resolution will fail if a SHA-1 or MD5 checksum file for the artifact is missing in the repository. To disable this behavior, set fail_on_missing_checksum = False in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    fail_on_missing_checksum = False,
)

Using a custom Coursier download url

By default bazel bootstraps Coursier via the urls specificed in versions.bzl. However in case they are not directly accessible in your environment, you can also specify a custom url to download Coursier. For example:

$ bazel build @maven_with_unsafe_shared_cache//... --repo_env=COURSIER_URL='https://my_secret_host.com/vXYZ/coursier.jar'

Please note it still requires the SHA to match.

artifact helper macro

The artifact macro translates the artifact's group:artifact coordinates to the label of the versionless target. This target is an alias that points to the java_import/aar_import target in the @maven repository, which includes the transitive dependencies specified in the top level artifact's POM file.

For example, @maven//:junit_junit is equivalent to artifact("junit:junit").

To use it, add the load statement to the top of your BUILD file:

load("@rules_jvm_external//:defs.bzl", "artifact")

Full group:artifact:[packaging:[classifier:]]version maven coordinates are also supported and translate to corresponding versionless target.

Note that usage of this macro makes BUILD file refactoring with tools like buildozer more difficult, because the macro hides the actual target label at the syntax level.

java_plugin_artifact helper macro

The java_plugin_artifact macro finds a java_plugin target which can be used to run an annotation procesor from a particular artifact.

For example, if you pull com.google.auto.value:auto-value into a maven_install, you can use the java_plugin_artifact macro in the plugins attribute of a target like java_library:

java_library(
    name = "some_lib",
    srcs = ["SrcUsingAuto.java"],
    plugins = [
        java_plugin_artifact("com.google.auto.value:auto-value", "com.google.auto.value.processor.AutoValueProcessor"),
    ],
)

Multiple maven_install declarations for isolated artifact version trees

If your WORKSPACE contains several projects that use different versions of the same artifact, you can specify multiple maven_install declarations in the WORKSPACE, with a unique repository name for each of them.

For example, if you want to use the JRE version of Guava for a server app, and the Android version for an Android app, you can specify two maven_install declarations:

maven_install(
    name = "server_app",
    artifacts = [
        "com.google.guava:guava:27.0-jre",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

maven_install(
    name = "android_app",
    artifacts = [
        "com.google.guava:guava:27.0-android",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

This way, rules_jvm_external will invoke coursier to resolve artifact versions for both repositories independent of each other. Coursier will fail if it encounters version conflicts that it cannot resolve. The two Guava targets can then be used in BUILD files like so:

java_binary(
    name = "my_server_app",
    srcs = ...
    deps = [
        # a versionless alias to @server_app//:com_google_guava_guava_27_0_jre
        "@server_app//:com_google_guava_guava",
    ]
)

android_binary(
    name = "my_android_app",
    srcs = ...
    deps = [
        # a versionless alias to @android_app//:com_google_guava_guava_27_0_android
        "@android_app//:com_google_guava_guava",
    ]
)

Detailed dependency information specifications

Although you can always give a dependency as a Maven coordinate string, occasionally special handling is required in the form of additional directives to properly situate the artifact in the dependency tree. For example, a given artifact may need to have one of its dependencies excluded to prevent a conflict.

This situation is provided for by allowing the artifact to be specified as a map containing all of the required information. This map can express more information than the coordinate strings can, so internally the coordinate strings are parsed into the artifact map with default values for the additional items. To assist in generating the maps, you can pull in the file specs.bzl alongside defs.bzl and import the maven struct, which provides several helper functions to assist in creating these maps. An example:

load("@rules_jvm_external//:defs.bzl", "artifact")
load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact(
            group = "com.google.guava",
            artifact = "guava",
            version = "27.0-android",
            exclusions = [
                ...
            ]
        ),
        "junit:junit:4.12",
        ...
    ],
    repositories = [
        maven.repository(
            "https://some.private.maven.re/po",
            user = "johndoe",
            password = "example-password"
        ),
        "https://repo1.maven.org/maven2",
        ...
    ],
)

Note when using bzlmod the syntax in MODULE.bazel is different than shown above.

Artifact exclusion

If you want to exclude an artifact from the transitive closure of a top level artifact, specify its group-id:artifact-id in the exclusions attribute of the maven.artifact helper:

load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact(
            group = "com.google.guava",
            artifact = "guava",
            version = "27.0-jre",
            exclusions = [
                maven.exclusion(
                    group = "org.codehaus.mojo",
                    artifact = "animal-sniffer-annotations"
                ),
                "com.google.j2objc:j2objc-annotations",
            ]
        ),
        # ...
    ],
    repositories = [
        # ...
    ],
)

You can specify the exclusion using either the maven.exclusion helper or the group-id:artifact-id string directly.

You can also exclude artifacts globally using the excluded_artifacts attribute in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    excluded_artifacts = [
        "com.google.guava:guava",
    ],
)

Compile-only dependencies

If you want to mark certain artifacts as compile-only dependencies, use the neverlink attribute in the maven.artifact helper:

load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact("com.squareup", "javapoet", "1.11.0", neverlink = True),
    ],
    # ...
)

This instructs rules_jvm_external to mark the generated target for com.squareup:javapoet with the neverlink = True attribute, making the artifact available only for compilation and not at runtime.

Test-only dependencies

If you want to mark certain artifacts as test-only dependencies, use the testonly attribute in the maven.artifact helper:

load("@rules_jvm_external//:specs.bzl", "maven")

maven_install(
    artifacts = [
        maven.artifact("junit", "junit", "4.13", testonly = True),
    ],
    # ...
)

This instructs rules_jvm_external to mark the generated target for junit:Junit with the testonly = True attribute, making the artifact available only for tests (e.g. java_test), or targets specifically marked as testonly = True.

Resolving user-specified and transitive dependency version conflicts

Use the version_conflict_policy attribute to decide how to resolve conflicts between artifact versions specified in your maven_install rule and those implicitly picked up as transitive dependencies.

The attribute value can be either default or pinned.

default: use Coursier's default algorithm for version handling.

pinned: pin the versions of the artifacts that are explicitly specified in maven_install.

For example, pulling in guava transitively via google-cloud-storage resolves to guava-26.0-android.

maven_install(
    name = "pinning",
    artifacts = [
        "com.google.cloud:google-cloud-storage:1.66.0",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ]
)
$ bazel query @pinning//:all | grep guava_guava
@pinning//:com_google_guava_guava
@pinning//:com_google_guava_guava_26_0_android

Pulling in guava-27.0-android directly works as expected.

maven_install(
    name = "pinning",
    artifacts = [
        "com.google.cloud:google-cloud-storage:1.66.0",
        "com.google.guava:guava:27.0-android",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ]
)
$ bazel query @pinning//:all | grep guava_guava
@pinning//:com_google_guava_guava
@pinning//:com_google_guava_guava_27_0_android

Pulling in guava-25.0-android (a lower version), resolves to guava-26.0-android. This is the default version conflict policy in action, where artifacts are resolved to the highest version.

maven_install(
    name = "pinning",
    artifacts = [
        "com.google.cloud:google-cloud-storage:1.66.0",
        "com.google.guava:guava:25.0-android",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ]
)
$ bazel query @pinning//:all | grep guava_guava
@pinning//:com_google_guava_guava
@pinning//:com_google_guava_guava_26_0_android

Now, if we add version_conflict_policy = "pinned", we should see guava-25.0-android getting pulled instead. The rest of non-specified artifacts still resolve to the highest version in the case of version conflicts.

maven_install(
    name = "pinning",
    artifacts = [
        "com.google.cloud:google-cloud-storage:1.66.0",
        "com.google.guava:guava:25.0-android",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ]
    version_conflict_policy = "pinned",
)
$ bazel query @pinning//:all | grep guava_guava
@pinning//:com_google_guava_guava
@pinning//:com_google_guava_guava_25_0_android

There may be cases where you want the default pinning strategy, but want one specific dependency to be pinned, no matter what. In these cases, you can use the force_version attribute on the maven.artifact helper to ensure this happens.

maven_install(
    name = "forcing_versions",
    artifacts = [
        # Specify an ancient version of guava, and force its use. If we try to use `[23.3-jre]` as the version,
        # the resolution will fail when using `coursier`
        maven.artifact(
            artifact = "guava",
            force_version = True,
            group = "com.google.guava",
            version = "23.3-jre",
        ),
        # And something that depends on a more recent version of guava
        "xyz.rogfam:littleproxy:2.1.0",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

In this case, once pinning is complete, guava 23.3-jre will be selected.

Overriding generated targets

You can override the generated targets for artifacts with a target label of your choice. For instance, if you want to provide your own definition of @maven//:com_google_guava_guava at //third_party/guava:guava, specify the mapping in the override_targets attribute:

maven_install(
    name = "pinning",
    artifacts = [
        "com.google.guava:guava:27.0-jre",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
    override_targets = {
        "com.google.guava:guava": "@//third_party/guava:guava",
    },
)

Note that the target label contains @//, which tells Bazel to reference the target relative to your main workspace, instead of the @maven workspace.

The dependency that has been overridden is made available prefixed with original_. That is, in the example above, the version of Guava that was resolved could be accessed as @maven//:original_com_google_guava_guava. The primary use case this is designed to support is to allow specific targets to have additional dependencies added (eg. to ensure a default implementation of key interfaces are available on the classpath without needing to modify every target)

Proxies

As with other Bazel repository rules, the standard http_proxy, https_proxy and no_proxy environment variables (and their uppercase counterparts) are supported.

Repository aliases

Maven artifact rules like maven_jar and jvm_import_external generate targets labels in the form of @group_artifact//jar, like @com_google_guava_guava//jar. This is different from the @maven//:group_artifact naming style used in this project.

As some Bazel projects depend on the @group_artifact//jar style labels, we provide a generate_compat_repositories attribute in maven_install. If enabled, JAR artifacts can also be referenced using the @group_artifact//jar target label. For example, @maven//:com_google_guava_guava can also be referenced using @com_google_guava_guava//jar.

The artifacts can also be referenced using the style used by java_import_external as @group_artifact//:group_artifact or @group_artifact for short.

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    generate_compat_repositories = True
)

load("@maven//:compat.bzl", "compat_repositories")
compat_repositories()

Repository remapping

If the maven_jar or jvm_import_external is not named according to rules_jvm_external's conventions, you can apply repository remapping from the expected name to the new name for compatibility.

For example, if an external dependency uses @guava//jar, and rules_jvm_external generates @com_google_guava_guava//jar, apply the repo_mapping attribute to the external repository WORKSPACE rule, like http_archive in this example:

http_archive(
    name = "my_dep",
    repo_mapping = {
        "@guava": "@com_google_guava_guava",
    }
    # ...
)

With repo_mapping, all references to @guava//jar in @my_dep's BUILD files will be mapped to @com_google_guava_guava//jar instead.

Hiding transitive dependencies

As a convenience, transitive dependencies are visible to your build rules. However, this can lead to surprises when updating maven_install's artifacts list, since doing so may eliminate transitive dependencies from the build graph. To force rule authors to explicitly declare all directly referenced artifacts, use the strict_visibility attribute in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    strict_visibility = True
)

It is also possible to change strict visibility value from default //visibility:private to a value specified by strict_visibility_value attribute.

Accessing transitive dependencies list

It is possible to retrieve full list of dependencies in the dependency tree, including transitive, source, javadoc and other artifacts. maven_artifacts list contains full versioned maven coordinate strings of all dependencies.

For example:

load("@maven//:defs.bzl", "maven_artifacts")

load("@rules_jvm_external//:defs.bzl", "artifact")
load("@rules_jvm_external//:specs.bzl", "parse")

all_jar_coordinates = [c for c in maven_artifacts if parse.parse_maven_coordinate(c).get("packaging", "jar") == "jar"]
all_jar_targets = [artifact(c) for c in all_jar_coordinates]

java_library(
  name = "depends_on_everything",
  runtime_deps = all_jar_targets,
)

Fetch and resolve timeout

The default timeout to fetch and resolve artifacts is 600 seconds. If you need to change this to resolve a large number of artifacts you can set the resolve_timeout attribute in maven_install:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    resolve_timeout = 900
)

Ignoring empty jars

By default, if any fetched jar is empty (has 0 bytes) the corresponding artifact will still be included in the dependency tree.

If you would like to avoid such artifacts, and treat jars that are empty (i.e. their checksum equals the checksum of an empty file) as if they were not found, you can set the ignore_empty_files attribute in maven_install to remove such artifacts from coursier's output:

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    # ...
    ignore_empty_files = True
)

This option may be useful if you see empty source jars when fetch_sources is enabled.

Duplicate artifact warning

By default you will be warned if there are duplicate artifacts in your artifact list. The duplicate_version_warning setting can be used to change this behavior. Use "none" to disable the warning and "error" to fail the build instead of warn.

maven_install(
    artifacts = [
        # ...
    ],
    repositories = [
        # ...
    ],
    duplicate_version_warning = "error"
)

Provide JVM options for Coursier with COURSIER_OPTS

You can set up COURSIER_OPTS environment variable to provide some additional JVM options for Coursier. This is a space-separated list of options.

Assume you'd like to override Coursier's memory settings:

COURSIER_OPTS="-Xms1g -Xmx4g"

Resolving issues with nonstandard system default JDKs

Try to use OpenJDK explicitly if your machine or environment is set up to use a non-standard default implementation of the JDK and you encounter errors similar to the following:

java.lang.NullPointerException
	at java.base/jdk.internal.reflect.UnsafeFieldAccessorImpl.ensureObj(UnsafeFieldAccessorImpl.java:58)
	at java.base/jdk.internal.reflect.UnsafeObjectFieldAccessorImpl.get(UnsafeObjectFieldAccessorImpl.java:36)
	at java.base/java.lang.reflect.Field.get(Field.java:418)
	at org.robolectric.shadows.ShadowActivityThread$_ActivityThread_$$Reflector0.getActivities(Unknown Source)
	at org.robolectric.shadows.ShadowActivityThread.reset(ShadowActivityThread.java:277)
	at org.robolectric.Shadows.reset(Shadows.java:2499)
	at org.robolectric.android.internal.AndroidTestEnvironment.resetState(AndroidTestEnvironment.java:640)
	at org.robolectric.RobolectricTestRunner.lambda$finallyAfterTest$0(RobolectricTestRunner.java:361)
	at org.robolectric.util.PerfStatsCollector.measure(PerfStatsCollector.java:86)
	at org.robolectric.RobolectricTestRunner.finallyAfterTest(RobolectricTestRunner.java:359)
	at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$2(SandboxTestRunner.java:296)
	at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:99)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:830)

or

java.lang.UnsatisfiedLinkError: libstdc++.so.6: cannot open shared object file: No such file or directory
	at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
	at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2444)
	at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2500)
	at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2716)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2629)
	at java.base/java.lang.Runtime.load0(Runtime.java:769)
	at java.base/java.lang.System.load(System.java:1840)
	at org.conscrypt.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:52)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  ...

Exporting and consuming artifacts from external repositories

If you're writing a library that has dependencies, you should define a constant that lists all of the artifacts that your library requires. For example:

# my_library/BUILD
# Public interface of the library
java_library(
  name = "my_interface",
  deps = [
    "@maven//:junit_junit",
    "@maven//:com_google_inject_guice",
  ],
)
# my_library/library_deps.bzl
# All artifacts required by the library
MY_LIBRARY_ARTIFACTS = [
  "junit:junit:4.12",
  "com.google.inject:guice:4.0",
]

Users of your library can then load the constant in their WORKSPACE and add the artifacts to their maven_install. For example:

# user_project/WORKSPACE
load("@my_library//:library_deps.bzl", "MY_LIBRARY_ARTIFACTS")

maven_install(
  artifacts = [
        "junit:junit:4.11",
        "com.google.guava:guava:26.0-jre",
  ] + MY_LIBRARY_ARTIFACTS,
)
# user_project/BUILD
java_library(
  name = "user_lib",
  deps = [
    "@my_library//:my_interface",
    "@maven//:junit_junit",
  ],
)

Any version conflicts or duplicate artifacts will resolved automatically.

Publishing to External Repositories

In order to publish an artifact from your repo to a maven repository, you must first create a java_export target. This is similar to a regular java_library, but allows two additional parameters: the maven coordinates and an optional template file to use for the pom.xml file.

# user_project/BUILD
load("@rules_jvm_external//:defs.bzl", "java_export")

java_export(
  name = "exported_lib",
  maven_coordinates = "com.example:project:0.0.1",
  pom_template = "pom.tmpl",  # You can omit this
  srcs = glob(["*.java"]),
  deps = [
    "//user_project/utils",
    "@maven//:com_google_guava_guava",
  ],
)

If you wish to publish an artifact with Kotlin source code to a maven repository you can use kt_jvm_export. This rule has the same arguments and generated rules as java_export, but uses kt_jvm_library instead of java_library.

# user_project/BUILD
load("@rules_jvm_external//:kt_defs.bzl", "kt_jvm_export")

kt_jvm_export(
  name = "exported_kt_lib",
  maven_coordinates = "com.example:project:0.0.1",
  srcs = glob(["*.kt"]),
)

In order to publish the artifact, use bazel run:

bazel run --define "maven_repo=file://$HOME/.m2/repository" //user_project:exported_lib.publish

Or, to publish to (eg) Sonatype's OSS repo:

MAVEN_USER=example_user MAVEN_PASSWORD=hunter2 bazel run --stamp \
  --define "maven_repo=https://oss.sonatype.org/service/local/staging/deploy/maven2" \
  --define gpg_sign=true \
  //user_project:exported_lib.publish`

Or, to publish to a Google Cloud Storage:

bazel run --define "maven_repo=gs://example-bucket/repository" //user_project:exported_lib.publish

Or, to publish to an Amazon S3 bucket:

bazel run --define "maven_repo=s3://example-bucket/repository" //user_project:exported_lib.publish

Or, to publish to a GCP Artifact Registry:

bazel run --define "maven_repo=artifactregistry://us-west1-maven.pkg.dev/project/repository" //user_project:exported_lib.publish

When using the gpg_sign option, the current default key will be used for signing, and the gpg binary needs to be installed on the machine.

Configuring the dependency resolver

rules_jvm_external supports different mechanisms for dependency resolution. These can be selected using the resolver attribute of maven_install. The default resolver is one backed by coursier.

Common options

All resolvers understand the following environment variables:

Environment variable Meaning
RJE_VERBOSE When set to 1 extra diagnostic logging will be sent to stderr

Configuring Coursier

The default resolver is backed by coursier, which is used in tools such as sbt. It supports being used without a lock file, but cannot handle resolutions which require Maven BOMs to be used. When using the coursier-backed resolver, the following environment variables are honoured:

Environment variable Meaning
COURSIER_CREDENTIALS Documented here on the coursier site. If set to an absolute path, this will be used for configuring the credentials

Configuring Maven

A Maven-backed resolver can be used by using setting the resolver attribute of maven_install to maven. This resolver requires the use of a lock file. For bootstrapping purposes, this file may simply be an empty file. When using the maven-backed resolver, the following environment variables are honoured:

Environment variable Meaning
RJE_ASSUME_PRESENT Prevents the resolver from checking remote repositories to see if a dependency is present, and just assumes it is
RJE_MAX_THREADS Integer giving the maximum number of threads to use
for downloads. The default value is whichever is lower: the number of processors on the machine, or 5
RJE_UNSAFE_CACHE When set to 1 will use your $HOME/.m2/repository directory to speed up dependency resolution

Using the unsafe cache option will use your local $HOME/.m2/repository as a source for dependency resolutions, but will not include any local paths in the generated lock file unless the repositories attribute contains m2local.

The Maven-backed resolver will use credentials stored in a $HOME/.netrc file when performing dependency resolution

IPv6 support

Certain IPv4/IPv6 dual-stack environments may require flags to override the default settings for downloading dependencies, for both Bazel's native downloader and Coursier as a downloader:

Add:

  • startup --host_jvm_args=-Djava.net.preferIPv6Addresses=true to your .bazelrc file for Bazel's native downloader.
  • -Djava.net.preferIPv6Addresses=true to the COURSIER_OPTS` environment variable to provide JVM options for Coursier.

For more information, read the official docs for IPv6 support in Bazel.

Developing this project

Verbose / debug mode

Set the RJE_VERBOSE environment variable to true to print coursier's verbose output. For example:

$ RJE_VERBOSE=true bazel run @maven//:pin

Tests

In order to run tests, your system must have an Android SDK installed. You can install the Android SDK using Android Studio, or through most system package managers.

$ bazel test //...

Installing the Android SDK on macOS

The instructions for installing the Android SDK on macOS can be hard to find, but if you're comfortable using HomeBrew, the following steps will install what you need and set up the ANDROID_HOME environment variable that's required in order to run rules_jvm_external's own tests.

brew install android-commandlinetools
export ANDROID_HOME="$(brew --prefix)/share/android-commandlinetools"
sdkmanager "build-tools;33.0.1" "cmdline-tools;latest" "ndk;21.4.7075529" "platform-tools" "platforms;android-33"
export ANDROID_NDK_HOME="$ANDROID_HOME/ndk/21.4.7075529"

You can add the export ANDROID_HOME to your .zshrc or similar config file.

Generating documentation

Use Stardoc to generate API documentation in the docs directory using generate_docs.sh.

rules_jvm_external's People

Contributors

ahumesky avatar aj-michael avatar alexeagle avatar aranguyen avatar artem-zinnatullin avatar bencodes avatar blorente avatar borkaehw avatar cheister avatar comius avatar dhalperi avatar dmivankov avatar fmeum avatar gergelyfabian avatar gibfahn avatar illicitonion avatar jin avatar justhecuke-zz avatar lukaciko avatar mandrean avatar meteorcloudy avatar philwo avatar sacsar avatar shs96c avatar sitaktif avatar thirtyseven avatar tsawada avatar utzcoz avatar vinnybod avatar yrom avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rules_jvm_external's Issues

Examples breake with Python toolchains (Bazel 0.27)

Same as #157, but for the "rules_jvm_external - examples" test.

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1017#c6a01850-dd28-4084-a6e7-23bbf367bd0d

ERROR: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/8ab3d639e2adad62d28400e780405787/external/android_test_support/tools/android/emulated_devices/generic_phone/BUILD.bazel:43:1: Couldn't build file external/android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2_images/emulator-meta-data.pb: Creating Android image for @android_test_support//tools/android/emulated_devices/generic_phone:android_23_x86_qemu2 failed (Exit 1) unified_launcher_head failed: error executing command
--
  | (cd /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/8ab3d639e2adad62d28400e780405787/sandbox/linux-sandbox/324/execroot/__main__ && \
  | exec env - \
  | bazel-out/host/bin/external/android_test_support/tools/android/emulator/unified_launcher_head '--action=boot' '--density=240' '--memory=2048' '--cache=32' '--vm_size=256' '--generate_output_dir=bazel-out/host/bin/external/android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2_images' '--skin=480x800' '--source_properties_file=external/androidsdk/system-images/android-23/default/x86/source.properties' '--system_images=external/androidsdk/system-images/android-23/default/x86/NOTICE.txt external/androidsdk/system-images/android-23/default/x86/advancedFeatures.ini external/androidsdk/system-images/android-23/default/x86/build.prop external/androidsdk/system-images/android-23/default/x86/kernel-qemu external/androidsdk/system-images/android-23/default/x86/kernel-ranchu external/androidsdk/system-images/android-23/default/x86/package.xml external/androidsdk/system-images/android-23/default/x86/ramdisk.img external/androidsdk/system-images/android-23/default/x86/system.img external/androidsdk/system-images/android-23/default/x86/userdata.img external/androidsdk/emulator/emulator external/androidsdk/emulator/lib/advancedFeatures.ini external/androidsdk/emulator/lib/advancedFeaturesCanary.ini external/androidsdk/emulator/lib/ca-bundle.pem external/androidsdk/emulator/lib/emu-original-feature-flags.protobuf external/androidsdk/emulator/lib/emulator_controller.proto external/androidsdk/emulator/lib/hardware-properties.ini external/androidsdk/emulator/lib/pc-bios/bios-256k.bin external/androidsdk/emulator/lib/pc-bios/bios.bin external/androidsdk/emulator/lib/pc-bios/efi-e1000.rom external/androidsdk/emulator/lib/pc-bios/efi-e1000e.rom external/androidsdk/emulator/lib/pc-bios/efi-virtio.rom external/androidsdk/emulator/lib/pc-bios/keymaps/Makefile external/androidsdk/emulator/lib/pc-bios/keymaps/ar external/androidsdk/emulator/lib/pc-bios/keymaps/bepo external/androidsdk/emulator/lib/pc-bios/keymaps/common external/androidsdk/emulator/lib/pc-bios/keymaps/cz external/androidsdk/emulator/lib/pc-bios/keymaps/da external/androidsdk/emulator/lib/pc-bios/keymaps/de external/androidsdk/emulator/lib/pc-bios/keymaps/de-ch external/androidsdk/emulator/lib/pc-bios/keymaps/en-gb external/androidsdk/emulator/lib/pc-bios/keymaps/en-us external/androidsdk/emulator/lib/pc-bios/keymaps/es external/androidsdk/emulator/lib/pc-bios/keymaps/et external/androidsdk/emulator/lib/pc-bios/keymaps/fi external/androidsdk/emulator/lib/pc-bios/keymaps/fo external/androidsdk/emulator/lib/pc-bios/keymaps/fr external/androidsdk/emulator/lib/pc-bios/keymaps/fr-be external/androidsdk/emulator/lib/pc-bios/keymaps/fr-ca external/androidsdk/emulator/lib/pc-bios/keymaps/fr-ch external/androidsdk/emulator/lib/pc-bios/keymaps/hr external/androidsdk/emulator/lib/pc-bios/keymaps/hu external/androidsdk/emulator/lib/pc-bios/keymaps/is external/androidsdk/emulator/lib/pc-bios/keymaps/it external/androidsdk/emulator/lib/pc-bios/keymaps/ja external/androidsdk/emulator/lib/pc-bios/keymaps/lt external/androidsdk/emulator/lib/pc-bios/keymaps/lv external/androidsdk/emulator/lib/pc-bios/keymaps/mk external/androidsdk/emulator/lib/pc-bios/keymaps/modifiers external/androidsdk/emulator/lib/pc-bios/keymaps/nl external/androidsdk/emulator/lib/pc-bios/keymaps/nl-be external/androidsdk/emulator/lib/pc-bios/keymaps/no external/androidsdk/emulator/lib/pc-bios/keymaps/pl external/androidsdk/emulator/lib/pc-bios/keymaps/pt external/androidsdk/emulator/lib/pc-bios/keymaps/pt-br external/androidsdk/emulator/lib/pc-bios/keymaps/ru external/androidsdk/emulator/lib/pc-bios/keymaps/sl external/androidsdk/emulator/lib/pc-bios/keymaps/sv external/androidsdk/emulator/lib/pc-bios/keymaps/th external/androidsdk/emulator/lib/pc-bios/keymaps/tr external/androidsdk/emulator/lib/pc-bios/kvmvapic.bin external/androidsdk/emulator/lib/pc-bios/linuxboot.bin external/androidsdk/emulator/lib/pc-bios/linuxboot_dma.bin external/androidsdk/emulator/lib/pc-bios/multiboot.bin external/androidsdk/emulator/lib/pc-bios/vgabios-cirrus.bin external/androidsdk/emulator/lib/pc-bios/vgabios-stdvga.bin external/androidsdk/emulator/lib/pc-bios/vgabios-virtio.bin external/androidsdk/emulator/lib64/gles_mesa/libGL.so external/androidsdk/emulator/lib64/gles_mesa/libGL.so.1 external/androidsdk/emulator/lib64/gles_swiftshader/libEGL.so external/androidsdk/emulator/lib64/gles_swiftshader/libGLES_CM.so external/androidsdk/emulator/lib64/gles_swiftshader/libGLESv2.so external/androidsdk/emulator/lib64/libEGL_translator.so external/androidsdk/emulator/lib64/libGLES12Translator.so external/androidsdk/emulator/lib64/libGLES_CM_translator.so external/androidsdk/emulator/lib64/libGLES_V2_translator.so external/androidsdk/emulator/lib64/libOpenglRender.so external/androidsdk/emulator/lib64/libc++.so external/androidsdk/emulator/lib64/libc++.so.1 external/androidsdk/emulator/lib64/libemugl_common.so external/androidsdk/emulator/lib64/libtcmalloc_minimal.so.4 external/androidsdk/emulator/lib64/libunwind-x86_64.so.8 external/androidsdk/emulator/lib64/libunwind.so.8 external/androidsdk/emulator/lib64/qt/lib/libQt5CoreAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5DBusAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5GuiAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5NetworkAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5PrintSupportAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5SvgAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5WidgetsAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5XcbQpaAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libfreetype.so.6 external/androidsdk/emulator/lib64/qt/lib/libsoftokn3.so external/androidsdk/emulator/lib64/qt/lib/libsqlite3.so external/androidsdk/emulator/lib64/qt/lib/libxkbcommon.so external/androidsdk/emulator/lib64/qt/lib/libxkbcommon.so.0 external/androidsdk/emulator/lib64/qt/lib/libxkbcommon.so.0.0.0 external/androidsdk/emulator/lib64/qt/plugins/iconengines/libqsvgicon.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqgif.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqicns.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqico.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqjpeg.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqsvg.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqtga.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqtiff.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqwbmp.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqwebp.so external/androidsdk/emulator/lib64/qt/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so external/androidsdk/emulator/lib64/qt/plugins/platforms/libqxcb.so external/androidsdk/emulator/lib64/vulkan/glslangValidator external/androidsdk/emulator/lib64/vulkan/libvk_swiftshader.so external/androidsdk/emulator/lib64/vulkan/libvulkan.so external/androidsdk/emulator/lib64/vulkan/libvulkan.so.1 external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Snorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Snorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Unorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Unorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Snorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Snorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Unorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Unorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGB8_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGB8_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGBA8_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGBA8_3D.spv external/androidsdk/emulator/lib64/vulkan/vk_swiftshader_icd.json external/androidsdk/emulator/qemu/linux-x86_64/qemu-system-i386' --flag_configured_android_tools '--adb=external/androidsdk/platform-tools/adb' '--emulator_x86=external/androidsdk/emulator/emulator' '--emulator_arm=external/androidsdk/emulator/emulator' '--adb_static=external/bazel_tools/tools/android/fail.sh' '--mksdcard=external/androidsdk/emulator/mksdcard' '--empty_snapshot_fs=bazel-out/host/bin/external/bazel_tools/tools/android/emulator/snapshots.img' '--bios_files=external/androidsdk/emulator/lib/pc-bios/bios-256k.bin,external/androidsdk/emulator/lib/pc-bios/bios.bin,external/androidsdk/emulator/lib/pc-bios/efi-e1000.rom,external/androidsdk/emulator/lib/pc-bios/efi-e1000e.rom,external/androidsdk/emulator/lib/pc-bios/efi-virtio.rom,external/androidsdk/emulator/lib/pc-bios/kvmvapic.bin,external/androidsdk/emulator/lib/pc-bios/linuxboot.bin,external/androidsdk/emulator/lib/pc-bios/linuxboot_dma.bin,external/androidsdk/emulator/lib/pc-bios/multiboot.bin,external/androidsdk/emulator/lib/pc-bios/vgabios-cirrus.bin,external/androidsdk/emulator/lib/pc-bios/vgabios-stdvga.bin,external/androidsdk/emulator/lib/pc-bios/vgabios-virtio.bin' --nocopy_system_images --single_image_file '--android_sdk_path=external/androidsdk' '--platform_apks=' '--default_properties_file=bazel-out/host/bin/external/android_test_support/tools/android/emulated_devices/generic_phone/_android_23_x86_qemu2_props.properties')
  | Execution platform: @bazel_tools//platforms:host_platform
  |  
  | Use --sandbox_debug to see verbose messages from the sandbox: unified_launcher_head failed: error executing command
  | (cd /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/8ab3d639e2adad62d28400e780405787/sandbox/linux-sandbox/324/execroot/__main__ && \
  | exec env - \
  | bazel-out/host/bin/external/android_test_support/tools/android/emulator/unified_launcher_head '--action=boot' '--density=240' '--memory=2048' '--cache=32' '--vm_size=256' '--generate_output_dir=bazel-out/host/bin/external/android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2_images' '--skin=480x800' '--source_properties_file=external/androidsdk/system-images/android-23/default/x86/source.properties' '--system_images=external/androidsdk/system-images/android-23/default/x86/NOTICE.txt external/androidsdk/system-images/android-23/default/x86/advancedFeatures.ini external/androidsdk/system-images/android-23/default/x86/build.prop external/androidsdk/system-images/android-23/default/x86/kernel-qemu external/androidsdk/system-images/android-23/default/x86/kernel-ranchu external/androidsdk/system-images/android-23/default/x86/package.xml external/androidsdk/system-images/android-23/default/x86/ramdisk.img external/androidsdk/system-images/android-23/default/x86/system.img external/androidsdk/system-images/android-23/default/x86/userdata.img external/androidsdk/emulator/emulator external/androidsdk/emulator/lib/advancedFeatures.ini external/androidsdk/emulator/lib/advancedFeaturesCanary.ini external/androidsdk/emulator/lib/ca-bundle.pem external/androidsdk/emulator/lib/emu-original-feature-flags.protobuf external/androidsdk/emulator/lib/emulator_controller.proto external/androidsdk/emulator/lib/hardware-properties.ini external/androidsdk/emulator/lib/pc-bios/bios-256k.bin external/androidsdk/emulator/lib/pc-bios/bios.bin external/androidsdk/emulator/lib/pc-bios/efi-e1000.rom external/androidsdk/emulator/lib/pc-bios/efi-e1000e.rom external/androidsdk/emulator/lib/pc-bios/efi-virtio.rom external/androidsdk/emulator/lib/pc-bios/keymaps/Makefile external/androidsdk/emulator/lib/pc-bios/keymaps/ar external/androidsdk/emulator/lib/pc-bios/keymaps/bepo external/androidsdk/emulator/lib/pc-bios/keymaps/common external/androidsdk/emulator/lib/pc-bios/keymaps/cz external/androidsdk/emulator/lib/pc-bios/keymaps/da external/androidsdk/emulator/lib/pc-bios/keymaps/de external/androidsdk/emulator/lib/pc-bios/keymaps/de-ch external/androidsdk/emulator/lib/pc-bios/keymaps/en-gb external/androidsdk/emulator/lib/pc-bios/keymaps/en-us external/androidsdk/emulator/lib/pc-bios/keymaps/es external/androidsdk/emulator/lib/pc-bios/keymaps/et external/androidsdk/emulator/lib/pc-bios/keymaps/fi external/androidsdk/emulator/lib/pc-bios/keymaps/fo external/androidsdk/emulator/lib/pc-bios/keymaps/fr external/androidsdk/emulator/lib/pc-bios/keymaps/fr-be external/androidsdk/emulator/lib/pc-bios/keymaps/fr-ca external/androidsdk/emulator/lib/pc-bios/keymaps/fr-ch external/androidsdk/emulator/lib/pc-bios/keymaps/hr external/androidsdk/emulator/lib/pc-bios/keymaps/hu external/androidsdk/emulator/lib/pc-bios/keymaps/is external/androidsdk/emulator/lib/pc-bios/keymaps/it external/androidsdk/emulator/lib/pc-bios/keymaps/ja external/androidsdk/emulator/lib/pc-bios/keymaps/lt external/androidsdk/emulator/lib/pc-bios/keymaps/lv external/androidsdk/emulator/lib/pc-bios/keymaps/mk external/androidsdk/emulator/lib/pc-bios/keymaps/modifiers external/androidsdk/emulator/lib/pc-bios/keymaps/nl external/androidsdk/emulator/lib/pc-bios/keymaps/nl-be external/androidsdk/emulator/lib/pc-bios/keymaps/no external/androidsdk/emulator/lib/pc-bios/keymaps/pl external/androidsdk/emulator/lib/pc-bios/keymaps/pt external/androidsdk/emulator/lib/pc-bios/keymaps/pt-br external/androidsdk/emulator/lib/pc-bios/keymaps/ru external/androidsdk/emulator/lib/pc-bios/keymaps/sl external/androidsdk/emulator/lib/pc-bios/keymaps/sv external/androidsdk/emulator/lib/pc-bios/keymaps/th external/androidsdk/emulator/lib/pc-bios/keymaps/tr external/androidsdk/emulator/lib/pc-bios/kvmvapic.bin external/androidsdk/emulator/lib/pc-bios/linuxboot.bin external/androidsdk/emulator/lib/pc-bios/linuxboot_dma.bin external/androidsdk/emulator/lib/pc-bios/multiboot.bin external/androidsdk/emulator/lib/pc-bios/vgabios-cirrus.bin external/androidsdk/emulator/lib/pc-bios/vgabios-stdvga.bin external/androidsdk/emulator/lib/pc-bios/vgabios-virtio.bin external/androidsdk/emulator/lib64/gles_mesa/libGL.so external/androidsdk/emulator/lib64/gles_mesa/libGL.so.1 external/androidsdk/emulator/lib64/gles_swiftshader/libEGL.so external/androidsdk/emulator/lib64/gles_swiftshader/libGLES_CM.so external/androidsdk/emulator/lib64/gles_swiftshader/libGLESv2.so external/androidsdk/emulator/lib64/libEGL_translator.so external/androidsdk/emulator/lib64/libGLES12Translator.so external/androidsdk/emulator/lib64/libGLES_CM_translator.so external/androidsdk/emulator/lib64/libGLES_V2_translator.so external/androidsdk/emulator/lib64/libOpenglRender.so external/androidsdk/emulator/lib64/libc++.so external/androidsdk/emulator/lib64/libc++.so.1 external/androidsdk/emulator/lib64/libemugl_common.so external/androidsdk/emulator/lib64/libtcmalloc_minimal.so.4 external/androidsdk/emulator/lib64/libunwind-x86_64.so.8 external/androidsdk/emulator/lib64/libunwind.so.8 external/androidsdk/emulator/lib64/qt/lib/libQt5CoreAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5DBusAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5GuiAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5NetworkAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5PrintSupportAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5SvgAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5WidgetsAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libQt5XcbQpaAndroidEmu.so.5 external/androidsdk/emulator/lib64/qt/lib/libfreetype.so.6 external/androidsdk/emulator/lib64/qt/lib/libsoftokn3.so external/androidsdk/emulator/lib64/qt/lib/libsqlite3.so external/androidsdk/emulator/lib64/qt/lib/libxkbcommon.so external/androidsdk/emulator/lib64/qt/lib/libxkbcommon.so.0 external/androidsdk/emulator/lib64/qt/lib/libxkbcommon.so.0.0.0 external/androidsdk/emulator/lib64/qt/plugins/iconengines/libqsvgicon.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqgif.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqicns.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqico.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqjpeg.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqsvg.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqtga.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqtiff.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqwbmp.so external/androidsdk/emulator/lib64/qt/plugins/imageformats/libqwebp.so external/androidsdk/emulator/lib64/qt/plugins/platforminputcontexts/libcomposeplatforminputcontextplugin.so external/androidsdk/emulator/lib64/qt/plugins/platforms/libqxcb.so external/androidsdk/emulator/lib64/vulkan/glslangValidator external/androidsdk/emulator/lib64/vulkan/libvk_swiftshader.so external/androidsdk/emulator/lib64/vulkan/libvulkan.so external/androidsdk/emulator/lib64/vulkan/libvulkan.so.1 external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Snorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Snorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Unorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacR11Unorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Snorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Snorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Unorm_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/EacRG11Unorm_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGB8_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGB8_3D.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGBA8_2DArray.spv external/androidsdk/emulator/lib64/vulkan/shaders/Etc2RGBA8_3D.spv external/androidsdk/emulator/lib64/vulkan/vk_swiftshader_icd.json external/androidsdk/emulator/qemu/linux-x86_64/qemu-system-i386' --flag_configured_android_tools '--adb=external/androidsdk/platform-tools/adb' '--emulator_x86=external/androidsdk/emulator/emulator' '--emulator_arm=external/androidsdk/emulator/emulator' '--adb_static=external/bazel_tools/tools/android/fail.sh' '--mksdcard=external/androidsdk/emulator/mksdcard' '--empty_snapshot_fs=bazel-out/host/bin/external/bazel_tools/tools/android/emulator/snapshots.img' '--bios_files=external/androidsdk/emulator/lib/pc-bios/bios-256k.bin,external/androidsdk/emulator/lib/pc-bios/bios.bin,external/androidsdk/emulator/lib/pc-bios/efi-e1000.rom,external/androidsdk/emulator/lib/pc-bios/efi-e1000e.rom,external/androidsdk/emulator/lib/pc-bios/efi-virtio.rom,external/androidsdk/emulator/lib/pc-bios/kvmvapic.bin,external/androidsdk/emulator/lib/pc-bios/linuxboot.bin,external/androidsdk/emulator/lib/pc-bios/linuxboot_dma.bin,external/androidsdk/emulator/lib/pc-bios/multiboot.bin,external/androidsdk/emulator/lib/pc-bios/vgabios-cirrus.bin,external/androidsdk/emulator/lib/pc-bios/vgabios-stdvga.bin,external/androidsdk/emulator/lib/pc-bios/vgabios-virtio.bin' --nocopy_system_images --single_image_file '--android_sdk_path=external/androidsdk' '--platform_apks=' '--default_properties_file=bazel-out/host/bin/external/android_test_support/tools/android/emulated_devices/generic_phone/_android_23_x86_qemu2_props.properties')
  | Execution platform: @bazel_tools//platforms:host_platform
  |  
  | Use --sandbox_debug to see verbose messages from the sandbox
  | File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/8ab3d639e2adad62d28400e780405787/sandbox/linux-sandbox/324/execroot/__main__/bazel-out/host/bin/external/android_test_support/tools/android/emulator/unified_launcher_head.runfiles/android_test_support/tools/android/emulator/unified_launcher.py", line 468
  | print ''
  | ^
  | SyntaxError: Missing parentheses in call to 'print'. Did you mean print('')?
  | ----------------
  | Note: The failure of target @android_test_support//tools/android/emulator:unified_launcher_head (with exit code 1) may have been caused by the fact that it is running under Python 3 instead of Python 2. Examine the error to determine if that appears to be the problem. Since this target is built in the host configuration, the only way to change its version is to set --host_force_python=PY2, which affects the entire build.
  |  
  | If this error started occurring in Bazel 0.27 and later, it may be because the Python toolchain now enforces that targets analyzed as PY2 and PY3 run under a Python 2 and Python 3 interpreter, respectively. See https://github.com/bazelbuild/bazel/issues/7899 for more information.
  | ----------------
  | Target //src/test:greeter_test failed to build

maven scopes

Would it make sense to have a scope argument on the maven.artifact function for a more direct relationship to maven's scope types? The main benefit I would see is that it provides a direct mechanism to translate between Maven and Bazel without requiring people deep-diving into Bazel's dependency graph modeling upfront.

You're probably aware but the 6 scopes are:

  • Compile (default).
  • Provided.
  • Runtime.
  • Test.
  • System.
  • Import (not relevant to Bazel directly maybe to this tool).

Test and compile feel like they would basically be nops in terms of additional behaviour.

Runtime, Provided, and System could probably set neverlink=True but might need some more thought.

Feels roughly related to #132 ignoring the maven type.

Support private repository requiring authentication with Authorization http header in a more secure manner

The current support for HTTP basic authentication relies on username:password@ URL schema.

There are few flaws with this approach:

  • Not ever repository server supports the username:password@ URL scheme. Some admins disable it intentionally for security reasons.
  • The username:password@ URL scheme requires persisting username and passwords in clear text in Bazel files, which are committed to SCM. This is almost impossible to deploy in any corporate environment.
    • Credentials should be supplied using an external configuration file and/or other form of parameters (eg., secure store, environment variables, etc.).
    • User need to be able to provide their own credentials at build time.

Bonus points:

  • Allow to provide URL overrides at build time, eg. use a local proxy/mirror instead of what is configured in the Bazel files.
  • Repository credentials are read from ~/.m2/settings.xml
  • Repository mirror urls are read from ~/.m2/settings.xml

Breaks with Python toolchains (Bazel 0.27)

Bazel 0.27 flips --incompatible_use_python_toolchains, which breaks this repo. Note that this pipeline was already failing due to a different error.

This flag causes Bazel to use the version of Python requested at analysis time, rather than whatever the system python command happens to be. You likely need to add python_version = "PY2" to your Python targets, and/or enable --force_host_python=PY2 in your bazelrc (if any of these are used in the host configuration).

The `+` operator for dicts is deprecated and no longer supported

external/gmaven_rules/import_external.bzl", line 313, in repository_rule
                {"jar_urls": attr.string_list(mandatory = True, allow_empty = False), "jar_sha256": attr.string(mandatory = True), "_packaging": attr.string(default = "jar"), "neverlink": attr.bool(), "runtime_deps": attr.string_list()} + COMMON_ATTRS
The `+` operator for dicts is deprecated and no longer supported. Please use the `update` method instead. You can temporarily enable the `+` operator by passing the flag --incompatible_disallow_dict_plus=false

Tested with commit 3f430f1.

Implement global exclusion for artifacts

Related: coursier/coursier#1122

Currently, we support artifact exclusions using the maven.artifact(exclusions = [...]) helper. This feeds the group:artifact coordinates into a --local-exclude-file for Coursier CLI.

The problem with this is that if artifact A excludes B from its transitive dependencies, and C also depends on B but doesn't exclude it, B will still be fetched into the transitive dependencies on A. This can be confusing and will lead to complex maven_install declarations since every artifact that transitively depends on B will need to exclude it.

Coursier CLI supports another flag called --exclude, which is basically a global exclude. I propose to add another top level exclusions attribute to maven_install, that simply feeds this collection of group:artifact into the CLI --exclude option.

The alternative is to take the per-artifact exclusions and feed them into --exclude, but this breaks the API affinity between maven_install and Coursier CLI, and can be even more confusing down the road.

More discussion from Bazel Slack: https://bazelbuild.slack.com/archives/CDCE4DFAP/p1555009607029700

cc @dkelmer

User defined artifacts get overridden by transitive dependencies

If a user explicitly references a dependency, we should use that version and not ones brought through transitive dependencies. Basically, prioritize user defined dependencies over anything else. For example:

maven_install(
    artifacts = [
        "com.google.cloud:google-cloud-storage:1.66.0",
        "com.google.guava:guava:27.1-jre",
    ]
)

Should result in a dependency tree that contains com.google.guava:guava:27.1-jre, and not the one brought by com.google.cloud:google-cloud-storage:1.66.0.

Consider an optional switch to turn all `deps` into `exports`, allowing access to all transitive classes just by depending on the top level artifact

This was motivated by a conversation with @simonstewart for better developer ergonomics and gradual adoption, because this is what Maven and Gradle users are used to.

The general idea is to depend on just the top level dependency, and get all the classes in the transitive classpath for free. This was previously implemented by @davidsantiago in jin/rules_maven#9, but we removed it in jin/rules_maven#16 with the concerns stated in the first comment and @dkelmer's comment in jin/rules_maven#16 (review)

We should consider the tradeoffs between ergonomics and the growth of classpath complexity. This can be an opt-in feature with major disclaimers on its implications.

Remove dependency on Maven binary

Consider using another fetching mechanism to download AARs. With Skylark maven_aar, downloads are failing randomly on the CI: bazelbuild/bazel#4979 (comment)

Since we know the exact repository where the files are downloaded from, possible candidates are:

  1. http_file
  2. a variant of java_import_external for aar_import

On top of not having maven as a dependency, these two rules use the more robust HttpDownloader underneath, which also supports multiple URLs for fallback.

Downloads aren't retried when repositories return 503

When I have a list of multiple repositories in our maven_install call, if the first URL returns a 503, it doesn't fall back to trying the second URL. Instead the build fails with this error:

ERROR: .../BUILD.bazel:5:1: no such package '@maven//': Error while fetching artifact with coursier:   Error:
https://...org/jetbrains/kotlin/kotlin-stdlib-common/1.3.0/kotlin-stdlib-common-1.3.0.jar: DownloadError(Caught java.io.IOException: Server returned HTTP response code: 503 for URL: https://...org/jetbrains/kotlin/kotlin-stdlib-common/1.3.0/kotlin-stdlib-common-1.3.0.jar (Server returned HTTP response code: 503 for URL: https://...org/jetbrains/kotlin/kotlin-stdlib-common/1.3.0/kotlin-stdlib-common-1.3.0.jar) while downloading https://...org/jetbrains/kotlin/kotlin-stdlib-common/1.3.0/kotlin-stdlib-common-1.3.0.jar)

If I comment out the URL that's currently 503ing, the build passes

POM with binary type dependencies

The POM for DynamoDB local com.amazonaws:DynamoDBLocal:1.11.477 in https://s3-us-west-2.amazonaws.com/dynamodb-local/release includes some binary dependencies:

    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>libsqlite4java-linux-amd64</artifactId>
      <version>${sqlite4java.version}</version>
      <type>so</type>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>sqlite4java-win32-x64</artifactId>
      <version>${sqlite4java.version}</version>
      <type>dll</type>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>sqlite4java-win32-x86</artifactId>
      <version>${sqlite4java.version}</version>
      <type>dll</type>
      <scope>runtime</scope>
    </dependency>
    <dependency>
      <groupId>com.almworks.sqlite4java</groupId>
      <artifactId>libsqlite4java-osx</artifactId>
      <version>${sqlite4java.version}</version>
      <type>dylib</type>
      <scope>runtime</scope>
    </dependency>

So we get:

The artifact for com.almworks.sqlite4java:libsqlite4java-linux-amd64:so:1.0.392 was not downloaded. Perhaps its packaging type is
not one of: jar,aar,bundle,eclipse-plugin?

Are there any suggestions for handling transitive dependencies of this sort?

Regenerate Rules

Hi,

I regenerated gmaven.bzl because I needed a newer library. I'm hosting the file on my own server and I changed my URL as appropriate.

When I try to run I get the following error:

ERROR: error loading package '': Encountered error while reading extension file 'gmaven.bzl': no such package '@gmaven_rules//': Prefix gmaven_rules-20180723 was given, but not found in the archive
ERROR: error loading package '': Encountered error while reading extension file 'gmaven.bzl': no such package '@gmaven_rules//': Prefix gmaven_rules-20180723 was given, but not found in the archive
INFO:

I've double checked what the original artifact looks like when using the artifact from https://github.com/bazelbuild/gmaven_rules/archive vs mine and it looks the same.

Any thoughts?

Artifact with classifier looking at wrong path

Hi,

I am trying to fetch some artifacts that have a classifier. Unfortunately, it seems as if this feature is not properly implemented.

For example:

maven.artifact(
    group = "com.digitalasset",
    artifact = "damlc",
    version = "100.12.1",
    classifier = "osx",
),
maven.artifact(
    group = "com.digitalasset",
    artifact = "damlc",
    version = "100.12.1",
    classifier = "linux",
)

Yields:

https://digitalassetsdk.bintray.com/DigitalAssetSDK/com/digitalasset/damlc/osx/damlc-osx.pom
https://digitalassetsdk.bintray.com/DigitalAssetSDK/com/digitalasset/damlc/linux/damlc-linux.pom

When according to: https://stackoverflow.com/a/47733421. It should yield:

https://digitalassetsdk.bintray.com/DigitalAssetSDK/com/digitalasset/damlc/100.12.1/damlc-100.12.1.pom
https://digitalassetsdk.bintray.com/DigitalAssetSDK/com/digitalasset/damlc/100.12.1/damlc-100.12.1-osx.jar
https://digitalassetsdk.bintray.com/DigitalAssetSDK/com/digitalasset/damlc/100.12.1/damlc-100.12.1-linux.jar

java_import invokes ijar, which clobbers inline functions in kotlin

Given bazelbuild/bazel#4549, the maven repository workspace shouldn't use java_import, but should use something similar to kt_jvm_import which does the same thing, but offers the compile_jar as the same as the regular jar. This will affect things like kotlin.test whose assertFailsWith<ExceptionType>() { throw ExceptionType() } inlines, and therefore needs the method bodies at compile-time.

Top-level repository aliases and cross-workspace dependencies

Quite a few projects in the Bazel ecosystem expect their java dependencies to be individually imported as repository rules in their own right, for example grpc-java expects to find the top-level @com_google_guava_guava//jar, which is created via the built-in rule jvm_maven_import_external (from jvm.bzl).

In my current projects, I tend to have a (generated) bunch of jvm_maven_import_external/java_import_external targets which supply both my own targets as well as third-party ones like grpc-java. That is, I don't use the grpc_java_repositories function to import those dependencies; I just ensure that I provide a compatible version in the expected place.

With rules_jvm_external the paradigm is slightly different. Resolution is now to many targets within a single repository rule, i.e. @com_google_guava_guava//jar becomes @maven//:com_google_guava_guava. This of course means that the loaded grpc-java rules can't find their dependencies when I switch to these new rules. If the grpc_java_repositories function is called, it will create a second repository rule for things which may already be available in the @maven workspace, and may cause classpath collisions downstream later.

I can see clear advantages to the new rules, particularly that different versions of the same artifact can now be resolved in different repositories (@maven//:foo_bar != @othermaven//:foo_bar). Downstream classpath collisions are clearly still possible there, so to some extent this grpc-java issue is just the natural outcome of an assumption of top-level repository safety/unity.

But, it might be useful to generate these @foo_bar top-level repository aliases for consistency with the old-style patterns. At least until there is a more obvious solution for reuse of Java deps across a full workspace including external dependency workspaces. Maybe this exists already! I'd be happy to find a solution to this problem :)

The built-in Bazel options such as shadowing and binding are only apparently functional for an entire workspace, so can't be used to generically alias individual targets in either the source or the target workspaces, i.e. it's not possible to redirect the @foo_bar//jar generated by java_import_external, nor target the @maven//:foo_bar for a single artifact.

A couple of options:

  1. Add a parameter to the maven_install rule to allow generation of an additional repositories.bzl file, containing a function which can be loaded to create new repository rules for some/all artifacts in the closure, mirroring the format generated by java_import_external.
  2. Make it easier for downstream workspaces to do the above, by exposing the generated @maven//:dep-tree.json file.

I have a hacked-up version of the first one, but it feels a bit ugly so far. Is there any plan for a migration path between java_import_external and maven_install? If not, would a PR for either option above be useful?

Breaks with Bazel HEAD

CI link: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1026#50cf7788-77e0-4c52-8baf-15f842d6d915

Source tree: b89ee49

Bazel version: built at bazelbuild/bazel@12c7942

Test log:

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //src/test:greeter_test
-----------------------------------------------------------------------------
+ '[' -z '' ']'
+ ANDROID_TESTBRIDGE_TEST_ONLY=
+ test_label=//src/test:greeter_test
+ test_entry_point=../android_test_support/instrumentation_test_runner
+ WORKSPACE_DIR=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__
+ adb=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/platform-tools/adb
+ aapt=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/aapt_binary
+ device_script=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2
+ data_deps=
++ join_paths /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__ ,
++ local base_dir=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__
++ local sep=,
++ shift 2
++ local paths=
++ local result=
++ echo
+ data_deps=
+ device_broker_type=WRAPPED_EMULATOR
+ target_apk=src/main/greeter_app.apk
+ instrumentation_apk=src/test/greeter_test_app.apk
+ support_apks=
++ join_paths /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__ ,
++ local base_dir=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__
++ local sep=,
++ shift 2
++ local paths=
++ local result=
++ echo
+ support_apks=
+ apks_to_install=src/main/greeter_app.apk,src/test/greeter_test_app.apk
+ device_script_fixtures=()
+ declare -A device_script_fixtures
+ host_service_fixture=
+ host_service_fixture_services=
+ test_suite_property_name=bazel.test_suite
++ cat
+ bazel_only_argv=--install_test_services=true
++ cat
+++ printf %s,
+++ printf %s,
+ argv='--aapt=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/aapt_binary --adb=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/platform-tools/adb --device_broker_type=WRAPPED_EMULATOR --device_script=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2 --data_deps= --test_label=//src/test:greeter_test --apks_to_install=src/main/greeter_app.apk,src/test/greeter_test_app.apk --fixture_scripts=, --hermetic_server_script= --hermetic_servers= --data_deps=, --test_filter= '
+ ../android_test_support/instrumentation_test_runner --wrapper_script_flag=--jvm_flag=-Dbazel.test_suite=com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite '--wrapper_script_flag=--jvm_flag=-Dargv=--install_test_services=true --aapt=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/aapt_binary --adb=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/platform-tools/adb --device_broker_type=WRAPPED_EMULATOR --device_script=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2 --data_deps= --test_label=//src/test:greeter_test --apks_to_install=src/main/greeter_app.apk,src/test/greeter_test_app.apk --fixture_scripts=, --hermetic_server_script= --hermetic_servers= --data_deps=, --test_filter= ' --aapt=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/aapt_binary --adb=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/platform-tools/adb --device_broker_type=WRAPPED_EMULATOR --device_script=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2 --data_deps= --test_label=//src/test:greeter_test --apks_to_install=src/main/greeter_app.apk,src/test/greeter_test_app.apk --fixture_scripts=, --hermetic_server_script= --hermetic_servers= --data_deps=, --test_filter=
JUnit4 Test Runner
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/external/android_test_support/opensource/entry_point_deploy.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Jun 07, 2019 2:04:40 AM com.google.android.apps.common.testing.broker.WrappedEmulatedDeviceBroker runEmulatorLaunchScript
INFO: Running emulator launch script: [/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2, --action=start, --adb_server_port=36811, --adb_port=34026, --emulator_port=52995, --logcat_path=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/test/greeter_test/test.outputs/device_logcat/logcat16102378533488436489.txt, --apks=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/src/main/greeter_app.apk /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/src/test/greeter_test_app.apk /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/12405704071759822584/services/test_services.apk /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/12405704071759822584/runner/android_test_orchestrator/stubapp/stubapp.apk, --system_apks=, --subprocess_log_dir=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/test/greeter_test/test.outputs/broker_logs/exec-1, --nolaunch_in_seperate_session, --start_vnc_on_port=0, --initial_locale=en-US, --initial_ime=com.android.inputmethod.latin/.LatinIME, --kvm_device=/dev/kvm, --grant_runtime_permissions=true, --logcat_filter='*:I', --enable_display, --nopreverify_apks, --noenable_console_auth, --enable_gps, --export_launch_metadata_path=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/emulator_meta14600193498143695207.pb, --net_type=fastnet, --long_press_timeout=5000]
Jun 07, 2019 2:04:54 AM com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite$Builder build
WARNING: AndroidDeviceTestSuite Native suite creation failed!
.E
Time: 0.005
There was 1 failure:
1) initializationError(com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite)
java.lang.IllegalStateException: Bad exit code: 1. Command: [/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../android_test_support/tools/android/emulated_devices/generic_phone/android_23_x86_qemu2, --action=start, --adb_server_port=36811, --adb_port=34026, --emulator_port=52995, --logcat_path=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/test/greeter_test/test.outputs/device_logcat/logcat16102378533488436489.txt, --apks=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/src/main/greeter_app.apk /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/src/test/greeter_test_app.apk /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/12405704071759822584/services/test_services.apk /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/12405704071759822584/runner/android_test_orchestrator/stubapp/stubapp.apk, --system_apks=, --subprocess_log_dir=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/test/greeter_test/test.outputs/broker_logs/exec-1, --nolaunch_in_seperate_session, --start_vnc_on_port=0, --initial_locale=en-US, --initial_ime=com.android.inputmethod.latin/.LatinIME, --kvm_device=/dev/kvm, --grant_runtime_permissions=true, --logcat_filter='*:I', --enable_display, --nopreverify_apks, --noenable_console_auth, --enable_gps, --export_launch_metadata_path=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/emulator_meta14600193498143695207.pb, --net_type=fastnet, --long_press_timeout=5000].
=====Error begin:
...
I0607 02:04:41.200695 140516606297920 emulated_device.py:568] Copying system image to /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/launcher_tmp12753000802223826896/tmpjFG2ZAandroid-emulator-launch/images/session/system.img
I0607 02:04:41.654309 140516606297920 emulated_device.py:3465] Running debugfs commands: ['unlink /vendor/lib/egl']
debugfs 1.44.1 (24-Mar-2018)
I0607 02:04:44.097393 140516606297920 emulated_device.py:717] Making sdcard on the fly due to a nonstandard size
I0607 02:04:44.734656 140516606297920 emulated_device.py:288] Emulator type: 2
I0607 02:04:44.765813 140516606297920 emulated_device.py:1790] Executing: ['/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/androidsdk/emulator/emulator', '-ports', '52995,34026', '-skin', u'480x800', '-timezone', 'America/Los_Angeles', '-cache', 'cache.img', '-memory', '2048', '-sdcard', 'sdcard.img', '-ramdisk', 'ramdisk.img', '-partition-size', '2047', '-no-snapshot-save', '-verbose', '-unix-pipe', 'sockets/qemu.mgmt', '-unix-pipe', 'sockets/device-forward-server', '-unix-pipe', 'sockets/tar-pull-server', '-unix-pipe', 'sockets/exec-server', '-unix-pipe', 'sockets/tar-push-server', '-unix-pipe', 'sockets/h2o', '-writable-system', '-show-kernel', '-engine', 'qemu2', '-kernel', 'kernel-ranchu', '-system', 'system.img', '-feature', 'AllowSnapshotMigration', '-feature', '-GLDMA', '-feature', '-OnDemandSnapshotLoad', '-data', 'userdata-qemu.img', '-gpu', 'swiftshader_indirect', '-no-audio', '-no-boot-anim', '-selinux', u'disabled', '-scale', '1.0', '-fixed-scale', '-netdelay', 'none', '-netspeed', 'full', '-avd', 'mobile_ninjas.adb.34026', '-qemu', u'-enable-kvm', '-L', '/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/launcher_tmp12753000802223826896/tmpjFG2ZAandroid-emulator-launch/bios', '-append', 'enable_test_harness=1']
I0607 02:04:44.768162 140516606297920 emulated_device.py:1824] Launching emulator in: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/launcher_tmp12753000802223826896/tmpjFG2ZAandroid-emulator-launch/images/session
I0607 02:04:44.768557 140516606297920 emulated_device.py:1830] Write emulator log to /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/testlogs/src/test/greeter_test/test.outputs/emulator_cO4CC9.log
I0607 02:04:44.769979 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 0 total attempts: 0
I0607 02:04:44.770735 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:45.772977 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 1 total attempts: 1
I0607 02:04:45.773252 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:46.774940 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 2 total attempts: 2
I0607 02:04:46.775212 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:47.776967 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 3 total attempts: 3
I0607 02:04:47.777350 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:48.779114 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 4 total attempts: 4
I0607 02:04:48.779392 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:49.781044 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 5 total attempts: 5
I0607 02:04:49.781318 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:50.782953 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 6 total attempts: 6
I0607 02:04:50.783231 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:51.784893 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 7 total attempts: 7
I0607 02:04:51.785182 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:52.786803 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 8 total attempts: 8
I0607 02:04:52.787070 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:53.788687 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 9 total attempts: 9
I0607 02:04:53.789071 140516606297920 emulated_device.py:3937] Checking if adb is listening.
I0607 02:04:54.790812 140516606297920 emulated_device.py:2401] system: False pm: False adb: False sdcard: False boot_complete: False launcher: False pipes: False current step attempts: 10 total attempts: 10
E0607 02:04:54.791110 140516606297920 emulated_device.py:3904] Emulator failed to launch: [Errno 10] No child processes
I0607 02:04:54.791465 140516606297920 emulated_device.py:3883] Emulator log below ==================================================
emulator: Android emulator version 29.0.9.0 (build_id 5537588) (CL:N/A)
emulator: Found AVD name 'mobile_ninjas.adb.34026'
emulator: Found AVD target architecture: x86
emulator: argv[0]: '/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/androidsdk/emulator/emulator'; program directory: '/opt/android-sdk-linux/emulator'
emulator:  Found directory: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/launcher_tmp12753000802223826896/tmpjFG2ZAandroid-emulator-launch/images/session

emulator: emuDirName: '/opt/android-sdk-linux/emulator'
emulator:  Found directory: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/launcher_tmp12753000802223826896/tmpjFG2ZAandroid-emulator-launch/images/session

emulator: Probing for /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/_tmp/4e4d30a0f396a30869334dc9e8417b33/launcher_tmp12753000802223826896/tmpjFG2ZAandroid-emulator-launch/images/session/kernel-ranchu-64: file missing
emulator: try dir /opt/android-sdk-linux/emulator
emulator: Trying emulator path '/opt/android-sdk-linux/emulator/qemu/linux-x86_64/qemu-system-i386'
emulator: Found target-specific 64-bit emulator binary: /opt/android-sdk-linux/emulator/qemu/linux-x86_64/qemu-system-i386
emulator: Adding library search path: '/opt/android-sdk-linux/emulator/lib64'
emulator: Adding library search path: '/opt/android-sdk-linux/emulator/lib64/gles_swiftshader'
emulator: Adding library search path: '/opt/android-sdk-linux/emulator/lib64/gles_angle'
emulator: Adding library search path: '/opt/android-sdk-linux/emulator/lib64/gles_angle9'
emulator: Adding library search path: '/opt/android-sdk-linux/emulator/lib64/gles_angle11'
emulator: Adding library search path: '/opt/android-sdk-linux/emulator/lib64/libstdc++'
emulator: Adding library search path for Qt: '/opt/android-sdk-linux/emulator/lib64/qt/lib'
emulator: Setting Qt plugin search path: QT_QPA_PLATFORM_PLUGIN_PATH=/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/__main__/../androidsdk/emulator/lib64/qt/plugins
emulator: Setting Qt to use software OpenGL: QT_OPENGL=software
emulator: Overriding pre-existing bad Qt high dpi settings...
emulator: Running :/opt/android-sdk-linux/emulator/qemu/linux-x86_64/qemu-system-i386
emulator: qemu backend: argv[00] = "/opt/android-sdk-linux/emulator/qemu/linux-x86_64/qemu-system-i386"
emulator: qemu backend: argv[01] = "-ports"
emulator: qemu backend: argv[02] = "52995,34026"
emulator: qemu backend: argv[03] = "-skin"
emulator: qemu backend: argv[04] = "480x800"
emulator: qemu backend: argv[05] = "-timezone"
emulator: qemu backend: argv[06] = "America/Los_Angeles"
emulator: qemu backend: argv[07] = "-cache"
emulator: qemu backend: argv[08] = "cache.img"
emulator: qemu backend: argv[09] = "-memory"
emulator: qemu backend: argv[10] = "2048"
emulator: qemu backend: argv[11] = "-sdcard"
emulator: qemu backend: argv[12] = "sdcard.img"
emulator: qemu backend: argv[13] = "-ramdisk"
emulator: qemu backend: argv[14] = "ramdisk.img"
emulator: qemu backend: argv[15] = "-partition-size"
emulator: qemu backend: argv[16] = "2047"
emulator: qemu backend: argv[17] = "-no-snapshot-save"
emulator: qemu backend: argv[18] = "-verbose"
emulator: qemu backend: argv[19] = "-unix-pipe"
emulator: qemu backend: argv[20] = "sockets/qemu.mgmt"
emulator: qemu backend: argv[21] = "-unix-pipe"
emulator: qemu backend: argv[22] = "sockets/device-forward-server"
emulator: qemu backend: argv[23] = "-unix-pipe"
emulator: qemu backend: argv[24] = "sockets/tar-pulemulator: autoconfig: -skindir (null)
emulator: Target arch = 'x86'
emulator: Auto-config: -qemu -cpu qemu32
emulator: Auto-detect: Kernel image requires new device naming scheme.
emulator: Auto-detect: Kernel does not support YAFFS2 partitions.
emulator: Using initial system image: system.img
emulator: No vendor image
emulator: autoconfig: -initdata (null)
emulator: WARNING: SELinux 'disabled' is no longer supported starting with API level 23, switching to 'permissive'
emulator: Physical RAM size: 2048MB

emulator: VM heap size 256MB is above maximum supported 192MB, setting it to that value
emulator: WARNING: System image is writable
emulator: Found 1 DNS servers: 169.254.169.254
emulator: [FeatureControl] Feature 'AllowSnapshotMigration' (13) state set to enabled
emulator: [FeatureControl] Feature 'GLDMA' (30) state set to disabled
emulator: [FeatureControl] Feature 'OnDemandSnapshotLoad' (21) state set to disabled
emulator: found magic skin width=480 height=800 bpp=16

emulator: ERROR: no init encryptionkey.img
emulator: ERROR: Encryption is requested but failed to create encrypt partition.
WARNING: engine selection is no longer supported, always using RACNHU.
WARNING: unexpected system image feature string, emulator might not function correctly, please try updating the emulator.
Unexpected feature list: AllowSnapshotMigration CarPropertyTable FastSnapshotV1 ForceANGLE ForceSwiftshader GLPipeChecksum GenericSnapshotsUI HAXM HVF HYPERV IgnoreHostOpenGLErrors KVM LocationUiV2 OnDemandSnapshotLoad QuickbootFileBacked ScreenRecording SnapshotAdb VideoPlayback VirtualScene Vulkan WindowsHypervisorPlatform WindowsOnDemandSnapshotLoad

I0607 02:04:54.791615 140516606297920 emulated_device.py:3884] Emulator log end ==================================================
I0607 02:04:54.791757 140516606297920 emulated_device.py:3897] cannot show log watchdog.out: file does not exist
I0607 02:04:54.791841 140516606297920 emulated_device.py:3897] cannot show log watchdog.err: file does not exist
Traceback (most recent call last):
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/unified_launcher.py", line 1129, in <module>
app.run(main)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/absl_py/absl/app.py", line 262, in run
_run_main(main, argv)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/absl_py/absl/app.py", line 227, in _run_main
sys.exit(main(argv))
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/unified_launcher.py", line 1099, in main
EntryPoint(reporter)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/unified_launcher.py", line 935, in EntryPoint
mini_boot, FLAGS.sim_access_rules_file, FLAGS.phone_number)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/unified_launcher.py", line 663, in _Run
snapshot_file=FLAGS.snapshot_file)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/unified_launcher.py", line 523, in _RestartDevice
snapshot_file=snapshot_file)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/emulated_device.py", line 1267, in StartDevice
loading_from_snapshot=loading_from_snapshot)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/emulated_device.py", line 1838, in _StartEmulator
self._PollEmulatorStatus(timer, loading_from_snapshot=loading_from_snapshot)
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/emulated_device.py", line 2402, in _PollEmulatorStatus
self._EnsureEmuRunning()
File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/f8cc5f5872dffc8c386385f64cf41a98/sandbox/linux-sandbox/339/execroot/__main__/bazel-out/k8-fastbuild/bin/src/test/greeter_test.runfiles/android_test_support/tools/android/emulator/emulated_device.py", line 3906, in _EnsureEmuRunning
raise Exception('Emulator has died')
Exception: Emulator has died
=====Error end

	at com.google.common.base.Preconditions.checkState(Preconditions.java:853)
	at com.google.android.apps.common.testing.broker.WrappedEmulatedDeviceBroker.runEmulatorLaunchScript(WrappedEmulatedDeviceBroker.java:555)
	at com.google.android.apps.common.testing.broker.WrappedEmulatedDeviceBroker.leaseDevice(WrappedEmulatedDeviceBroker.java:264)
	at com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite$Builder.makeNativeTestSuite(AndroidDeviceTestSuite.java:140)
	at com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite$Builder.build(AndroidDeviceTestSuite.java:103)
	at com.google.android.apps.common.testing.suite.AndroidDeviceTestSuite.suite(AndroidDeviceTestSuite.java:64)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.internal.runners.SuiteMethod.testFromSuiteMethod(SuiteMethod.java:35)
	at org.junit.internal.runners.SuiteMethod.<init>(SuiteMethod.java:24)
	at org.junit.internal.builders.SuiteMethodBuilder.runnerForClass(SuiteMethodBuilder.java:11)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
	at com.google.testing.junit.runner.internal.junit4.MemoizingRequest.createRunner(MemoizingRequest.java:48)
	at com.google.testing.junit.runner.internal.junit4.MemoizingRequest.getRunner(MemoizingRequest.java:34)
	at com.google.testing.junit.runner.junit4.JUnit4TestModelBuilder.get(JUnit4TestModelBuilder.java:49)
	at com.google.testing.junit.runner.junit4.JUnit4TestModelBuilder.get(JUnit4TestModelBuilder.java:27)
	at com.google.testing.junit.runner.util.MemoizingSupplier.get(MemoizingSupplier.java:42)
	at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:92)
	at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:144)
	at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:82)

FAILURES!!!
Tests run: 1,  Failures: 1


BazelTestRunner exiting with a return value of 1
JVM shutdown hooks (if any) will run now.
The JVM will exit once they complete.

-- JVM shutdown starting at 2019-06-07 02:04:54 --

Failed build on Buildkite Windows pipeline

Error while trying to read dep-tree.json: java.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(454): CreateProcessW("C:\msys64\usr\bin\cat" D:/b/wpicawde/external/maven/dep-tree.json): The system cannot find the file specified.
--
  | and referenced by '//ui/espresso/IntentsBasicSample:IntentsBasicSampleTestLib'

We'll need to replace

cat = "C:\\msys64\\usr\\bin\\cat" if (_is_windows(repository_ctx)) else repository_ctx.which("cat")
with another approach.

Related: bazelbuild/bazel#3766 and bazelbuild/bazel#7309

Make exposing annotation processors easy

Right now, all dependencies are downloaded into @maven workspace. This is great, until we need to include an annotation processor as a java_plugin: then we need to break out of the workspace and define our own rule --- it's no longer possible for someone to expect all third party dependencies to be in the same namespace, which opens the door for mistakes to be made in build files by people not aware that they need to include a different rule from a different workspace.

bazel-deps is similarly focused on "just downloading bits from maven" as rules_jvm_external but provides an processorClasses property, which exports the annotation processor as expected without impacting the UI exposed to people writing build files.

bazel_maven_repository solves the same problem less elegantly by allowing people to specify build file fragments.

The ideal solution would be a field on maven.artifact that allowed a list of annotation processor classes to listed (mirroring bazel-deps approach).

Transitive bazel dependencies have public visibility

As a reasonably new user I found it extremely surprising that a transitive dependency that was not declared in the artifacts attribute had public visibility. I would have expected that the way to make a transitive artifact public was to supply a 2-part spec in the artifacts attribute.

This would make it much easier for relative beginners to read the rules as the public targets would be explicitly declared.

Was this ever considered?

Other than that, it has been reasonably easy to get started. Thanks!

Failed to resolve deps for com.google.firebase:firebase-firestore

Steps to reproduce:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

android_sdk_repository(name = "androidsdk")

local_repository(
    name = "rules_jvm_external",
    path = "../../",
)

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [ 
        "junit:junit:4.12",
        "android.arch.lifecycle:common:1.1.1",
        "android.arch.lifecycle:viewmodel:1.1.1",
        "androidx.test.espresso:espresso-web:3.1.1",
        "com.android.support:design:27.0.2",
        "com.google.firebase:firebase-firestore:18.1.0",
    ],  
    repositories = [ 
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
    ],  
)
  • Edit BUILD file to add the artifact("com.google.firebase:firebase-firestore") line.
load("@rules_jvm_external//:defs.bzl", "artifact")

android_library(
    name = "my_lib",
    exports = [ 
        artifact("android.arch.lifecycle:common"),
        artifact("android.arch.lifecycle:viewmodel"),
        artifact("androidx.test.espresso:espresso-web"),
        artifact("junit:junit"),
        artifact("com.android.support:design"),
        artifact("com.google.firebase:firebase-firestore"),
    ],  
)

android_binary(
    name = "my_app",
    custom_package = "com.example.bazel",
    manifest = "AndroidManifest.xml",
    deps = [":my_lib"],
)
  • Run bazel build :my_app and see the following error:
ERROR: /Users/senpo/Sources/github/bazelbuild/rules_jvm_external/examples/simple/BUILD:3:1: no such package '@maven//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_senpo/d029bda8bea034c8607a9968bc5c1c7e/external/rules_jvm_external/coursier.bzl", line 374
		generate_imports(repository_ctx = repository_ctx, d..., ...)
	File "/private/var/tmp/_bazel_senpo/d029bda8bea034c8607a9968bc5c1c7e/external/rules_jvm_external/coursier.bzl", line 228, in generate_imports
		fail((((("The artifact for " + artifa...)))
The artifact for io.grpc:grpc-android:jar:1.16.1 was not downloaded. Perhaps the packaging type is not one of: jar, aar, bundle?
Parsed artifact data:{"coord": "io.grpc:grpc-android:jar:1.16.1", "file": None, "dependencies": ["io.opencensus:opencensus-contrib-grpc-metrics:0.12.3", "com.google.code.findbugs:jsr305:3.0.2", "io.grpc:grpc-context:1.16.1", "com.google.code.gson:gson:2.7", "io.grpc:grpc-core:1.16.1", "org.checkerframework:checker-compat-qual:2.5.2", "org.codehaus.mojo:animal-sniffer-annotations:1.17", "com.google.j2objc:j2objc-annotations:1.1", "com.google.errorprone:error_prone_annotations:2.2.0", "io.opencensus:opencensus-api:0.12.3", "com.google.guava:guava:26.0-android"]} and referenced by '//:my_lib'

Update Bazel docs to reference rules_jvm_external

macOS, Windows CI: Ongoing concurrent download for https://oss.sonatype.org/...

https://buildkite.com/bazel/rules-jvm-external/builds/109#82cb6eb7-5f52-4bd0-af58-9d69bf8dd0f1

ERROR: /Users/buildkite/builds/buildkite-imacpro-7/bazel/rules-jvm-external/tests/integration/BUILD:3:1: no such package '@other_maven//': Unable to run coursier: Error while downloading https://oss.sonatype.org/content/repositories/releases/io/get-coursier/coursier-cli_2.12/1.1.0-M9/coursier-cli_2.12-1.1.0-M9-standalone.jar: Ongoing concurrent download for https://oss.sonatype.org/content/repositories/releases/io/get-coursier/coursier-cli_2.12/1.1.0-M9/coursier-cli_2.12-1.1.0-M9-standalone.jar, ignoring it

Use versionless name in jvm_import and don't use alias

I am using rules_scala and I have a target using alias for external dependencies.

scala_library(
    name = "my_project",
    srcs = glob(["src/main/**/*.scala"]),
    visibility = ["//visibility:public"],
    deps = [
        "@maven//:com_box_box_java_sdk",
        "@maven//:com_eclipsesource_minimal_json_minimal_json",
        "@maven//:com_amazonaws_amazon_kinesis_connectors",
    ],
)

In rules_scala, there is a deps checker which will log a buildozer command to fix the deps. In my case for example, @maven//:com_amazonaws_amazon_kinesis_connectors is not used, so a command like

buildozer 'remove deps @mvn//:com_amazonaws_amazon_kinesis_connectors_1_3_0' //my_project:my_project

will be logged. However, the resolved deps name is used rather than the alias name, so buildozer fails to fix BUILD.bazel.

I investigated how deps checker works but I realized there is no easy fix in rules_scala. I am wondering why do we need alias in rules_jvm_external in first place?

Is it possible to have an option to use versionless name in jvm_import directly and don't use alias? It will solve the problem above easily.

Thanks.

Support "orbit" package types

When using rules_jvm_external on a project pulling in an old version of Dropwizard ("io.dropwizard:dropwizard-core:0.7.1",), Bazel fails miserably because one of Dropwizard's declared dependencies in the POM being of package type orbit:

org.eclipse.jetty.orbit:javax.servlet:3.0.0.v201112011016

<groupId>org.eclipse.jetty.orbit</groupId>
<artifactId>javax.servlet</artifactId>
<version>3.0.0.v201112011016</version>
<name>Jetty Orbit :: Servlet API</name>
<description>
This artifact originates from the Orbit Project at Eclipse, it is an osgi bundle and is signed as well.
</description>
<packaging>orbit</packaging>

Fetching/resolving it with the coursier CLI works fine though. Should be an easy fix.

No resource identifier found for attribute in package

I'm trying to include an external library and use it via xml. The resource processor is unable to find identifiers in the imported package instead searching for it in the current package.

I'm installing the libraries via

maven_install(
    artifacts = [
        "androidx.constraintlayout:constraintlayout:1.1.3",
        "androidx.annotation:annotation:1.0.2",
        "androidx.appcompat:appcompat:1.0.2",
        "androidx.core:core-ktx:1.0.1",
        "androidx.recyclerview:recyclerview:1.0.0",
        "com.google.android.material:material:1.0.0",
    ],
    repositories = [
        "https://jcenter.bintray.com/",
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
    ],
    fetch_sources = True,
)

And importing them via

kt_android_library(
    name = "lib",
    srcs = glob(["java/**/*.kt"]),
    custom_package = "com.matthewcheok.bazeltest",
    deps = [
        ":resources",
        "@maven//:androidx_constraintlayout_constraintlayout",
        "@maven//:androidx_annotation_annotation",
        "@maven//:androidx_appcompat_appcompat",
        "@maven//:androidx_core_core",
        "@maven//:androidx_core_core_ktx",
        "@maven//:androidx_drawerlayout_drawerlayout",
        "@maven//:androidx_lifecycle_lifecycle_common",
        "@maven//:androidx_lifecycle_lifecycle_viewmodel",
        "@maven//:androidx_fragment_fragment",
        "@maven//:androidx_recyclerview_recyclerview", 
        "@maven//:com_google_android_material_material",
    ],
    visibility = ["//visibility:public"],
)

The error is

Error at 10 : /var/folders/x1/s1jm_0bs2gn3nrs5cmbf6l7r0000gn/T/resource_validator_tmp5683318188242105018/tmp-expanded/res/layout/activity_main.xml:10: error: No resource identifier found for attribute 'cardCornerRadius' in package 'com.matthewcheok.bazeltest'
5	:      android:layout_width="match_parent"
6	:      android:layout_height="match_parent"
7	:      android:orientation="vertical">
8	:
9	:      <com.google.android.material.card.MaterialCardView
10	:          android:layout_width="match_parent"
11	:          android:layout_height="72dp"
12	:          android:layout_margin="20dp"
13	:          android:layout_marginTop="12dp"
14	:          android:layout_marginBottom="12dp"

Error at 10 : /var/folders/x1/s1jm_0bs2gn3nrs5cmbf6l7r0000gn/T/resource_validator_tmp5683318188242105018/tmp-expanded/res/layout/activity_main.xml:10: error: No resource identifier found for attribute 'contentPadding' in package 'com.matthewcheok.bazeltest'
5	:      android:layout_width="match_parent"
6	:      android:layout_height="match_parent"
7	:      android:orientation="vertical">
8	:
9	:      <com.google.android.material.card.MaterialCardView
10	:          android:layout_width="match_parent"
11	:          android:layout_height="72dp"
12	:          android:layout_margin="20dp"
13	:          android:layout_marginTop="12dp"
14	:          android:layout_marginBottom="12dp"

The sample repo is available at https://github.com/matthewcheok/bazel_xml_validation

unexpected keyword 'fail_on_missing_checksum' in call to maven_install(name = "maven", repositories = [], artifacts = [], fetch_sources = False, use_unsafe_shared_cache = False)

I met this error with WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_JVM_EXTERNAL_TAG = "2.0.1"
RULES_JVM_EXTERNAL_SHA = "55e8d3951647ae3dffde22b4f7f8dee11b3f70f3f89424713debd7076197eaca"

http_archive(
    name = "rules_jvm_external",
    strip_prefix = "rules_jvm_external-%s" % RULES_JVM_EXTERNAL_TAG,
    sha256 = RULES_JVM_EXTERNAL_SHA,
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % RULES_JVM_EXTERNAL_TAG,
)

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "com.github.pagehelper:pagehelper:4.1.6",
    ],
    repositories = [
        "https://jcenter.bintray.com/",
        "https://repo1.maven.org/maven2",
    ],
    fail_on_missing_checksum = False
)

And work on mac osx with bazel version:

$ bazel version
Build label: 0.25.0
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed May 1 21:47:49 2019 (1556747269)
Build timestamp: 1556747269
Build timestamp as int: 1556747269

should I upgrate bazel or rules_jvm_external? it seems the latest vesion now.

Annotation processor not found

I added a java_plugin to enable support for annotation processors but it doesn't seem to work:

java_plugin(
    name = "epoxy_plugin",
    processor_class = "com.airbnb.epoxy.EpoxyProcessor",
    deps = [
        "@maven//:com_airbnb_android_epoxy",
        "@maven//:com_airbnb_android_epoxy_annotations",
    ],
)

java_library(
    name = "epoxy_lib",
    exported_plugins = ["epoxy_plugin"],
)

I'm getting the following error:

warning: [kapt] Can't find annotation processor class com.airbnb.epoxy.EpoxyProcessor: com.airbnb.epoxy.EpoxyProcessor
src/main/java/com/matthewcheok/bazeltest/MainActivity.kt:19:17: error: unresolved reference: itemCustomView
                itemCustomView {
                ^
src/main/java/com/matthewcheok/bazeltest/MainActivity.kt:20:21: error: unresolved reference: id
                    id("custom view $i")
                    ^
src/main/java/com/matthewcheok/bazeltest/MainActivity.kt:21:21: error: expression 'title' of type 'CharSequence!' cannot be invoked as a function. The function 'invoke()' is not found
                    title("this is a green custom view item")
                    ^

The unresolved references are likely because we're not getting the code generation expected.

The sample repo is at https://github.com/matthewcheok/bazel-epoxy-annotation-processor

Scala Akka example tests are failing on Windows

PS C:\Users\jingwen\Code\rules_jvm_external\examples\scala_akka> bazel test //... --test_output=errors
INFO: Analysed 4 targets (0 packages loaded, 0 targets configured).
INFO: Found 2 targets and 2 test targets...
FAIL: //:test_test_suite_src_test_scala_hello_ConfigurationSpec.scala (see C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/testlogs/test_test_suite_src_test_scala_hello_ConfigurationSpec.scala/test.log)
INFO: From Testing //:test_test_suite_src_test_scala_hello_ConfigurationSpec.scala:
==================== Test output for //:test_test_suite_src_test_scala_hello_ConfigurationSpec.scala:
C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_ConfigurationSpec.scala: line 334: C:\users\jingwen\_bazel_jingwen\viloy4hd\execroot\__main__\bazel-out\x64_windows-fastbuild\bin\test_test_suite_src_test_scala_hello_ConfigurationSpec.scala.runfiles/local_jdk/bin/jar.exe: No such file or directory
C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_ConfigurationSpec.scala: line 83: printf: missing unicode digit for \u
C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_ConfigurationSpec.scala: ERROR: /c/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_ConfigurationSpec.scala failed because C:\users\jingwen\_bazel_jingwen�iloy4hdecroot\__main_azel-outd_windows-fastbuilin        est_test_suite_src_test_scala_hello_ConfigurationSpec.scala.runfiles/local_jdk/bin/jar.exe failed
================================================================================
FAIL: //:test_test_suite_src_test_scala_hello_EndpointSpec.scala (see C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/testlogs/test_test_suite_src_test_scala_hello_EndpointSpec.scala/test.log)
INFO: From Testing //:test_test_suite_src_test_scala_hello_EndpointSpec.scala:
==================== Test output for //:test_test_suite_src_test_scala_hello_EndpointSpec.scala:
C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_EndpointSpec.scala: line 334: C:\users\jingwen\_bazel_jingwen\viloy4hd\execroot\__main__\bazel-out\x64_windows-fastbuild\bin\test_test_suite_src_test_scala_hello_EndpointSpec.scala.runfiles/local_jdk/bin/jar.exe: No such file or directory
C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_EndpointSpec.scala: line 83: printf: missing unicode digit for \u
C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_EndpointSpec.scala: ERROR: /c/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/bin/test_test_suite_src_test_scala_hello_EndpointSpec.scala failed because C:\users\jingwen\_bazel_jingwen�iloy4hdecroot\__main_azel-outd_windows-fastbuilin  est_test_suite_src_test_scala_hello_EndpointSpec.scala.runfiles/local_jdk/bin/jar.exe failed
================================================================================
INFO: Elapsed time: 3.911s, Critical Path: 2.98s
INFO: 4 processes: 4 local.
INFO: Build completed, 2 tests FAILED, 3 total actions
//:test_test_suite_src_test_scala_hello_ConfigurationSpec.scala          FAILED in 1.9s
  C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/testlogs/test_test_suite_src_test_scala_hello_ConfigurationSpec.scala/test.log
//:test_test_suite_src_test_scala_hello_EndpointSpec.scala               FAILED in 2.2s
  C:/users/jingwen/_bazel_jingwen/viloy4hd/execroot/__main__/bazel-out/x64_windows-fastbuild/testlogs/test_test_suite_src_test_scala_hello_EndpointSpec.scala/test.log

Unable to fetch org.pantsbuild:jarjar:1.6.6

Repro:

maven_install(
    name = "other_maven",
    artifacts = [
        "org.pantsbuild:jarjar:1.6.6",
    ],
    fetch_sources = True,
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)
$ bazel query @other_maven//:all
ERROR: no such package '@other_maven//': Traceback (most recent call last):
	File "/usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/rules_jvm_external/coursier.bzl", line 380
		generate_imports(repository_ctx = repository_ctx, d..., ...)
	File "/usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/rules_jvm_external/coursier.bzl", line 228, in generate_imports
		fail((((("The artifact for " + artifa...)))
The artifact for org.eclipse.sisu:org.eclipse.sisu.inject:0.3.2 was not downloaded. Perhaps the packaging type is not one of: jar, aar, bundle?
Parsed artifact data:{"coord": "org.eclipse.sisu:org.eclipse.sisu.inject:0.3.2", "file": None, "dependencies": []}
ERROR: no such package '@other_maven//': Traceback (most recent call last):
	File "/usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/rules_jvm_external/coursier.bzl", line 380
		generate_imports(repository_ctx = repository_ctx, d..., ...)
	File "/usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/rules_jvm_external/coursier.bzl", line 228, in generate_imports
		fail((((("The artifact for " + artifa...)))
The artifact for org.eclipse.sisu:org.eclipse.sisu.inject:0.3.2 was not downloaded. Perhaps the packaging type is not one of: jar, aar, bundle?
Parsed artifact data:{"coord": "org.eclipse.sisu:org.eclipse.sisu.inject:0.3.2", "file": None, "dependencies": []}
Loading: 0 packages loaded
    Fetching @other_maven; Generating BUILD targets.. 16s

The issue is that org.eclipse.sisu:org.eclipse.sisu.inject:0.3.2 type is eclipse-plugin as seen by Coursier, so it wasn't resolved/fetched, even though it has a jar:

https://mvnrepository.com/artifact/org.eclipse.sisu/org.eclipse.sisu.inject/0.3.2
http://central.maven.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/

Adding eclipse-plugin to the --artifact-type flag alongside jar,aar,bundle fixes this:

cmd.extend(["--artifact-type", "jar,aar,bundle"])

The generated target is a java_import:

$ bazel query --output=build @other_maven//:org_eclipse_sisu_org_eclipse_sisu_inject_0_3_2
# /usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/other_maven/BUILD:165:1
java_import(
  name = "org_eclipse_sisu_org_eclipse_sisu_inject_0_3_2",
  tags = ["maven_coordinates=org.eclipse.sisu:org.eclipse.sisu.inject:0.3.2"],
  jars = ["@other_maven//:v1/https/repo1.maven.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/org.eclipse.sisu.inject-0.3.2.jar"],
  srcjar = "@other_maven//:v1/https/repo1.maven.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/org.eclipse.sisu.inject-0.3.2-sources.jar",
  deps = [],
)

Make bazel download files

Coursier optimistically downloads every single jar file required when starting a build. When the number of dependencies is large, this can cause a significant delay before anything starts being built, particularly since jar files are significantly larger than pom files (on the whole)

If, instead, maven_install output an http_file for each jar or source jar required, builds could start a lot sooner and we could ensure that all downloads happen through Bazel's own mechanisms.

There doesn't appear to be a command line flag to make this happen automatically, but once we have resolved all the dependencies, the URLs to download from tend to be a simple transformation of the maven coordinates.

Special handling needed for pom-only dependencies

I was really excited to try this out (v1.2) and went with a single library I'm familiar with that brings in a whole big tree of dependencies.

maven_install(
    artifacts = [
        "nz.ac.waikato.cms.weka:weka-stable:3.8.1",
    ],
    repositories = [
        "https://repo1.maven.org/maven2",
    ],
)

When I ran bazel query @maven//:all (thanks @jin), it blew up with the following, because com.github.fommil.netlib:all (transitive dep) is a pom-only dependency and exists solely to aggregate other dependencies:

The artifact for com.github.fommil.netlib:all:pom:1.1.2 was not downloaded. Perhaps its packaging type is
not one of: jar,aar,bundle,eclipse-plugin?

Fix Java integration tests to be compatible with OpenJDK 11

They were disabled in #127

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tests/integration:ArtifactExclusionsTest
-----------------------------------------------------------------------------
JUnit4 Test Runner
.E
Time: 0.009
There was 1 failure:
1) test_excludedArtifacts_notOnClassPath(com.jvm.external.ArtifactExclusionsTest)
java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
	at com.jvm.external.ArtifactExclusionsTest.test_excludedArtifacts_notOnClassPath(ArtifactExclusionsTest.java:16)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at com.google.testing.junit.runner.internal.junit4.CancellableRequestFactory$CancellableRunner.run(CancellableRequestFactory.java:89)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at com.google.testing.junit.runner.junit4.JUnit4Runner.run(JUnit4Runner.java:112)
	at com.google.testing.junit.runner.BazelTestRunner.runTestsInSuite(BazelTestRunner.java:153)
	at com.google.testing.junit.runner.BazelTestRunner.main(BazelTestRunner.java:84)

FAILURES!!!
Tests run: 1,  Failures: 1


BazelTestRunner exiting with a return value of 1
JVM shutdown hooks (if any) will run now.
The JVM will exit once they complete.

-- JVM shutdown starting at 2019-04-25 09:29:31 --

Other logs: https://buildkite.com/bazel/rules-jvm-external/builds/390#b676ef53-ae08-46ce-ae22-a2d4bc336479

v2.0: Unable to generate a target for com.android.support:appcompat-v7:aar:28.0.0

Artifact coordinates: com.android.support:appcompat-v7:aar:28.0.0
Parsed data: {"coord": "com.android.support:appcompat-v7:aar:28.0.0", "file": "v1/https/maven.google.com/com/android/support/appcompat-v7/28.0.0/appcompat-v7-28.0.0.aar", "dependencies": ["android.arch.lifecycle:viewmodel:aar:1.1.1", "com.android.support:customview:aar:28.0.0", "com.android.support:animated-vector-drawable:aar:28.0.0", "com.android.support:print:aar:28.0.0", "com.android.support:asynclayoutinflater:aar:28.0.0", "com.android.support:support-core-utils:aar:28.0.0", "com.android.support:collections:28.0.0", "com.android.support:support-fragment:aar:28.0.0", "com.android.support:viewpager:aar:28.0.0", "com.android.support:support-vector-drawable:aar:28.0.0", "com.android.support:interpolator:aar:28.0.0", "com.android.support:versionedparcelable:aar:28.0.0", "com.android.support:support-core-ui:aar:28.0.0", "com.android.support:swiperefreshlayout:aar:28.0.0", "android.arch.lifecycle:runtime:aar:1.1.1", "com.android.support:support-annotations:28.0.0", "android.arch.core:runtime:aar:1.1.1", "com.android.support:coordinatorlayout:aar:28.0.0", "com.android.support:loader:aar:28.0.0", "android.arch.lifecycle:livedata:aar:1.1.1", "com.android.support:support-compat:aar:28.0.0", "com.android.support:localbroadcastmanager:aar:28.0.0", "com.android.support:cursoradapter:aar:28.0.0", "com.android.support:drawerlayout:aar:28.0.0", "android.arch.lifecycle:livedata-core:aar:1.1.1", "android.arch.lifecycle:common:1.1.1", "com.android.support:slidingpanelayout:aar:28.0.0", "com.android.support:documentfile:aar:28.0.0", "android.arch.core:common:1.1.1"]}

Most definitely caused by dbc4aa7

Add sha1s to gmaven.bzl

This would improve security. Also I hypothesize that this will make things work better with --experimental_repository_cache flag. (not 100% sure how that works, but appears to work with other types of builds once a hash is added)

Add `maven_coordinates=` tag to targets

If you want to build a jar for upload that uses these rules, you need to generate a pom file. One way to do that is with this rule https://github.com/google/bazel-common/blob/f1115e0f777f08c3cdb115526c4e663005bec69b/tools/maven/pom_file.bzl

The way this rule works is it checks your dependencies for a tag in the format maven_coordinates={actual maven coordinates}. It would be useful if these rules propagated this tag (or some alternative solution to this) so that pom file generation could use the same coordinate values specified only once.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.