Git Product home page Git Product logo

maven-ext-repos-from-env's Introduction

Maven Central Java CI/CD License SonarCloud Status SonarCloud Coverage

Maven Extension: Repos from System Env

This Maven extension allows to add additional remote repositories to the Maven execution by solely using OS level environment variables or Java system properties (without touching settings.xml nor pom.xml). Furthermore it allows to place certain artifacts directly under .mvn/repository (see details below). This is achieved by modifying the in-memory Maven settings before using them.

While most of the time setting the remote repositories in the settings.xml (and potentially also in pom.xml) is the recommended approach, for cases where the settings.xml is not under the development team's control it can be useful to configure this extension.

In case the relevant environment variables are not set this extension has no effect (apart from the implicit file repo .mvn/repository if it exists). This allows to

  • Minimise the changes in the regular project setup (only the extension has to be added, all mirrors, repositories from settings.xml may remain active for local developers or CI servers)
  • Add additional repositories with the help of environment variables or system properties.

This is helpful for constrained build environments (without full control over the settings.xml file) and also to enforce a common repository across all developers without additional manual set up. As this extension always adds a new repository it is mostly useful for cases where no POM repositories are being used. Just adding credentials for an existing (POM or settings.xml) repository is not supported yet (issue 28).

Simple Usage

Step 1: Configure the extension for your repository

In directory .mvn, create (or adjust) the file extensions.xml:

<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd">
    <extension>
        <groupId>biz.netcentric.maven.extension</groupId>
        <artifactId>maven-ext-repos-from-env</artifactId>
        <version>1.2.0</version>
    </extension>
</extensions>

See Maven documentation for reference on how extensions can be activated.

The artifact is available at Maven Central.

Step 2: Setup the environment variables or system properties for your build

In all the following examples environment variables are used, but this extension can also be configured with Java system properties having the same name as the environment variables. In case both a variable and property are given with the same name the system property takes precedence.

Remote https repo using credentials

The following environment variables can be set to have the Maven extension above automatically add both the repository (could be otherwise in pom.xml or in settings.xml) and the authentication for the server (would have to be configured in settings.xml without this extension):

export MVN_SETTINGS_REPO_URL=https://repo.myorg.com/path/to/repo
export MVN_SETTINGS_REPO_USERNAME=username
export MVN_SETTINGS_REPO_PASSWORD=password

This leads to using authentication via basic auth to the given repository.

Remote https repo without authentication

For the case no authentication is necessary, setting only one environment variable is sufficient:

export MVN_SETTINGS_REPO_URL=https://repo.myorg.com/path/to/repo_no_auth

For this case, no corresponding server entry is generated for this repository.

Using multiple repositories

It is also possible to use multiple repositories by placing an infix between the environment prefix MVN_SETTINGS_REPO and the configuration specific suffix:

# REPO_NAME1
export MVN_SETTINGS_REPO_NAME1_URL=https://repo.myorg.com/path/to/repo
export MVN_SETTINGS_REPO_NAME1_USERNAME=username1
export MVN_SETTINGS_REPO_NAME1_PASSWORD=password1
# REPO_NAME2
export MVN_SETTINGS_REPO_NAME2_URL=https://repo.myorg.com/path/to/repo
export MVN_SETTINGS_REPO_NAME2_USERNAME=username2
export MVN_SETTINGS_REPO_NAME2_PASSWORD=password2

In this example two repositories and two server entries for are created. The order can be important for performance reasons, the repositories are added in alphabetical order of their names.

Using file repositories

As generally true for Maven repositories, it is also possible to use file urls. To reference a file repository within the build repository itself, use the property maven.multiModuleProjectDirectory in the value of MVN_SETTINGS_REPO_URL, e.g. MVN_SETTINGS_REPO_URL=file://${maven.multiModuleProjectDirectory}/vendor1/repository. If the directory .mvn/repository exists, it is automatically added as file repository.

This approach can be useful for parent poms or importing dependencies (scope import).

Using the verbose logging mode

With the environment variable MVN_SETTINGS_REPO_LOG_VERBOSE, some more logging can be activated:

export MVN_SETTINGS_REPO_LOG_VERBOSE=true

Bypassing mirrors

In case the settings.xml defines one or multiple mirrors, those are automatically disabled for the newly added repositories by adding their ids to the mirrorOf value with a leading ! as documented in Advanced Mirror Specification. To disable this mirror bypass set the environment variable MVN_DISABLE_BYPASS_MIRRORS to true.

Repository order

By default, the repositories as configured in env are queried after the default repositories in settings.xml. For the case that the system env repo is also a proxy repository for Maven Central (i.e. containing all required artifacts), it can be forced to be used first (before the repositories from settings.xml).

export MVN_SETTINGS_ENV_REPOS_FIRST=true

Setting this flag is only relevant for performance reasons, the overall build will work with or without this flag. For the general order of repositories (both from settings and POM) refer to https://maven.apache.org/guides/mini/guide-multiple-repositories.html#repository-order.

Distribute configuration with code

In case environment variables are too much effort to set one can also leverage the .mvn/jvm.config file to set the configuration via Java system properties.

An example file might look like this

-DMVN_SETTINGS_REPO_NAME1_URL=https://repo.myorg.com/path/to/repo -DMVN_SETTINGS_REPO_NAME1_USERNAME=username1 -DMVN_SETTINGS_REPO_NAME1_PASSWORD=password1

Such a configuration can be distributed through the SCM along with the code (in case the values are not supposed to be treated as secrets).

Usage with Adobe Experience Manager Cloud Manager

Step 1: Configure the extension for your repository

Reference the Maven extension in the .mvn/extensions.xml file as described above.

Step 2: Setup the environment for your build using Adobe IO

Prerequisites:

Once everything is set up, the environment variables of the Cloud Manager build can be set as follows:

aio cloudmanager:set-pipeline-variables \
   <PIPELINE_ID> \
   --programId=<PROGRAM_ID> \
   --variable \
     MVN_SETTINGS_REPO_URL <REPO_URL> \
     MVN_SETTINGS_REPO_USERNAME <REPO_USER> \
   --secret \
     MVN_SETTINGS_REPO_PASSWORD <REPO_PASSWORD>  

The parameters <PIPELINE_ID> and <PROGRAM_ID> can be derived from URLs when browsing the Cloud Manager. The call needs to be made for each pipeline as set up in cloud manager (all non-prod and the prod pipeline).

See also official Adobe documentation and reference on GitHub

maven-ext-repos-from-env's People

Contributors

ghenzler avatar kwin avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

rampai94

maven-ext-repos-from-env's Issues

Allow to parameterize the server id for POM repositories

In case of POM repositories only the credentials are being maintained in the settings.xml. In order to support that use case better it would be helpful to optionally allow to parameterize the server id (as it must match the repository id from the pom.xml). In such a case the repository URL doesn't need to be populated obviously (because it is contained in the POM repository).

Keep original maven repository as primary repository

thanks for this extension, it works very well! however, in my observation registering a private repository increased the duration of the "Build & Unit Testing" step in cloud manager by a magnitude (from <10 minutes to nearly 40 minutes).

the reason seems to be that now everything is downloaded from the private maven repository. and as the cloud manager does keep a local maven repository with release artifacts already download between execution steps (unlike e.g. travis where the cache can be used for this), it downloads the "whole internet" (tons of maven artifacts) on each run.

did you try to register the private repository as a secondary maven repository, so the primary source for downloading is still the adobe-configured repository, and the private repository is only used as fallback? is this possible with this extensions?

Circumvent mirrors defined in the settings.xml

Currently all repositories are overridden by a mirror defined in the settings.xml. To ease usage with custom settings.xml and bypass its mirror it should optionally be possible to exclude the repository ids used in the extension from the defined mirrors.

This should be feasible by calling https://maven.apache.org/ref/3.6.3/maven-settings/apidocs/org/apache/maven/settings/Mirror.html#setMirrorOf-java.lang.String- with a modified value according to https://maven.apache.org/guides/mini/guide-mirror-settings.html#advanced-mirror-specification

Clarify documentation on implicit file repositories

The readme currently states

In case the relevant environment variables are not set, this extension has no effect.

This is wrong for the implicit file repository .mvn/repository which is always added even if no environment variables are set.
Either the documentation needs to be fixed or the implementation needs to adhere to the documentation (the latter probably requires an additional flag to be able to enable the implicit file repository only).

Support for file urls

File urls are generally supported for maven repositories: https://maven.apache.org/guides/introduction/introduction-to-repositories.html#internal-repositories

Using file urls with artifacts provided in git repo itself can be useful, especially if it is only about parent poms. The big advantage is that

  • No remote repository on the internet has to be maintained
  • It can also be used for development team members that don't have access to such a repository

For the case the directory .mvn/repository exists in project structure (similar path as ~/.m2/repository), it should automatically be registered as repository. For other location a URL like file://${maven.multiModuleProjectDirectory}/artifacts/vendor1 or file://${maven.multiModuleProjectDirectory}/artifacts/vendor2 shall be supported (automatically replacing ${maven.multiModuleProjectDirectory} with the reactor root.

Improve logging

It should always be logged on INFO level that an additional remote repository has been added (

logger.debug("Repository from system env: " + repoFromEnv.getUrl() + " (id: "+repoFromEnv.getId()
)

In addition some more debug logs would be helpful which expose

  1. that not all necessary environment variables are set (which ones?)
  2. which remote repositories are already considered (https://maven.apache.org/ref/3.3.9/maven-core/apidocs/org/apache/maven/execution/MavenExecutionRequest.html#getRemoteRepositories()) to be able to detect duplicate remote repo URLs
  3. which mirrors are currently configured (https://maven.apache.org/ref/3.3.9/maven-core/apidocs/org/apache/maven/execution/MavenExecutionRequest.html#getMirrors()) as mirror may prevent the feature from working IMHO

Avoid warning when using implicit file repository in .mvn/repository

[WARNING] 
[WARNING] Some problems were encountered while building the effective model for com.corp.assets:corp-assets-root:pom:2.0.0-SNAPSHOT
[WARNING] 'repositories.repository.id' must not contain any of these characters \/:"<>|?* but found /
[WARNING] 'pluginRepositories.pluginRepository.id' must not contain any of these characters \/:"<>|?* but found /
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING] 

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.