Git Product home page Git Product logo

marathon-vault-plugin's Introduction

Please note that this project is no longer being maintained.

Marathon Vault plugin Download release

Marathon version v1.5.8 v1.6.352
Build status Build1 Build2

Plugin for Marathon which injects secrets stored in Vault via environment variables.

How to reference secrets in marathon.json

The following example marathon.json fragment will read Vault path secret/shared/abc/xyz (secret/shared is taken from the configuration), extract field password from that path and inject the field value into an environment variable named ENV_NAME:

{
  "env": {
    "ENV_NAME": {
      "secret": "secret_ref"
    }
  },
  "secrets": {
    "secret_ref": {
      "source": "/abc/xyz@password"
    }
  }
}

If the provided Vault path or field is not found, the environment variable will not be set. The same applies when it cannot be read because of permissions or other types of errors. Either way, it will be logged as an error in Marathon logs.

The path in the secret source can be "shared" or "app-private" and it depends on the secret source format. The path is shared if the secret source starts with /, otherwise it is a private path. Both paths have a root defined in configuration (sharedPathRoot for shared path and privatePathRoot for private path).

Shared path to a secret

For a shared secret source path, a Vault path is constructed as <sharedPathRoot>/<path from the secret source>. E.g., with secret/shared as a value configured in sharedPathRoot, and /abc/xyz@password as the secret source, the resulting Vault path will be secret/shared/abc/xyz, field name password. This kind of secret reference allows you to share secrets between applications.

Private path to a secret

For a private secret source path, a Vault path is constructed as <privatePathRoot>/<marathon path and service name>/<path from the secret source>. E.g., with secret/marathon as a value configured in privatePathRoot, and abc/xyz@password as the secret source of an application with Marathon id test/test-app, the resulting Vault path will be secret/marathon/test/test-app/abc/xyz, field name password. This concept will guarantee that secrets cannot be read from other applications, but on the other hand identical secrets will need to be stored multiple times in Vault (separately for each Marathon application).

Installation

Please consult the Start Marathon with plugins section of the official docs for a general overview of how plugins are enabled.

The plugin configuration JSON file will need to reference the Vault plugin as follows:

{
  "plugins": {
    "marathon-vault-plugin": {
      "plugin": "mesosphere.marathon.plugin.task.RunSpecTaskProcessor",
      "implementation": "com.avast.marathon.plugin.vault.VaultPlugin",
      "configuration": {
        "address": "http://address_to_your_vault_instance:port",
        "token": "access_token",
        "sharedPathRoot": "secret/shared/",
        "privatePathRoot": "secret/private/",
        "ssl": {
            "verify": "false", // don't use in production
            "trustStoreFile": "/path/to/truststore/file",
            "keyStoreFile": "/path/to/keystore/file",
            "keyStorePassword": "keystore_passw0rd",
            "pemFile": "/path/to/pem/file",
            "pemUTF8": "string contents extracted from the PEM file",
            "clientPemFile": "/path/to/client/pem/file",
            "clientKeyPemFile": "/path/to/client/pem/file"
        }
      }
    }
  }
}

Properties sharedPathRoot and privatePathRoot are optional. Default value for both properties is root (which means /).

The ssl section is optional and it directly configures the underlying Vault client but only the options documented here are passed through.

In this version, only token-based login is supported. Never use the Vault's initial root token - we recommend creating a separate token with long-enough validity and restricted (read-only) access to secrets.

You will also need to start Marathon with the secrets feature being enabled. See Marathon command line flags for more details. In short, it can be enabled by

  • specifying --enable_features secrets in Marathon command line
  • specifying environment variable MARATHON_ENABLE_FEATURES=secrets when starting Marathon

marathon-vault-plugin's People

Contributors

augi avatar petr-k avatar rpliva avatar sjoerdmulder avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

marathon-vault-plugin's Issues

Can't specify SSL cert

Hi Avast,

In marathon-vault-plugin/src/main/scala/com/avast/marathon/plugin/vault/VaultPlugin.scala:31 you set a Vault config, but you do not allow for users to optionally set an SSL cert. Since we run Vault entirely in SSL mode, our secret fetching fails with:

[2017-12-28 20:28:49,020] ERROR Secret docker_pass in /holding/application/test2 application cannot be read from Vault (source: secret/shared/docker_pass@docker_pass) (com.avast.marathon.plugin.vault.VaultPlugin:marathon-akka.actor.default-dispatcher-25)
com.bettercloud.vault.VaultException: com.bettercloud.vault.rest.RestException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

In https://github.com/BetterCloud/vault-java-driver, it specifies how to build a SSL config that the Vault Driver can use.

Exception when starting marathon

I tried run this plugin with Marathon 1.6 and 1.7, but with no luck. Any idea?

[2019-06-10` 22:12:19,570] INFO  Started TaskTrackerUpdateStepsProcessorImpl with steps:
* continueOnError(notifyHealthCheckManager)
* continueOnError(notifyRateLimiter)
* continueOnError(notifyLaunchQueue)
* continueOnError(postTaskStatusEvent)
* continueOnError(scaleApp) (mesosphere.marathon.core.task.tracker.impl.InstanceTrackerUpdateStepProcessorImpl:JMX exporting thread)
[2019-06-10 22:12:19,622] ERROR Terminating due to uncaught exception in thread JMX exporting thread:1 (mesosphere.marathon.MarathonApp:JMX exporting thread)
com.google.inject.CreationException: Unable to create injector, see the following errors:

1) Error injecting constructor, com.fasterxml.jackson.core.JsonParseException: Unexpected character ('/' (code 47)): maybe a (non-standard) comment? (not recognized as one since Feature 'ALLOW_COMMENTS' not enabled for parser)
 at [Source: [B@288ca5f0; line: 12, column: 31]
  at mesosphere.marathon.core.CoreModuleImpl.<init>(CoreModuleImpl.scala:51)
  while locating mesosphere.marathon.core.CoreModuleImpl
  at mesosphere.marathon.core.CoreGuiceModule.configure(CoreGuiceModule.scala:191)
  while locating mesosphere.marathon.core.CoreModule
    for the 1st parameter of mesosphere.marathon.core.CoreGuiceModule.taskTracker(CoreGuiceModule.scala:59)
  at mesosphere.marathon.core.CoreGuiceModule.taskTracker(CoreGuiceModule.scala:59)
  while locating mesosphere.marathon.core.task.tracker.InstanceTracker
    for the 2nd parameter of mesosphere.marathon.core.task.update.impl.TaskStatusUpdateProcessorImpl.<init>(TaskStatusUpdateProcessorImpl.scala:26)
  while locating mesosphere.marathon.core.task.update.impl.TaskStatusUpdateProcessorImpl
  at mesosphere.marathon.core.CoreGuiceModule.configure(CoreGuiceModule.scala:194)
  while locating mesosphere.marathon.core.task.update.TaskStatusUpdateProcessor annotated with @com.google.inject.name.Named(value=ThrottlingTaskStatusUpdateProcessor)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('/' (code 47)): maybe a (non-standard) comment? (not recognized as one since Feature 'ALLOW_COMMENTS' not enabled for parser)
 at [Source: [B@288ca5f0; line: 12, column: 31]
	at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702)
	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558)
...
Caused by: akka.actor.InvalidActorNameException: actor name [instanceTracker] is not unique!
	at akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:129)
	at akka.actor.dungeon.Children.reserveChild(Children.scala:134)
	at akka.actor.dungeon.Children.reserveChild$(Children.scala:132)
	at akka.actor.ActorCell.reserveChild(ActorCell.scala:370)
	at akka.actor.dungeon.Children.makeChild(Children.scala:272)
	at akka.actor.dungeon.Children.attachChild(Children.scala:48)
	at akka.actor.dungeon.Children.attachChild$(Children.scala:47)
	at akka.actor.ActorCell.attachChild(ActorCell.scala:370)
	at akka.actor.ActorSystemImpl.actorOf(ActorSystem.scala:717)
	at mesosphere.marathon.core.leadership.LeadershipModuleImpl.startWhenLeader(LeadershipModule.scala:45)
	at mesosphere.marathon.core.task.tracker.InstanceTrackerModule.instanceTrackerActorRef$lzycompute(InstanceTrackerModule.scala:41)
	at mesosphere.marathon.core.task.tracker.InstanceTrackerModule.instanceTrackerActorRef(InstanceTrackerModule.scala:40)
	at mesosphere.marathon.core.task.tracker.InstanceTrackerModule.instanceTracker$lzycompute(InstanceTrackerModule.scala:25)
	at mesosphere.marathon.core.task.tracker.InstanceTrackerModule.instanceTracker(InstanceTrackerModule.scala:24)

support of file-based secrets

Hi,
Are there plans to support file-based secrets for marathon? From what I experience - it is not working right now

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.