Git Product home page Git Product logo

libvault's Introduction

libvault

travis-ci.com hex.pm hex.pm hex.pm github.com

Highly configurable library for HashiCorp's Vault which handles authentication for multiple backends, and reading, writing, listing, and deleting secrets for a variety of engines.

When possible, it tries to emulate the CLI, with read, write, list and delete and auth methods. An additional request method is provided when you need further flexibility with the API.

HTML docs can be found at https://hexdocs.pm/libvault.

API Preview

{:ok, vault} =
  Vault.new(
    engine: Vault.Engine.KVV2,
    auth: Vault.Auth.UserPass
  )
  |> Vault.auth(%{username: "username", password: "password"})

{:ok, db_pass} = Vault.read(vault, "secret/path/to/password")

{:ok, %{"version" => 1 }} = Vault.write(vault, "secret/path/to/creds", %{secret: "secrets!"})

Configuration / Adapters

Hashicorp's Vault is highly configurable. Rather than cover every possible option, this library strives to be flexible and adaptable. Auth backends, Secret Engines, and HTTP clients are all replaceable, and each behaviour asks for a minimal contract.

HTTP Adapters

The following HTTP Adapters are provided:

Be sure to add applications and dependencies to your mix file as needed.

JSON Adapters

Most JSON libraries provide the same methods, so no default adapter is needed. You can use Jason, JSX, Poison, or whatever encoder you want.

Defaults to Jason or Poison if present.

See Vault.JSON.Adapter for the full behaviour interface.

Auth Adapters

Adapters have been provided for the following auth backends:

In addition to the above, a generic backend is also provided (Vault.Auth.Generic). If support for auth provider is missing, you can still get up and running quickly, without writing a new adapter.

Secret Engine Adapters

Most of Vault's Secret Engines use a replaceable API. The Vault.Engine.Generic adapter should handle most use cases for secret fetching.

Vault's KV version 2 broke away from the standard REST convention. So KV has been given its own adapter:

Additional request methods

The core library only handles the basics around secret fetching. If you need to access additional API endpoints, this library also provides a Vault.request method. This should allow you to tap into the complete vault REST API, while still benefiting from token control, JSON parsing, and other HTTP client niceties.

Installation and Usage

Installation

Ensure that any adapter dependencies have been included as part of your application's dependencies:

def deps do
  [
    {:libvault, "~> 0.2.0"},

    # tesla, required for Vault.HTTP.Tesla
    {:tesla, "~> 1.3.0"},

    # pick your HTTP client - Mint, iBrowse or hackney
    {:mint, "~> 0.4.0"},
    {:castore, "~> 0.1.0"},

    # Pick your json parser - Jason or Poison
    {:jason, ">= 1.0.0"}
  ]
end

Usage

vault =
  Vault.new([
    engine: Vault.Engine.KVV2,
    auth: Vault.Auth.UserPass,
    json: Jason,
    credentials: %{username: "username", password: "password"}
  ])
  |> Vault.auth()

{:ok, db_pass} = Vault.read(vault, "secret/path/to/password")
{:ok, %{"version" => 1 }} = Vault.write(vault, "secret/path/to/creds", %{secret: "secrets!"})

You can configure the vault client up front, or change configuration on the fly.

  vault =
    Vault.new()
    |> Vault.set_auth(Vault.Auth.Approle)
    |> Vault.set_engine(Vault.Engine.Generic)
    |> Vault.auth(%{role_id: "role_id", secret_id: "secret_id"})

  {:ok, db_pass} = Vault.read(vault, "secret/path/to/password")

  vault = Vault.set_engine(Vault.Engine.KVV2) // switch to versioned secrets

  {:ok, db_pass} = Vault.write(vault, "kv/path/to/password", %{ password: "db_pass" })

See the full Vault client for additional methods.

Testing Locally

When possible, tests run against a local vault instance. Otherwise, tests run against the Vault Spec, using bypass to test to confirm the success case, and follows vault patterns for failure.

  1. Install the Vault Go CLI https://www.vaultproject.io/downloads.html

  2. In the current directory, set up a local dev server with sh scripts/setup-local-vault

  3. Vault (at this time) can't be run in the background without a docker instance. For now, set up the local secret engine paths with sh scripts/setup-engines.sh

libvault's People

Contributors

cjuniet avatar hexedpackets avatar hkrutzer avatar jvantuyl avatar kianmeng avatar matthewoden avatar philss avatar yordis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

libvault's Issues

Unable to delete KV v2 versions with Tesla Mint adapter

When deleting a KV v2 version, Vault returns a 204 HTTP response and no body. The Mint adapter treats this as nil, not an empty string, and attempting to JSON decode the nil body causes an exception to be thrown.

Example exception:

iex(20)> vault
%Vault{  
  auth: Vault.Auth.Token,
  auth_path: nil,
  credentials: %{token: "root"},
  engine: Vault.Engine.KVV2,
  host: "http://localhost:8200",
  http: Vault.HTTP.Tesla,
  http_options: [],
  json: Jason,
  token: "root",
  token_expires_at: ~N[2022-07-20 20:54:19.764878]
}
iex(21)> Vault.delete(vault, path, versions: [1], full_response: true)
** (ArgumentError) errors were found at the given arguments:

  * 1st argument: not an iodata term

    :erlang.iolist_to_binary(nil)
    (jason 1.3.0) lib/jason.ex:69: Jason.decode/2
    (libvault 0.2.3) lib/vault/http/http.ex:71: Vault.HTTP.request/4
    (libvault 0.2.3) lib/vault/engine/generic.ex:131: Vault.Engine.Generic.request/4

Loosen tesla dep

Failed to use "tesla" because
  libvault (version 0.1.2) requires ~> 1.0.0
  mix.exs specifies ~> 1.2.1

** (Mix) Hex dependency resolution failed, change the version requirements of your dependencies or unlock them (by using mix deps.update or mix deps.unlock). If you are unable to resolve the conflicts you can try overriding with {:dependency, "~> 1.0", override: true}

Dialyzer issues on Vault due to default params

lib/myapp/vault_config_provider.ex:15:unknown_function
Function Vault.auth/1 does not exist.
________________________________________________________________________________
lib/myapp/vault_config_provider.ex:15:unknown_function
Function Vault.new/1 does not exist.
________________________________________________________________________________
lib/myapp/vault_config_provider.ex:28:unknown_function
Function Vault.read/3 does not exist.

It seems that Vault.read, Vault.new and Vault.auth are failing because they have default inputs so they are arity of 2 but when I call them, I am not passing the second parameter so dialyzer things the function needs to be arity of 1

Or something else ... since Vault.new is definitely arity of 1 so I am not sure.

here is the usage

{:ok, vault} =
      Vault.new(
        engine: Vault.Engine.KVV2,
        http: Vault.HTTP.Tesla,
        json: Jason,
        host: System.fetch_env!("VAULT_ADDR"),
        auth: Vault.Auth.Kubernetes,
        credentials: %{
          role: System.fetch_env!("VAULT_K8S_ROLE"),
          jwt: File.read!(System.fetch_env!("VAULT_JWT_TOKEN_PATH"))
        }
      )
      |> Vault.auth()

    {:ok, vault_secrets} = Vault.read(vault, System.fetch_env!("VAULT_ENV_PATH"), [])

Unsupported path error is raised

Hello, I am implementing vault in an elixir project and tried libvault at first with no luck. Here is the error after authenticating successfully:

iex([email protected])3> {:ok, db_pass} = Vault.read(vault, "aws/sts/staging--www--stickfi--s3-cred")                  ** (MatchError) no match of right hand side value: {:error, ["1 error occurred:\n\n* unsupported path"]}

Here it is working with vaultex:

iex([email protected]) 2> Vaultex.Client.read "aws/sts/staging--www--stickfi--s3-cred", :token, {"foo-bar-baz"}
{:ok,
 %{
   "access_key" => "foobarbaz",
   "secret_key" => "qux",
   "security_token" => "really-long-alphanumeric-string"
 }}

Consider Engine-First API

Picking up from here: praekeltfoundation/exvault#13 (comment)

  • Having the engine be a thing "inside" the client rather than a wrapper around it makes it a bit awkward to use some of the richer APIs (like KV v2) that don't map directly onto the four basic operations. That's not necessarily a problem, but it does make using those APIs somewhat less comfortable and intuitive. For ExVault, my hope was to end up with an API that felt more like using the vault CLI.

Open to consideration.

Remove Poison dependency since it seems to be optional

The way I'm reading the documentation, the json library can be any of the given libraries and Poison is purely optional.
Since I"m already using Jason in my application, I would want to use that (and Vault.new/0 does return a vault structure with Jason as default json module.

However, I did encounter a dependency conflict as Poison recently a 4.0 version, which another dependency of my application already locked onto, but introducing libvault caused a dependency error, since it still requires ~> 3.0.

UserPass.login/2 raises "function Tesla.Adapter.Httpc.request/5 is undefined or private"

I am trying to get a token with Vault.Auth.UserPass.login/2 but I am getting an error

6> client = Vault.new([ engine: Vault.Engine.KVV2, auth: Vault.Auth.UserPass, credentials: %{username: "myuser", password: "psswrd"},http: Tesla.Adapter.Httpc, host: "https://vault.mydomain/"])   

7> Vault.Auth.UserPass.login(client, %{username: "myuser", password: "psswrd"})
** (UndefinedFunctionError) function Tesla.Adapter.Httpc.request/5 is undefined or private
    (tesla 1.3.3) Tesla.Adapter.Httpc.request(:post, "https://ault.mydomain//v1/auth/userpass/login/myuser", "{\"password\":\"psswrd\"}", [{"Content-Type", "application/json"}], [])
    (libvault 0.2.3) lib/vault/http/http.ex:70: Vault.HTTP.request/4
    (libvault 0.2.3) lib/vault/auth/userpass.ex:34: Vault.Auth.UserPass.login/2

I am getting this error with Tesla.Adapter.Hackney also

elixir -v
Erlang/OTP 23 [erts-11.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]
Elixir 1.10.4 (compiled with Erlang/OTP 22)
:libvault, "0.2.3"
:tesla, "1.3.3"

Add possibility to pass options to the HTTP client

I'm trying to replacevaultex with libvault, but one problem is holding me back: I can't pass options to the HTTP client and therefore I cannot configure SSL using the {:ssl, [cacertfile: path_to_file]} option from hackney.

I know that I can use the global application to configure that, using config :tesla, adapter: {Tesla.Adapter.Hackney, ssl: [certfile: "certs/client.crt"]}, but this is not a good approach, since the certificate may be useful only for one host, and if I'm using Tesla for more than communicating with Vault, this can be a problem.

My proposal is to add the ability to pass arbitrary options to the HTTP client at runtime. I think the usage would be something like this:

vault =
  Vault.new([
    engine: Vault.Engine.KVV2,
    http: Vault.HTTP.Tesla,
    http_client_opts: {Tesla.Adapter.Hackney, ssl: [certfile: "certs/client.crt"]},
    credentials: %{username: "username", password: "password"}
  ])

And the implementation of Vault.HTTP would have to pass down this new Keyword value to the HTTP adapter.

What do you think? Is this a valid issue? I can work on this if you agree. Thank you.


PS: This is a very nice and flexible library to communicate with Vault!
Please consider adding it to the list of Elixir libs: https://github.com/hashicorp/vault/blob/master/website/source/api/libraries.html.md#elixir

Update "Example Usage" section in the documentation

I've tried following the steps in the Example Usage section from https://hexdocs.pm/libvault/Vault.html#module-setup-and-usage, and some configurations that seem to be required now are not present:

  • a http client needs to be set explicitly, otherwise the following error pops up:

    Vault.auth(vault)
    {:error, ["http client not set"]}

  • a host needs to be set, otherwise the following error pops up:

    Vault.auth(vault)
    ** (CaseClauseError) no case clause matching: []
    (hackney 1.16.0) /home/sr/src/example/deps/hackney/src/hackney_url.erl:248: :hackney_url.parse_netloc/2
    (hackney 1.16.0) /home/sr/src/example/deps/hackney/src/hackney.erl:349: :hackney.request/5
    (libvault 0.2.2) lib/vault/http/http.ex:70: Vault.HTTP.request/4
    (libvault 0.2.2) lib/vault/auth/userpass.ex:34: Vault.Auth.UserPass.login/2
    (libvault 0.2.2) lib/vault.ex:366: Vault.auth/2

  • Using hackney as http client, setting the engine, auth, credentials and host, I'm now getting

    vault = Vault.new([engine: Vault.Engine.KVV2, auth: Vault.Auth.UserPass, credentials: %{username: "user", password: "pass"}, host: "https://myvault.local:8200/", http: :hackney])
    %Vault{
    auth: Vault.Auth.UserPass,
    auth_path: nil,
    credentials: %{password: "pass", username: "user"},
    engine: Vault.Engine.KVV2,
    host: "https://myvault.local:8200/",
    http: :hackney,
    http_options: [],
    json: Jason,
    token: nil,
    token_expires_at: nil
    }
    iex(2)> Vault.auth(vault)
    ** (ArgumentError) argument error
    (hackney 1.16.0) /home/sr/src/example/deps/hackney/src/hackney_headers_new.erl:47: :hackney_headers_new.new/1
    (hackney 1.16.0) /home/sr/src/example/deps/hackney/src/hackney.erl:331: :hackney.request/5
    (libvault 0.2.2) lib/vault/http/http.ex:70: Vault.HTTP.request/4
    (libvault 0.2.2) lib/vault/auth/userpass.ex:34: Vault.Auth.UserPass.login/2
    (libvault 0.2.2) lib/vault.ex:366: Vault.auth/2

and I don't really know how to continue.

  • Also, how would one configure a ca cert for vault instances running with self-signed certificates?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.