Git Product home page Git Product logo

argoproj-labs / argocd-vault-plugin Goto Github PK

View Code? Open in Web Editor NEW
786.0 16.0 185.0 2.86 MB

An Argo CD plugin to retrieve secrets from Secret Management tools and inject them into Kubernetes secrets

Home Page: https://argocd-vault-plugin.readthedocs.io

License: Apache License 2.0

Makefile 0.10% Go 99.86% Dockerfile 0.05%
argo-cd gitops argocd-plugin secret-management aws-secrets-manager kubernetes vault gcp-secret-manager azure-keyvault secrets-manager

argocd-vault-plugin's Introduction

argocd-vault-plugin

Pipeline Code Scanning Go Report Card Downloads codecov

An Argo CD plugin to retrieve secrets from various Secret Management tools (HashiCorp Vault, IBM Cloud Secrets Manager, AWS Secrets Manager, etc.) and inject them into Kubernetes resources

Why use this plugin?

This plugin is aimed at helping to solve the issue of secret management with GitOps and Argo CD. We wanted to find a simple way to utilize Secret Management tools without having to rely on an operator or custom resource definition. This plugin can be used not just for secrets but also for deployments, configMaps or any other Kubernetes resource.

Documentation

You can our full set of documentation at https://argocd-vault-plugin.readthedocs.io/

Contributing

Interested in contributing? Please read our contributing documentation here to get started!

Blogs

Presentations

argocd-vault-plugin's People

Contributors

c0deltin avatar crenshaw-dev avatar dathumpingrabbit avatar denraf avatar dependabot[bot] avatar fty4 avatar fvdnabee avatar jbaskeen avatar jessebot avatar jhoover4 avatar jkayani avatar kavfixnel avatar khetanir avatar kroustou avatar kserik avatar loxley avatar lucasmarshall avatar marcportabellaclotet-mt avatar markusbauerbe avatar michaelsp avatar pascalbourdier avatar skuethe avatar sspreitzer avatar stevemar avatar tcardonne avatar teejaded avatar tico24 avatar vtomasr5 avatar werne2j avatar yveszelros avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

argocd-vault-plugin's Issues

FATA[0000] rpc error: code = Unknown desc = multiple application sources defined: Helm,Plugin

Is your feature request related to a problem? Please describe.
Below is whats working

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: your-application-name
spec:
  destination:
    name: ''
    namespace: default
    server: 'https://kubernetes.default.svc'
  source:
    path: .
    plugin:
      name: argocd-vault-plugin
    repoURL: http://your-repo/
    targetRevision: HEAD
  project: default

However, if we want to use source as helm type instead of plain yamls, ie argo application that is of helm source type, and we cant use the plugin together as there is only one type you can use.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: {{ .name }}
  namespace: argo-cd
spec:
  destination:
    namespace: {{ .namespace }}
    server: {{ .api }}
  project: {{ .argocdProject }}
  source:
    helm:
      valueFiles:
      - values.yaml
    plugin:
      name: argocd-vault-plugin
    path: {{ .path }}
    repoURL: {{ .repo }}
    targetRevision: {{ .targetRevision }}
  syncPolicy:
    automated:
      selfHeal: true
argocd app create argovault-test --config-management-plugin argocd-vault-plugin \
--repo https://github.com/test/test.git \
--path Cluster/test/cluster-resources  \
--dest-server https://kubernetes.default.svc \
--dest-namespace default \
--values value.yaml
FATA[0000] rpc error: code = Unknown desc = multiple application sources defined: Helm,Plugin

Describe the solution you'd like

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

getting an issue while using secrets object with deployment objects

I'm getting an issue while using secrets object with deployment objects.
repo >>
https://github.com/anup1384/arogcd-vault-demo.git

ComparisonError
rpc error: code = Unknown desc = Manifest generation error (cached): argocd-vault-plugin generate ./ failed exit status 1: Error: Error making API request. URL: GET http://vault.abc.com/v1/deployment Code: 403. Errors: * permission denied Usage: argocd-vault generate [flags] Flags: -c, --config-path string path to a file containing Vault configuration (YAML, JSON, envfile) to use -h, --help help for generate -s, --secret-name string name of a Kubernetes Secret containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD) Error making API request. URL: GET http://vault.abc.com/v1/deployment Code: 403. Errors: * permission denied
<<<

Kubernetes authorization service account should include permissions for tokenreviews

Describe the bug
When using the kubernetes authorization, the serviceaccount needs tokenreviews for vault to be valid. It is the same as coreos/vault-operator#359.

To Reproduce
Using a service account with the vault.

Expected behavior
Document that references this requirement.

Screenshots
2021-03-24T01:10:42.768Z [ERROR] auth.kubernetes.auth_kubernetes_6370701c: login unauthorized due to: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"tokenreviews.authentication.k8s.io is forbidden: User "system:serviceaccount:argocd:argocd-server" cannot create resource "tokenreviews" in API group "authentication.k8s.io" at the cluster scope","reason":"Forbidden","details":{"group":"authentication.k8s.io","kind":"tokenreviews"},"code":403

Additional context
I make it work with the following objects for the previous SA (argocd/default):

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: repo-server
    app.kubernetes.io/name: argocd-repo-server
    app.kubernetes.io/part-of: argocd
  name: argocd-repo-server
rules:
- apiGroups:
  - 'authentication.k8s.io'
  resources:
  - 'tokenreviews'
  verbs:
  - '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: repo-server
    app.kubernetes.io/name: argocd-repo-server
    app.kubernetes.io/part-of: argocd
  name: argocd-repo-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: argocd-repo-server
subjects:
- kind: ServiceAccount
  name: default
  namespace: argocd

Vault kv2 secret versioning support

Does this plugin support vault secret versioning?
If not, do you plan on supporting it in the future?

Example:

kind: Secret
apiVersion: v1
metadata:
  name: example-secret
  annotations:
    vault_secret_version: '5'  # ability to specify secret revision
    avp_path: "path/to/secret"
type: Opaque
data:
  password: <password-vault-key>

Add vault path as annotation

We need to add the logic to handle a path as an annotation in in resource

kind: Secret
apiVersion: v1
metadata:
  name: example-secret
  annotations:
    path: "path/to/vault"

[Q] Updating secrets

Hi!

First of all, thank you for open-sourcing this, it looks promising.

I haven't used it yet, but I was wondering how updating secrets works? If you update the value in Vault, do you just press sync in ArgoCD to update the value in the Kubernetes secret?

Refactor config and use of constants

We should refactor the config to take advantage of enums and constants instead of having hard coded switches. Also, we should pull anything in the code that looks like it should be a constant out and have less hard coded strings in the code

How to use plugin with kustomize

Hello,
I'm new to ArgoCD and I'm facing a strange issue.

I'm using a custom plugin to get secret from Vault and produce a K8s secret.

This is my application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prometheus-self
  namespace: argocd
  labels:
    app: prometheus
    component: self
spec:
  destination:
    namespace: argocd
    server: https://kubernetes.default.svc
  project: default
  source:
    path: overlays/qa/clusters/my-cluster
    repoURL: https://myrepo.com
    targetRevision: HEAD
    plugin:
      name: argocd-vault-plugin

In my repository at path overlays/qa/clusters/my-cluster I have:

├── kustomization.yaml
├── patches
│   ├── alertmanager_patch.yaml
│   ├── grafana_patch.yaml
│   ├── ingress_controller_patch.yaml
│   ├── prometheus_operator_patch.yaml
│   └── pushgateway_patch.yaml
├── remote_secrets
│   └── my_secret.yaml
├── self.yaml

and this is my kustomation.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../../base/prometheus
- ./remote_secrets/my_secret.yaml
- ./self.yaml

If I use the plugin in my application, it isn't able to create other resource then ./remote_secrets/my_secret.yaml.

If instead I don't specify the plugin and let kustomize to work, the secret isn't update with the value from Vault.

Is this a bug or is it suppose to work like that ?

Thanks,
Antonio

Replace: could not replace all placeholders in SecretTemplate data: [replaceString: missing Vault value for placeholder passcode in string password: <passcode>]

Describe the bug
Eventhough the vault value exists in vault it throws error that value doesnt exist
To Reproduce
Steps to reproduce the behavior:
Create vault.json with vault details.
{
"VAULT_ADDR": "http://172.18.0.2:31030",
"AUTH_TYPE": "approle",
"TYPE": "vault",
"ROLE_ID": "ee948507-e4b5-1982-5186-d1de6db3e946",
"SECRET_ID": "fdde601e-fdb8-0ddb-050a-34ef1657a447"
}

Create secret.yaml with place holders i.e with angular brackets <>
kind: Secret
apiVersion: v1
metadata:
name: example-secret
annotations:
avp_path: "secret/data/mysql/webapp/"
type: Opaque
data:
**

password: ""

**

Run the below command:

argocd-vault-plugin generate secret.yaml -c vault.json
Error: Replace: could not replace all placeholders in SecretTemplate data: [replaceString: missing Vault value for placeholder passcode in string password: ]
Usage:
argocd-vault generate [flags]

Flags:
-c, --config-path string path to a file containing Vault configuration (YAML, JSON, envfile) to use
-h, --help help for generate
-s, --secret-name string name of a Kubernetes Secret containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD)

Replace: could not replace all placeholders in SecretTemplate data: [replaceString: missing Vault value for placeholder passcode in string password: ]
Login to vault and read the key
$ vault kv get secret/mysql/webapp
====== Metadata ======
Key Value


created_time 2021-01-30T08:00:26.757330401Z
deletion_time n/a
destroyed false
version 1

====== Data ======
Key Value


passcode my-long-passcode

Option to ignore if no secret for type found

Is your feature request related to a problem? Please describe.
The plugin tries to find secrets by appending the resource type to the path_perfix. If, however, you have no secrerts for a particular resource/resource type then you get errors like the following (using a namespace as the example):

Error making API request.

URL: GET http://<vault_server>/v1/namespace
Code: 403. Errors:

* 1 error occurred:
	* permission denied

Describe the solution you'd like
For the plugin to ignore resources that have no secrets templated and simply output them as they aer

Describe alternatives you've considered
N/A

Additional context
N/A

Unable to register the plugin , following the steps of adding the configManagementPlugins

Describe the bug
A clear and concise description of what the bug is.
unable to register the argocd-vault-plugin and didnt show up on the ui
To Reproduce
Steps to reproduce the behavior:

  1. plugin was installed successfully to argocd - repo server via the init container
  2. however, following on from that, try to add the configuration management plugin to the argocd-cm
  3. unable to see the plugin registered on argoCD

if we shell into the repo-server pod, following what the configuration management plugin tries to perform, as below
generate command will not yield anything as it expects yaml files to generate manifest

repo-server-65fb95bfc8-hk92g:~$ argocd-vault-plugin generate ./
Error: Must provide a supported Vault Type
Usage:
argocd-vault generate [flags]

Flags:
-c, --config-path string path to a file containing Vault configuration (YAML, JSON, envfile) to use
-h, --help help for generate
-s, --secret-name string name of a Kubernetes Secret containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD)

Must provide a supported Vault Type

Expected behavior
expect the plugin registered and displayed at argoCDUI

Shouldn't "secretData" be named "stringData" according to Kubernetes Secrets?

Describe the bug
At first I thought it was not a bug because the replace works when testing inside the pod, but then k8s actually tries to parse the secret it fails because it doesn't recognize "secretData". The naming looks to be incorrect and doesn't match the api stringData https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#secretlist-v1-core

Steps to reproduce
Create a secret with secretData to be replaced as-is (for example with a json object):

kind: Secret
apiVersion: v1
metadata:
  name: example-secret
  annotations:
    avp_path: "avp/data/test"
type: Opaque
secretData:
  credentials.json:
    <sample>

Note that it works for the actual replace, as tested inside the pod:

argocd@argocd-repo-server-7c4cf688fd-nxxjk:~$ argocd-vault-plugin generate process/ -s argocd-vault-plugin-credentials
apiVersion: v1
kind: Secret
metadata:
  annotations:
    avp_path: avp/data/test
  name: example-secret
secretData:
  credentials.json: |
    {
      "user": "secret",
      "pass": "secret",
      "extra_secret": "secret"
    }
type: Opaque

But then the secret object fails to be parsed by k8s
image

Expected behavior
When passing string data to be emitted as-is we should use the field stringData, but the plugin handles it for a field named secretData. I expect that simply renaming the field will fix this problem.

Support multi-document yaml files

Is your feature request related to a problem? Please describe.
If I try an run the plugin against a multi document yaml file (i.e. one wth "---" document separators) I seem to get a failure.

Describe the solution you'd like
The plugin to support mutli-document yaml files.

Describe alternatives you've considered
Obviously the yaml documents can be split to separate files, but sometimes it is nice to be able to maintain relevant groupings in code.

Additional context
N/A

Allow path to secret to be secret key specific

We should allow specifying a path to a secret on a key by key basis. This would add support to anyone that use the one key per secret model. We can discuss the actual syntax but something like..

apiVersion: v1
kind: Secret
metadata:
  name: example-secret
data:
  sample-secret: <path:path/to/secret#key-name>
type: Opaque

Vault Authentication

We will need to be able to handle multiple kinds of vault authentication for the plugin.

  • AppRole
  • Token
  • Github
  • etc.

We will also want to do this in a way that is extensible so that new auth methods can be added as use cases come up.
Not sure if we want to go the way of having some sort of config or secret that we read with the auth type and the token/credentials to go along with it. Or some other way of doing it. Will leave that up to discussion.

Read templates from stdin

Is your feature request related to a problem? Please describe.

It'd be nice for AVP to read YAMLs from stdin directly, rather than expecting a path on the filesystem.

Describe the solution you'd like
Something like:

argocd-vault-plugin generate - < my-template.yaml

where - is the path argument to tell AVP read from stdin

Describe alternatives you've considered

  • Piping to an external file and then using that one works just fine

Additional context
See integrations of AVP with other YAML generators; i.e, Helm, Kustomize

Please can the plugin be made to work with .yml files as well as .yaml files

Is your feature request related to a problem? Please describe.
Because I'm lazy, I tend to call my files .yml instead of .yaml. This throws up the error: no YAML files were found in ./

Describe the solution you'd like
Given I have a file ending in .yml
When I run the plugin
Then the plugin pulls the secret(s) from vault

Vault secret changes awareness

Can the plugin regenerate the kubernetes secret if it detects a change in the secret in vault?
This would allow secrets to be completely dynamic.

Support to Read Base64 format and Decode it in Secret - for storing Certificates

@jkayani I should Thank everyone for the wonderful work.

We are expecting something similar. Describing few challenges as per my understanding and testing.

Tested the for both use cases,

          THING: <mykey>

The plugin does replace data key from Vault as-is ( but, Ref: then the plugin will base64 the Vault data before replacing the placeholder..). In our case, not the correct behaviour then.
and

          THING: <mykey>

Expected stringData inside the key should replace value from Vault as-is which is not working.

Configured yaml file looks as below:

apiVersion: v1
kind: Secret
metadata:
  name: example-secret
  annotations:
    avp_path: <kv/data/secret-folder>
type: Opaque
data:
  MyKey: <MyKey>

Thoughts on this.

Expecting a new Feature or an Existing Options are not Aware
The plugin works perfectly fine with the plain text stored in Vault get's the value back in to the secret. Is there a way we can decode values present in a Vault secret path are encoded in the base64 format at the secret end. This helps us to store certificate formats such as .crt, .pem etc...

  • Store certificates in base64 encoded format in Vault secret path and values are decoded when retrieves it from Vault server. Just to give an example. Same is achieved using Consul Template to render the Vault secret with side car container Vault k8s auth method.

Ex: ```{{with secret "secret/example"}} MyKey = {{.Data.data.MyKeyCerts | base64Decode}}{{end}}

So, is there any way we can decode base64 format data before it replaces the placeholder with the actual value --> something like "<base64decode MyKey>".

Originally posted by @adit0012m in #92 (comment)

Use vault environment variables natively

Currently we are wrapping some vault environment variabels such as vault-namespace, vault_skip_verify, vault_capath and vault_cacert with a AVP prefix.

This currently works but everytime we want to add support for a new environment variable it requires new code.

We should look to remove the avp prefix and stop wrapping the vault variables so that we can just passthrough any vault env vars through the plugin

Confusing error messages when an incorrect vault path is used

Describe the bug
I use Vault kv-v2, and in order to access a secret using the api you have to put /data/ in the path.

There are two bugs here:

  1. It would be nice for this documentation to remind users of that.
  2. If you get the path wrong and vault sends a 404, the resulting error message is confusing.

To Reproduce
Steps to reproduce the behavior:

  1. In vault, in a KV v2 Secrets Engine, do a vault kv put example/path/and exampleKey=123
  2. Retrieve that key using vault kv get -field=exampleKey example/path/and
  3. Vault will return 123
  4. Set up a test yaml file and set the avp_path annotation to match what you just tested. eg: avp_path: example/path/and
  5. When you run the plugin, you will get the following error:

Replace: could not replace all placeholders in SecretTemplate data: [replaceString: missing Vault value for placeholder username in string sample-secret: <exampleKey>]

What has actually happened is that Vault has returned a 404 on that path... but the user is not aware of this, especially as the vault kv get command works.

The fix for this is to change the avp_path value to be:

avp_path: example/data/path/and

Expected behavior
The user should be told when Vault returns a 404 to the whole path. Currently the user thinks that the issue lies purely with the key itself.

Use token if still valid

Right now we get a new token everytime the tool is called. We should store the token and check if its still valid and using that if it is still valid before trying to get a new token

Add support for vault namespaces when configuring using a secret

Is your feature request related to a problem? Please describe.
Our vault instance uses namespaces, and that needs to be set on every request to vault.

Describe the solution you'd like
The plugin should be able to read an extra var from the secret for the VAULT_NAMESPACE and pass it on each call to the vault instance.

Describe alternatives you've considered
There is a workaround to set VAULT_NAMESPACE as an env var inside the argocd-repo-server pod. But it's inconvenient to configure both a secret containing the configuration for the plugin (plus all the serviceaccount/role/rolebinding stuff), and still have to configure env vars for the pods.

Investigate supporting other secret types for IBM Cloud Secrets Manager

Currently we only support Arbitrary secrets for IBM Cloud Secrets Manager.
We should look into how we could possibly support Username/password along with Arbitrary.

This would require some ability to specify the username or password field in the <placeholder> or assume a certain naming convention for the secret keys.

There is no way to read data already base64 encoded from vault

Describe the bug
The plugin seems to always encode the data read from vault when specified in the data key, because according to the specification that's how the data should be passed into the secret (reference). To pass string data without encoding, the secret accepts a stringData key which the plugin does not encode and pass as-is. The problem then is that there is no way to read data already encoded in base64 from vault, because if we use stringData Kubernetes expects it to not be encoded, and if we use data the plugin encodes the already encoded data that is read from vault, resulting in invalid "double" base64 encoded data inside the secret in Kubernetes.

To Reproduce

  1. Add some base64 encoded data/string in a vault path
  2. Reference that secret using the data key in the secret
  3. Get the secret object as yaml from k8s and check that the data key contains double encoded data, i.e. the secret is not valid.

Expected behavior
I expect the key data to accept already encoded data from vault. I understand that this might cause confusion to users, but IMHO it seems correct to assume that if we are reading non-encoded data from vault, we should use the key stringData?

Additional context
Maybe this was a conscious UX decision, as the users will reference string secrets in vault and want them encoded into the data key inside the secret in vault. Perhaps we can have a flag to run the plugin in a special mode that expects already base64 encoded data when its provided in the key data, therefore not encoding the values read from vault (already encoded) ?

[Issue] Manifest generation error

Hi all

I follow the instruction .

When i deploy a simply like below

kind: Secret
apiVersion: v1
metadata:
  name: example-secret
  annotations:
    avp_path: "path/to/mysecret"
type: Opaque
stringData:
  mykey_invault: <mykey_invault>

Argo side appear error shows..

rpc error: code = Unknown desc = Manifest generation error (cached): argocd-vault-plugin generate ./ failed exit status 1: Error: Must provide a supported Vault Type Usage: argocd-vault-plugin generate [flags] Flags: -c, --config-path string path to a file containing Vault configuration (YAML, JSON, envfile) to use -h, --help help for generate -s, --secret-name string name of a Kubernetes Secret containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD) Must provide a supported Vault Type

What i am missing ? any hints?

PATH_PREFIX deprecation

We should look into deprecating the PATH_PREFIX as it seems to be a bit confusing and does not seem to get any usage. We can remove that as the default secret location and then force the annotation or allow paths as the secret value instead of just an annotation

0.2.0 fails to set client token if cant write config

Describe the bug
If the code fails to write to the filesystem, the token never gets set to the client

Errors:\n\n* missing client token\nUsage:\n  argocd-vault generate <path> [flags]\n\nFlags:\n  -c, --config-path string   path to a file containing Vault configuration (YAML, JSON, envfile) to use\n  -h, --help                 help for generate\n  -s, --secret-name string   name of a Kubernetes Secret containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD)\n\nError making API request.\n\nURL: GET \nCode: 400. Errors:\n\n* missing client token"```

Version 0.2.x seems to be broken

Workaround is to continue to use version 0.1.0

failed to unmarshal manifest: error converting YAML to JSON: yaml: mapping values are not allowed in this context

There seems to be an issue with some yaml marshaling somewhere

Improve testing

We currently sit around 50% code coverage, we should look to try and bolster our test suite. Some areas that need to be improved but not the only areas to target:

  • Github Auth
  • Kubernetes Auth
  • cmd/utils
  • cmd/generate

Unnecessary panics over non map/string list elements

Describe the bug

Given a YAML like this:

apiVersion: v1
kind: Secret
metadata:
  name: a-secret
  annotations:
    avp_path: secret/data/my-app/regsecret
    kv_version: "2"
type: Opaque
data:
  CONTRIVED_EXAMPLE:
    - 1
    - 2

the plugin will panic like this:

panic: Deserialized YAML list node is non map[string]interface{} nor string

goroutine 1 [running]:
github.com/IBM/argocd-vault-plugin/pkg/kube.replaceableInner(0xc000120108, 0xc000120108)
... stacktrace

The reason is because of the way list elements are handled in replacableInner() and replaceInner().

To Reproduce
See above

Expected behavior

The proper behavior should just be to skip over non-string, non-map, non-list elements since those are ineligible for containing <placeholder>'s since they are non-strings. For string, map, and list elements that appear in a list, we should check for <placeholder>'s.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

I don't think this has been an actual problem for anyone using the plugin since the YAML files processed (k8s manifests) don't typically have lists of Ints, further lists, or Bools. But it's still weird behavior that should be fixed

usage example

Is your feature request related to a problem? Please describe.
Hi,
This is more like a documentation request.

I'm trying to use this plugin with Vault and ArgoCD using K8s Secret
Currently after installing it i'm not sure it works as expected. (I'm new to ArgoCD)

My setup: I'm trying to deploy helm chart and I'm storing overrides file(with sensitive secrets) in my Vault.

Describe the solution you'd like
The repo with usage example would be nice to have. Thank you.

Update: maybe a short blog/doc/readme (I can write one as well), but so far I can't make it work with my setup.

Documentation

We want to get documentation set for at least contributing and usage early. Showing how the plugin will be able to be used and how someone could help add code

Add contributing documentation

We need to add documentation on how someone can contribute to the project, we currently have a blank contributing file

Custom CA certificate during connection to vault

Is your feature request related to a problem? Please describe.
Vault server in my organization uses self-signed certificate to serve TLS traffic. Without possibility to configure CA certificate plugin execution generates error:

Error: Put "https://vault:8200/v1/auth/approle/login": x509: certificate signed by unknown authority

Describe the solution you'd like
Add option to provide and configure CA certificate for argocd-vault-plugin for vault connections.

Incorrect permissions when installing the plugin

Describe the bug
The readme says to do the following:

 args:
    - wget -O argocd-vault-plugin
      https://github.com/IBM/argocd-vault-plugin/releases/download/v0.1.0/argocd-vault-plugin_0.1.0_linux_amd64

      mv argocd-vault-plugin /custom-tools/

Firstly, this is not the latest version of the plugin.

Secondly, if one does this then the binary does not have execution permissions and users will get this when trying to manually execute on the argocd-repo-server

argocd@argocd-repo-server-7b44865f66-7pjhw:~$ argocd-vault-plugin --help
bash: /usr/local/bin/argocd-vault-plugin: Permission denied

As a workaround, I installed the plugin using this:

args:
  - wget -O argocd-vault-plugin
    https://github.com/IBM/argocd-vault-plugin/releases/download/v0.2.2/argocd-vault-plugin_0.2.2_linux_amd64
    
    chmod 755 argocd-vault-plugin && mv argocd-vault-plugin /custom-tools/

To Reproduce
Steps to reproduce the behavior:

  1. Follow the readme
  2. exec onto one of the argocd-repo-servers
  3. Run the command: argocd-vault-plugin --help
  4. See error

Expected behavior
I should be able to follow the readme to install the plugin without error

Timeout when calling the /login endpoint on vault api

Describe the bug
We are using the plugin to connect to IBM Secrets manager and noticed that the plugin was having http timeout failures when attempting to making a PUT request to auth/ibmcloud/login. Put "<IBM_VAULT_URL>/v1/auth/ibmcloud/login": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

When testing those requests it seems the request regularly takes >10s and the DefaultHttpClient use by the VaultClient has a timeout of 10s, which is causing the timeouts I believe.

Having spoken to @werne2j, it seems the Vault default timeout is actually 60s, and the plan is to use the default hashicorp httpclient at some point - I propose in the interim the solution is to increase the timeout to 60s to match the default client.

To Reproduce
Steps to reproduce the behavior:
Run argocd-vault-plugin generate against a yaml, whilst configured for the IBM Secrets manager.

Expected behavior
Command to run successfully and replace the placeholders with password values

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Handle excluded yaml files in rendered manifests

Describe the bug
When a helm chart has a manifest wrapped in a conditional, and that condition is not met, an empty (containing only commented lines) manifest is rendered. These manifests are not handled as it is assumed that every manifest will have (amongst other things) a Kind.

To Reproduce
Steps to reproduce the behavior:

  1. Using argo-cd's helm chart, set createSecret to false. The argocd-secret will render to a manifest containing only comments
  2. Attempt to run a generate against this yaml
  3. You will see the following error:
    ToYAML: could not convert replaced template into : error unmarshaling JSON: while decoding JSON: Object 'Kind' is missing in '{}'

Expected behavior
The empty manifest should be ignored

Add support for another backend

As the name of the repo says we currently only support Vault. The whole idea was that we could potentially add more backends over time.

We want to look into another possible backend that could be added to show the pattern for adding other backends so we can be more than just a vault tool but a secret tool.

Possible secret tools:

  • AWS Secret Manager
  • GCP Secret Manager
  • Azure key Vault
  • Key Protect

Plugin find/replace logic

Here's how this plugin needs to work, roughly:

  • Accept a path in a Git repository to a collection of templated YAML files (where a file can have multiple manifests with --- concatenation). We'll want the templated YAML we expect to be well-defined (so loading into a struct)

  • Read each manifest, and perform a find/replace of Vault values based on the Kind (i.e, there may be 2 manifests with <placeholder> in them, but they should be replaced differently since they're from different kinds of resources. In other words, placeholder's don't have to be unique across manifests for a given repo)

  • Stream the resulting YAML back out for Argo to apply

Some assumptions:

  • We're planning to allow users to supply a path annotation in their templated YAML, signifying the Vault path to find/replace from. For multiple kinds of resources, that might look like:
/foo/deployment => collection of key/value pairs to replace in a `kind: Deployment`
/foo/secret => collection of key/value pairs to replace in a `kind: Secret`
etc

Some design questions:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.