daspawnw / vault-crd Goto Github PK

View Code? Open in Web Editor NEW

174.0 6.0 33.0 318 KB

Vault CRD for sharing Vault Secrets with Kubernetes

Home Page: https://vault.koudingspawn.de

License: Apache License 2.0

Java 99.61% Dockerfile 0.39%

vault-crd's People

Stargazers

Watchers

vault-crd's Issues

Helm chart 1.6.0

Hi guys!
Any news about vault-crd v1.6.0 in helm chart?

Hey, it'd be great if DOCKERCFGSECRET was harmonized with the format that is created by kubectl and added an the auth field to the secret that is created in k8s, while not necessary for the the secret to work with kubernetes for image pulls, some external tools like Weavework's flux expect it to exist(fluxcd/flux#1596).

In the 1.13.0 release of kubectl they had also stripped out the auth field, but then re-added it in kubernetes/kubernetes#72344

Consider merging efforts with ESO

Hey!

Great project!

We are working on a similar project that also supports HashiVault and maybe we should merge efforts on a single project? https://github.com/external-secrets/external-secrets

We chat on #external-secrets channel on kubernetes slack, and we have community meetings every other Wednesday and you are very welcome to join!

This is a project that already is the merged efforts of some other initiatives, the most popular one being KES (Kubernetes External Secrets from Godaddy, which will be deprecated in favor of the newer ESO).

You folks already support some interesting Vault mechanisms that we don't support in ESO and would love to see that happening there, if you think it would make sense to implement them there. Also with Golang we get first-class k8s sdk support which helps a lot most of the time.

Let me know what you think!

Publish Vault-CRD Helm Chart

First off, thank you for the great work you've done with this project. This issue is in regards to the DaspawnW/vault-crd-helm repository. I was wondering if this chart was published anywhere? If not, can we consider publishing it?

I forked the project to try it out for myself, you can find it here. I can open a pull request if needed.

Package Chart Release

make release

Add Repository

helm repo add vault-crd https://raw.githubusercontent.com/null93/vault-crd-helm/chart-repository

Install Vault-CRD

helm install vault vault-crd/vault-crd

vault-crd seems to have issues reading vault 0.10.0's response

Hi,
we're currently having some issues getting vault-crd to work, the java container throws the following error:

org.springframework.web.client.RestClientException: Error while extracting response for type [class de.koudingspawn.vault.vault.impl.pki.PKIResponse] and content type [application/json]; nest
ed exception is org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Cannot deserialize instance of `java.lang.String` out of START_ARRAY token; nested except
ion is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.lang.String` out of START_ARRAY token
 at [Source: (PushbackInputStream); line: 1, column: 124] (through reference chain: de.koudingspawn.vault.vault.impl.pki.PKIResponse["data"]->de.koudingspawn.vault.vault.impl.pki.VaultRespons
eData["ca_chain"])
        at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:115) ~[spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:732) ~[spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:686) ~[spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:437) ~[spring-web-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at de.koudingspawn.vault.vault.VaultCommunication.createPki(VaultCommunication.java:41) ~[classes!/:0.0.1-SNAPSHOT]

thats our input object:

apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: test-pki
spec:
  path: "pki_exampe/issue/vaultcrdrole"
  type: "PKI"
  pkiConfiguration:
    commonName: "kaese.example.de"
    ttl: "7h"

Vault roles/policys are configured accordingly and a manual vault write using the token generated with this policy and the path above returns the set of certs.
Any ideas what this could be?

btw: vault itself issues the cert perfectly and you can get it from vault afterwards ( at least the public part :) ) but the response from vault seems to irritate vault-crd somehow.

best regards,
Bjoern

Vault Setup info

Hi @DaspawnW ,

I found this link https://koudingspawn.de/how-vault-crd-can-help/ quite informative. I am using sealed-secrets for doing all the secrets management via GitOps process, which works quite good. I had only few questions.

How is the Vault setup in K8s? (Helm install or a Vault operator is setup)? Could you elaborate on how the vault server is setup on K8s via your blog.
How is the persistence of secrets managed in Vault? Are you using a storage backend mentioned here https://www.vaultproject.io/docs/configuration/storage/ ?

Kevin

Please first apply custom resource definition and then restart vault-crd

Pod fails with this message when launching helm chart using token.

fabric8io/[email protected] too old for kubernetes 1.15.6

Upon a recent upgrade from kubernetes 1.12.8 to 1.15.6 vault-crd stopped working within our cluster with the following error:

io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 36053598 (57536768)
	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onMessage(WatchConnectionManager.java:254) [kubernetes-client-4.1.0.jar!/:na]

According to the compatibility matrix for fabric8io's kubernetes-client, the version vault-crd is using 4.1.0 only supports connecting to kubernetes 1.9.0.

It would appear that in order to support versions of kubernetes from 1.10.0 and later, vault-crd needs to be updated to use the latest available version of the kubernetes-client library.

Versions used when finding bug:

vault-crd: 1.2.1, 1.3.1
kubernetes 1.15.6

Full log output:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.1.RELEASE)

2020-01-23 11:18:47.133  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-6688df545b-f7dsr with PID 1 (/opt/vault-crd.jar started by root in /opt)
2020-01-23 11:18:47.138  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : No active profile set, falling back to default profiles: default
2020-01-23 11:18:48.802  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2020-01-23 11:18:48.833  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2020-01-23 11:18:48.833  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/9.0.13
2020-01-23 11:18:48.848  INFO 1 --- [           main] o.a.catalina.core.AprLifecycleListener   : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
2020-01-23 11:18:48.941  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2020-01-23 11:18:48.941  INFO 1 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1738 ms
2020-01-23 11:18:49.399  INFO 1 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
2020-01-23 11:18:49.640  WARN 1 --- [           main] i.f.k.client.internal.VersionUsageUtils  : The client is using resource type 'customresourcedefinitions' with unstable version 'v1beta1'
2020-01-23 11:18:50.464  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2020-01-23 11:18:50.532  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2020-01-23 11:18:50.534  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Started VaultApplication in 3.788 seconds (JVM running for 4.301)
2020-01-23 11:18:50.642  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret0 in namespace default
2020-01-23 11:18:50.655  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret1 in namespace default
2020-01-23 11:18:50.661  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret2 in namespace default
2020-01-23 11:18:50.667  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret3 in namespace default
2020-01-23 11:18:50.673  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret4 in namespace default
2020-01-23 11:18:50.681  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret5 in namespace default
2020-01-23 11:18:50.686  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret6 in namespace default
2020-01-23 11:18:50.694  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret7 in namespace default
2020-01-23 11:18:50.700  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret8 in namespace default
2020-01-23 11:18:50.721  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret9 in namespace default
2020-01-23 11:18:50.726  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret10 in namespace default
2020-01-23 11:18:50.739  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret11 in namespace default
2020-01-23 11:18:50.747  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret12 in namespace default
2020-01-23 11:18:50.753  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret13 in namespace default
2020-01-23 11:18:50.759  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret14 in namespace default
2020-01-23 11:18:50.773  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret15 in namespace default
2020-01-23 11:18:50.782  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret16 in namespace default
2020-01-23 11:18:50.792  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret17 in namespace default
2020-01-23 11:18:50.800  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret18 in namespace default
2020-01-23 11:18:50.808  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret19 in namespace default
2020-01-23 11:18:50.813  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret20 in namespace default
2020-01-23 11:18:50.821  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret21 in namespace default
2020-01-23 11:18:50.829  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret22 in namespace default
2020-01-23 11:18:50.835  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret23 in namespace default
2020-01-23 11:18:50.840  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret24 in namespace default
2020-01-23 11:18:50.845  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret25 in namespace default
2020-01-23 11:18:50.849  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret26 in namespace default
2020-01-23 11:18:50.857  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret27 in namespace default
2020-01-23 11:18:50.867  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret28 in namespace default
2020-01-23 11:18:50.880  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret29 in namespace default
2020-01-23 11:19:00.511  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
2020-01-23 11:19:00.660  INFO 1 --- [TaskScheduler-1] o.s.v.a.LifecycleAwareSessionManager     : Scheduling Token renewal
2020-01-23 11:19:01.088  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
2020-01-23 11:19:38.663  INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2020-01-23 11:19:38.664  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2020-01-23 11:19:38.672  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 8 ms
2020-01-23 11:19:58.160 ERROR 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Watch for custom resource failed

io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 36053598 (57536768)
	at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onMessage(WatchConnectionManager.java:254) [kubernetes-client-4.1.0.jar!/:na]
	at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:310) [okhttp-3.9.1.jar!/:na]
	at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:222) [okhttp-3.9.1.jar!/:na]
	at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:101) [okhttp-3.9.1.jar!/:na]
	at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:265) [okhttp-3.9.1.jar!/:na]
	at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:204) [okhttp-3.9.1.jar!/:na]
	at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) [okhttp-3.9.1.jar!/:na]
	at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.9.1.jar!/:na]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_201]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_201]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201]

2020-01-23 11:20:08.168  INFO 1 --- [       Thread-5] o.s.s.c.ThreadPoolTaskScheduler          : Shutting down ExecutorService 'vaultThreadPoolTaskScheduler'

Issues with Vault running HTTPS

Hi,

My vault is running on a https setup. I dont see any doc related on how to setup vault-crd with accessing vault in https.

Is there any flag which has to be set in the deployment file. Kindly help on the same.

Or is there any native path or trust store in kubernetes where i can upload the vault cert and make the connection working.

Below is the error.

kubectl logs vault-crd-756ffc95bf-vv5jb -n vault-crd

. ____ _ __ _ _
/\ / ' __ _ () __ __ _ \ \ \
( ( )__ | '_ | '| | ' / ` | \ \ \
\/ )| |)| | | | | || (| | ) ) ) )
' || .__|| ||| |_, | / / / /
=========||==============|/=////
:: Spring Boot :: (v2.2.4.RELEASE)

2020-06-08 16:59:44.315 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-756ffc95bf-vv5jb with PID 1 (/opt/vault-crd.jar started by root in /opt)
2020-06-08 16:59:44.319 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : No active profile set, falling back to default profiles: default
2020-06-08 16:59:46.423 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2020-06-08 16:59:46.440 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2020-06-08 16:59:46.440 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.30]
2020-06-08 16:59:46.512 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2020-06-08 16:59:46.512 INFO 1 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 2124 ms
2020-06-08 16:59:47.031 INFO 1 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
2020-06-08 16:59:48.820 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
2020-06-08 16:59:48.907 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2020-06-08 16:59:48.914 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Started VaultApplication in 5.197 seconds (JVM running for 5.867)
2020-06-08 16:59:49.061 INFO 1 --- [//10.96.0.1/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for sample-vault in namespace default
2020-06-08 16:59:49.212 WARN 1 --- [//10.96.0.1/...] o.s.v.a.LifecycleAwareSessionManager : Cannot enhance VaultToken to a LoginToken: Token self-lookup failed; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://192.168.142.164:8200/v1/auth/token/lookup-self": sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
2020-06-08 16:59:49.239 ERROR 1 --- [//10.96.0.1/...] d.k.vault.kubernetes.EventHandler : Failed to generate secret for vault resource sample-vault in namespace default failed with exception:

de.koudingspawn.vault.vault.communication.SecretNotAccessibleException: Couldn't communicate with vault
at de.koudingspawn.vault.vault.VaultCommunication.createPki(VaultCommunication.java:55) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.vault.impl.PkiSecretGenerator.generateSecret(PkiSecretGenerator.java:25) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.vault.VaultService.generateSecret(VaultService.java:18) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.EventHandler.addHandler(EventHandler.java:27) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:47) [classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:40) [classes!/:0.0.1-SNAPSHOT]
at io.fabric8.kubernetes.client.utils.WatcherToggle.eventReceived(WatcherToggle.java:49) [kubernetes-client-4.9.0.jar!/:na]
at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onMessage(WatchConnectionManager.java:237) [kubernetes-client-4.9.0.jar!/:na]
at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:322) [okhttp-3.14.6.jar!/:na]
at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219) [okhttp-3.14.6.jar!/:na]
at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105) [okhttp-3.14.6.jar!/:na]
at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:273) [okhttp-3.14.6.jar!/:na]
at okhttp3.internal.ws.RealWebSocket$1.onResponse(RealWebSocket.java:209) [okhttp-3.14.6.jar!/:na]
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:174) [okhttp-3.14.6.jar!/:na]
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.14.6.jar!/:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://192.168.142.164:8200/v1/pki_int/issue/example-dot-com": sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:751) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:421) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at de.koudingspawn.vault.vault.VaultCommunication.lambda$createPki$0(VaultCommunication.java:48) ~[classes!/:0.0.1-SNAPSHOT]
at org.springframework.vault.core.VaultTemplate.doWithSession(VaultTemplate.java:466) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at de.koudingspawn.vault.vault.VaultCommunication.createPki(VaultCommunication.java:48) ~[classes!/:0.0.1-SNAPSHOT]
... 17 common frames omitted
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Supress Vault secret existence checks / "Failed secret modification" K8s events

Hello, thank you for vault-crd, it's been a really useful component in our stack!

We've run into a slight issue with getting a lot of FailedModification K8s events, which are of type Failure, whereas it doesn't seem anything "wrong" is happening. It's not a big problem, but it obscures other important K8s events in our monitoring.

Repro steps:

Create KV2 secret in Vault
Deploy Vault K8s resource with corresponding Vault KV2 secret path
Remove the Vault KV2 secret, without touching .yaml definitions in K8s (Vault resource stays unchanged)

Not, every now and then, vault-crd will refresh, checking for Vault secret existence and state. If it's not there, it brings up this notification:

vault-crd/src/main/java/de/koudingspawn/vault/kubernetes/scheduler/ScheduledRefresh.java

Line 41 in 35219d5

 eventNotification.storeNewEvent(MODIFICATION_FAILED, "Modification of secret failed with exception " + e.getMessage(), resource); 

This seems sensible, since the secret is not present. However, I believe our use case is a bit less obvious.

Our use case – default secrets + override

We have multiple development environments of the same apps. For convenience, we share a _default KV2 secret in Vault for each service, which contains all default ENV VARs for the service. Then, if a developer wants to override them or add new, they can create a new Vault secret, specific to the environment. For example:

---
apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: "kv-myservice-default-secrets"
spec:
  path: "kv-myservice/_default"
  type: "KEYVALUEV2"
  changeAdjustmentCallback:
    type: deployment
    name: "myservice"
{{ end }}

# Vault secrets for env (override the default-secrets)
---
apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: "kv-myservice-env-secrets"
spec:
  path: "kv-myservice/myenvname"
  type: "KEYVALUEV2"
  changeAdjustmentCallback:
    type: deployment
    name: "myservice"

Then in Deployment, we do this:

          envFrom:
            - secretRef:
                name: "kv-myservice-default-secrets"
            - secretRef:
                name: "kv-myservice-env-secrets"
                optional: true
            {{- end }}

—

So in our situation, it seems that having a Vault resource with no correspondent Vault secret is an "acceptable" state (although only for override secrets; the _default should always have a corresponding Vault secret).

Is it possible to somehow supress these events? Perhaps I missed something in the documentation. Otherwise, would you consider extending the resource API to cover such a use case? For example:

apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: "kv-myservice-env-secrets"
spec:
  path: "kv-myservice/myenvname"
  checkVaultSecretExistence: false
  type: "KEYVALUEV2"
  changeAdjustmentCallback:
    type: deployment
    name: "myservice"

( checkVaultSecretExistence: false)

It's just an example API I've come up with now, I'm sure there is a better name.

Env info

vault-crd version: 1.6.3
k8s version: 1.24 (EKS)

k8s 1.26

Hi, we recently upgraded out k8s from 1.24 to 1.26 and now we see this error

unable to build kubernetes objects from release manifest: resource mapping not found for name: "vault-crd-pod-running-policy" namespace: "vault" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"

do you have a newer helm chart version for k8s >= 1.25 ?

Support DOCKERCFG in a version 2 KV store

It appears as though the DOCKERCFG Secret Type does not currently work with a Version 2 KV secret engine in Vault. I had to create a new version 1 KV store in Vault and put Docker credentials in it in order for vault-crd to properly create the kubernetes.io/dockercfg secret.

As a quick solution, it would be nice to just document that DOCKERCFG requires a v1 secret engine to save others the time of working through this issue. :-)

Thanks for the great project!

add support for kubernetes.io/dockerconfigjson

Hello,

I am currently using an operator which requires a .docker/config.json instead of .dockercfg.
Would that be possible to implement the secret type kubernetes.io/dockerconfigjson. It seems that it is the recommended way now and I can't find many recent mentions of kubernetes.io/dockercfg but I can't find any deprecation or reason for that either.

The main difference is the field name named .dockerconfigjson instead of .dockercfg and it contains a Docker auth token instead of username/password/email.

It would be awesome if the operator could transform username/password into a Docker Auth Token but that seems a bit harder than just pulling the token directly from Vault.

Also it might be interesting to add on the documentation that DOCKERCFG only works with KV1 or will fail with KV2 (or maybe it is not intended, if so I can try to provide more information on a separate issue).

It seems duplicating the current DOCKERCFG implementation can create a DOCKERCONFIGJSON that pulls the auth token easily. I can provide a PR for this if you want me to. However, it makes sense to support both username/password and auth token (as kubectl does, I have not checked how it is implemented there).

Best Regards,

Facing dennied access to secrets

Modifying the values of vault-crd-helm i updated the version of the vault-crd image to 1.9.1 to avoid the log4j vulnerability.

What happened:

The application starts crashing and restarting constantly showing the following errors:

Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. secrets is forbidden: User "system:serviceaccount:vault-crd:vault-crd-serviceaccount" cannot list resource "secrets" in API group "" at the cluster scope.

Message: secrets is forbidden: User "system:serviceaccount:vault-crd:vault-crd-serviceaccount" cannot watch resource "secrets" in API group "" at the cluster scope.

What you expected to happen:

Initialize the application correctly.

How to reproduce it:

Change the image tag in the values
vaultCRD.tag | Image tag | 1.9.1
Execute the helm upgrade command

helm upgrade vault-crd vault-crd/vault-crd --version 1.6.3 -f custom-values.yaml

Solution:

Add list and watch permissions to the ClusterRole resource:

- apiGroups:
      - ""
    resources:
      - secrets
    verbs:
      - get
      - create
      - patch
      - update
      - delete
      - list
      - watch

Application working:

[main] de.koudingspawn.vault.VaultApplication   : Starting VaultApplication v0.0.1-SNAPSHOT using Java 11.0.1

Vault-crd helm : vault-crd pod restarts with error "main] d.k.v.kubernetes.KubernetesConnection : Please first apply custom resource definition and then restart vault-crd"

Hi,
I am installing vault-crd using helm charts (version: 1.6.3) but after deploying vault-crd pod restarts with below error : -
. ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _ | \ \ \
\/ )| |)| | | | | || (| | ) ) ) )
' || .__|| ||| |_, | / / / /
=========||==============|/=///_/
:: Spring Boot :: (v2.2.4.RELEASE)

2023-01-24 09:29:01.280 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-6589b6c7bd-v4hdq with PID 1 (/opt/vault-crd.jar started by root in /)
2023-01-24 09:29:01.284 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : No active profile set, falling back to default profiles: default
2023-01-24 09:29:02.950 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2023-01-24 09:29:02.965 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-01-24 09:29:02.965 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.30]
2023-01-24 09:29:03.035 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2023-01-24 09:29:03.035 INFO 1 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 1688 ms
2023-01-24 09:29:03.439 INFO 1 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
2023-01-24 09:29:04.245 ERROR 1 --- [ main] d.k.v.kubernetes.KubernetesConnection : Please first apply custom resource definition and then restart vault-crd
`

if a `Secret` of the same name as a `Vault` already exists, it does not fail.

I created this Secret via kubectl apply

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: ajayg2
data:
  key1: ZnJvbXNlY3JldA==

I then create a Vault object that should use the same name.

apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: ajayg2
spec:
  type: "KEYVALUE"
  path: "secret/infra/jayg"

In the logs I see the line Received action: ADDED for ajayg2 in namespace X
but kubectl get secret -o yaml ajayg2 shows me that it has the original secret, not the one from Vault and it does not have vault.koudingspawn.de/* annotations.
If I wait for the next Refresh from vault-crd and get the yaml again, I see that it has the secret value from Vault and it has the annotations indicating that it is controlled by the CRD.

I feel this is fairly inconsistent. I think the initial create should fail because it finds a Secret that is not controlled by the CRD.
But if it is going to be added, it needs to update the Secret with the right values immediately and not wait for the first refresh pass.

copying labels creates ArgoCD synchronization problem

Copying all labels from the vault-crd into the secret makes ArgoCD think the secret is managed and therefore out-of-sync with the application definition. ArgoCD will prune (aka delete) the secret, and then vault-crd will re-created it, ad-infinitum. A possible solution is sketched in #53.

Facing permission denied

First of all, Thanks for nice project.

My vault running with backend consul with HA and I expose Vault using ingress. export VAULT_ADDR=http://vault-internal.172.31.14.138.nip.io

Error

2021-07-30 02:35:37.711  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Refresh of secret internal-new-config in namespace kube-vault failed with exception

de.koudingspawn.vault.vault.communication.SecretNotAccessibleException: Couldn't load secret from vault path internal/new/config
        at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:136) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:111) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.vault.impl.KeyValueV2Generator.getHash(KeyValueV2Generator.java:34) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.scheduler.impl.KeyValueV2Refresh.hashHasChanged(KeyValueV2Refresh.java:33) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.scheduler.impl.KeyValueV2Refresh.refreshIsNeeded(KeyValueV2Refresh.java:28) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.scheduler.ScheduledRefresh.refreshCertificates(ScheduledRefresh.java:43) ~[classes!/:0.0.1-SNAPSHOT]
        at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275]
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_275]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_275]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_275]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_275]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_275]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_275]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_275]
Caused by: org.springframework.vault.VaultException: Status 403 Forbidden [new/config]: 1 error occurred:
        * permission denied

; nested exception is org.springframework.web.client.HttpClientErrorException$Forbidden: 403 Forbidden: [{"errors":["1 error occurred:\n\t* permission denied\n\n"]}
]
        at org.springframework.vault.client.VaultResponses.buildException(VaultResponses.java:86) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
        at org.springframework.vault.core.VaultVersionedKeyValueTemplate.lambda$doRead$0(VaultVersionedKeyValueTemplate.java:118) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
        at org.springframework.vault.core.VaultTemplate.doWithSession(VaultTemplate.java:466) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
        at org.springframework.vault.core.VaultVersionedKeyValueTemplate.doRead(VaultVersionedKeyValueTemplate.java:100) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
        at org.springframework.vault.core.VaultVersionedKeyValueTemplate.get(VaultVersionedKeyValueTemplate.java:89) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
        at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:123) ~[classes!/:0.0.1-SNAPSHOT]
        ... 17 common frames omitted
Caused by: org.springframework.web.client.HttpClientErrorException$Forbidden: 403 Forbidden: [{"errors":["1 error occurred:\n\t* permission denied\n\n"]}
]
        at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:109) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:170) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:112) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:586) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
        at org.springframework.vault.core.VaultVersionedKeyValueTemplate.lambda$doRead$0(VaultVersionedKeyValueTemplate.java:103) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
        ... 21 common frames omitted

2021-07-30 02:35:37.713  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...

Steps I followed

Created Policy

cat <<EOF > policy.hcl
path "testpki/issue/testrole" {
  capabilities = ["create", "read", "update"]
}

path "secret/*" {
  capabilities = ["read"]
}
EOF
vault write sys/policy/testpolicy [email protected]
vault token create -policy=testpolicy -display-name=testtoken

RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: vault-crd-serviceaccount
  namespace: kube-vault
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: vault-crd-clusterrole
rules:
  - apiGroups:
    - apiextensions.k8s.io
    resources:
    - customresourcedefinitions
    verbs:
    - get
  - apiGroups:
      - "koudingspawn.de"
    resources:
      - vault
    verbs:
      - list
      - watch
      - get
  - apiGroups:
      - ""
    resources:
      - secrets
    verbs:
      - get
      - create
      - patch
      - update
      - delete
  - apiGroups:
      - extensions
      - apps
    resources:
      - deployments
    verbs:
      - update
      - get
      - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: vault-crd-clusterrole-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: vault-crd-clusterrole
subjects:
  - kind: ServiceAccount
    name: vault-crd-serviceaccount
    namespace: kube-vault
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: vault.koudingspawn.de
spec:
  group: koudingspawn.de
  scope: Namespaced
  names:
    plural: vault
    singular: vault
    kind: Vault
    shortNames:
      - vt
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                path:
                  type: string
                  pattern: '^.*?\/.*?(\/.*?)?$'
                type:
                  type: string
                  enum:
                    - PKI
                    - PKIJKS
                    - CERT
                    - CERTJKS
                    - DOCKERCFG
                    - KEYVALUE
                    - KEYVALUEV2
                    - PROPERTIES
                pkiConfiguration:
                  type: object
                  properties:
                    commonName:
                      type: string
                    altNames:
                      type: string
                    ipSans:
                      type: string
                    ttl:
                      type: string
                      pattern: '^[0-9]{1,}[hm]$'
                jksConfiguration:
                  type: object
                  properties:
                    password:
                      type: string
                    alias:
                      type: string
                    keyName:
                      type: string
                    caAlias:
                      type: string
                versionConfiguration:
                  type: object
                  properties:
                    version:
                      type: integer
                propertiesConfiguration:
                  type: object
                  properties:
                    context:
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                    files:
                      type: object
                      x-kubernetes-preserve-unknown-fields: true
                dockerCfgConfiguration:
                  type: object
                  properties:
                    type:
                      type: string
                      enum:
                        - KEYVALUE
                        - KEYVALUEV2
                    version:
                      type: integer
                changeAdjustmentCallback:
                  type: object
                  properties:
                    type:
                      type: string
                    name:
                      type: string
              required:
                - type
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: vault-crd
  name: vault-crd
  namespace: kube-vault
spec:
  selector:
    matchLabels:
      app: vault-crd
  replicas: 1
  template:
    metadata:
      labels:
        app: vault-crd
    spec:
      serviceAccountName: vault-crd-serviceaccount
      containers:
      - name: vault-crd
        image: daspawnw/vault-crd:1.6.4
        env:
        - name: KUBERNETES_VAULT_URL
          value: "http://vault-internal.172.31.14.138.nip.io/v1/"
        - name: KUBERNETES_VAULT_TOKEN
          value: "s.sOARfxb4nDTsUCKmAVc379ru"
        ports:
          - containerPort: 8080
        livenessProbe:
          httpGet:
            port: 8080
            path: "/actuator/health"
          initialDelaySeconds: 30
          failureThreshold: 3
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 5
      restartPolicy: Always
---

Create secrets

vault kv put internal/new/config username="db-readonly-username" 
cat <<EOF > vault-crd-secrets-new.yaml
apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: internal-new-config
spec:
  path: "internal/new/config"
  type: "KEYVALUEV2"
  versionConfiguration:
    version: 1
EOF
kubectl create -f vault-crd-secrets-new.yaml

Please let me know if I missing!!

Vault Token in ENV var

Neat implementation! Suggest that you move the KUBERNETES_VAULT_TOKEN from a static definition on the ENV of the deployment to a separate secret file that the deployment mounts in with secretKeyRef. Anyone with the ability to get/list deployments will be able to steal the vault token and get all the secrets it has access to. Fewer roles/clusterroles allow get/list of secrets than deployments, so this improves the access control surrounding that sensitive token.

        env:
        - name: KUBERNETES_VAULT_TOKEN
          valueFrom:
            secretKeyRef:
              name: vault-crd-secret-name
              key: "token"

Sync between k8s and Vault

Hello,
I created my secrets on an external vault, and configured the authentication between vault-crd (helm chart) and the external vault. The secrets are not created on my Kubernetes cluster, please advise if I need a vault server enterprise or how can I troubleshoot the problem, I didn't get any error in the vault-crd logs.

Regards,

vault serviceAccount authentication failing on vault-crd-chart 1.6.1

values.yaml:
# Specifies the used authentication method the following values are allowed: token | serviceAccount
vaultAuth: serviceAccount
# Token with access to the resources that Vault-CRD shares from Vault to Kubernetes. Required if vaultAuth = token
vaultToken: ""
# Path to authentication backend in HashiCorp Vault. Only used if vaultAuth = serviceAccount
vaultAuthPath: auth/dev-kubernetes
vaultRole: "dev-role"

I have vault sidecars installed in the cluster and working with service accounts with these same credentials.

2021-02-05 16:27:31.758 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Refresh of secret assets-cred in namespace dnext failed with exception

de.koudingspawn.vault.vault.communication.SecretNotAccessibleException: Couldn't load secret from vault path dev-kv/gitlab/credentials/registry/high-five
at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:136) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.vault.VaultCommunication.getDockerCfg(VaultCommunication.java:67) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.vault.impl.DockerCfgGenerator.getHash(DockerCfgGenerator.java:35) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.scheduler.impl.DockerCfgRefresh.dockerCfgHashHasChanged(DockerCfgRefresh.java:36) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.scheduler.impl.DockerCfgRefresh.refreshIsNeeded(DockerCfgRefresh.java:30) ~[classes!/:0.0.1-SNAPSHOT]
at de.koudingspawn.vault.kubernetes.scheduler.ScheduledRefresh.refreshCertificates(ScheduledRefresh.java:43) ~[classes!/:0.0.1-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_275]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_275]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_275]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_275]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_275]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_275]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_275]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_275]
Caused by: org.springframework.vault.authentication.VaultLoginException: Cannot login using Kubernetes: missing client token; nested exception is org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: [{"errors":["missing client token"]}
]
at org.springframework.vault.authentication.VaultLoginException.create(VaultLoginException.java:64) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.authentication.KubernetesAuthentication.login(KubernetesAuthentication.java:107) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.authentication.LifecycleAwareSessionManager.doGetSessionToken(LifecycleAwareSessionManager.java:291) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.authentication.LifecycleAwareSessionManager.getSessionToken(LifecycleAwareSessionManager.java:277) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultTemplate.lambda$getSessionInterceptor$1(VaultTemplate.java:276) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.vault.client.VaultClients.lambda$createRestTemplate$0(VaultClients.java:128) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:77) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:742) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:586) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.lambda$doRead$0(VaultVersionedKeyValueTemplate.java:103) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultTemplate.doWithSession(VaultTemplate.java:466) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.doRead(VaultVersionedKeyValueTemplate.java:100) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueTemplate.get(VaultVersionedKeyValueTemplate.java:89) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at org.springframework.vault.core.VaultVersionedKeyValueOperations.get(VaultVersionedKeyValueOperations.java:72) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:125) ~[classes!/:0.0.1-SNAPSHOT]
... 18 common frames omitted
Caused by: org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: [{"errors":["missing client token"]}
]
at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:101) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:170) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:112) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:785) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:743) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:677) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:421) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
at org.springframework.vault.authentication.KubernetesAuthentication.login(KubernetesAuthentication.java:96) ~[spring-vault-core-2.2.1.RELEASE.jar!/:2.2.1.RELEASE]
... 36 common frames omitted

vault-crd sometimes crashes with nullpointer exception

Hi, we're currently experiencing the following issue from time to time. Doesn't happen too often but sometimes this happens:

2018-08-13 12:09:29.213  INFO 7 --- [pool-1-thread-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
2018-08-13 12:09:29.257  INFO 7 --- [pool-1-thread-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
2018-08-13 12:10:29.213  INFO 7 --- [pool-1-thread-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
2018-08-13 12:10:29.923 ERROR 7 --- [pool-1-thread-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task.

java.lang.NullPointerException: null
        at de.koudingspawn.vault.kubernetes.KubernetesService.modifySecret(KubernetesService.java:58) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.EventHandler.modifyHandler(EventHandler.java:44) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.scheduler.ScheduledRefresh.refreshCertificates(ScheduledRefresh.java:42) ~[classes!/:0.0.1-SNAPSHOT]
        at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_171]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]

(this continues for every request afterwards...)

Simply killing the pod so that it gets recreated gets things back going again.

Any idea what this could be?

Btw: is there any chance (beside a sidecar constantly requesting a new cert) to implement a healtcheck?
Apart from this vault-crd is running really fine! :)

If you need more logs just drop me a line...

best regards,
Bjoern

specific vault tokens do not work

k8s: 1.14.8
daspawnw/vault-crd:1.4.1

vault tokens in the below format do not work.

Example: s.eByyHGuJOTtxvPllisdtnJz

java.lang.IllegalArgumentException: Unexpected char 0x0a at 26 in X-Vault-Token value: s.eByyHGuJOTtxvPllisdtnJz

Add support mount multiple secrets

Hi..
When vault come into different path for applying some policy for different departement like database credentials, apps configuration and others, We need to merge into one environment configuration in kubernetes apps.
Rather than create more than one vault kind for each path, I think this can be done by mount multiple path of vault in one kind vault-crd then combine it into one secrets, then mount it into kubernetes apps environment for only single secrets

Error with CERTJKS file

Hey, I am new to vault. I met some problems while uploading my Jks file to vault and then load it to K8s secret. Here is what I did:

upload jks to vault:

 vault kv put secret/XXX [email protected]

create vault resources in k8s.

apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: test-certjks
spec:
  path: "secret/XXX"
  type: "CERTJKS"

the secret is not created in vault. and vault pod is throwing this error:

2020-02-21 21:59:21.240  WARN 1 --- [/172.20.0.1/...] i.f.k.c.d.i.WatchConnectionManager       : Exec Failure

java.lang.NullPointerException: null
    at de.koudingspawn.vault.vault.impl.SharedVaultResponseMapper.getPublicKey(SharedVaultResponseMapper.java:164) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.impl.SharedVaultResponseMapper.mapJks(SharedVaultResponseMapper.java:95) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.impl.CertJksGenerator.generateSecret(CertJksGenerator.java:26) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.VaultService.generateSecret(VaultService.java:18) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.EventHandler.addHandler(EventHandler.java:27) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:38) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:31) ~[classes!/:0.0.1-SNAPSHOT]
    at io.fabric8.kubernetes.client.utils.WatcherToggle.eventReceived(WatcherToggle.java:49) ~[kubernetes-client-4.1.0.jar!/:na]
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onMessage(WatchConnectionManager.java:232) ~[kubernetes-client-4.1.0.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:310) ~[okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:222) ~[okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:101) ~[okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:265) ~[okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:204) ~[okhttp-3.9.1.jar!/:na]
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.9.1.jar!/:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]

I am not sure if I used the right command to upload jks file into vault. I also tried:

vault write secret/*** [email protected]

I got the error:

Error writing data to secret/XX: Error making API request.


Code: 404. Errors:


WARNING! The following warnings were returned from Vault:

  * Invalid path for a versioned K/V secrets engine. See the API docs for the
  appropriate API endpoints to use. If using the Vault CLI, use 'vault kv put'
  for this operation.

Could you please help me with this?

[Question] Does creating K8s secrets store them in ETCD

Hello!

I skimmed through the documentation and got that this project creates Kubernetes Secrets from Vault secrets.
Yet, I have not found out if the Kubernetes Secrets created through this process are stored in ETCD or there is some special handling for them.

Feature request - Link to chart and use gh-pages for helm repo

Firstly This is a life saver, thank you.

Please document in the main readme.md of this project that there is a helm repo. Please include the link
In the helm chart repo, consider using the helm-chart-releaser or similar and the gh-pages. raw.githubusercontent is a little messy

Note for the install people can use

helm upgrade vault-crd vault-crd \
    --repo https://raw.githubusercontent.com/DaspawnW/vault-crd-helm/master \
    --namespace vault-crd \
    --set vaultCRD.vaultUrl=http://localhost:8080/v1/ \
    --set vaultCRD.vaultAuth=serviceAccount \
    --set vaultCRD.vaultRole=test \
    --install

That way the command is consistent if they are doing an install or an upgrade
With the helm charts releaser the url of the charts repo would be https://daspawnw.github.io/vault-crd-helm

Support for kv backend version 2

I'd love to see support for version 2 of the kv backend, I'd really like to use the versioning functionality that it added.

getting `SecretNotAccessibleException` when trying to create a "KEYVALUEV2"

vault kv get secret-v2/infra/jayg
works from a command line as does
vault kv get -version=1 secret-v2/infra/jayg

trying to apply this yaml

apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: jayg-v2
spec:
  type: "KEYVALUEV2"
  path: "secret-v2/infra/jayg"
  versionConfiguration:
    version: 1

results in this traceback

de.koudingspawn.vault.vault.communication.SecretNotAccessibleException: The secret secret-v2/infra/jayg is not available or in the wrong format.
    at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:122) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.impl.KeyValueV2Generator.generateSecret(KeyValueV2Generator.java:27) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.VaultService.generateSecret(VaultService.java:18) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.EventHandler.addHandler(EventHandler.java:27) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:38) [classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:31) [classes!/:0.0.1-SNAPSHOT]
    at io.fabric8.kubernetes.client.utils.WatcherToggle.eventReceived(WatcherToggle.java:49) [kubernetes-client-4.1.0.jar!/:na]
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onMessage(WatchConnectionManager.java:232) [kubernetes-client-4.1.0.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:310) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:222) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:101) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:265) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:204) [okhttp-3.9.1.jar!/:na]
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.9.1.jar!/:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]de.koudingspawn.vault.vault.communication.SecretNotAccessibleException: The secret secret-v2/infra/jayg is not available or in the wrong format.
    at de.koudingspawn.vault.vault.VaultCommunication.getVersionedSecret(VaultCommunication.java:122) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.impl.KeyValueV2Generator.generateSecret(KeyValueV2Generator.java:27) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.vault.VaultService.generateSecret(VaultService.java:18) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.EventHandler.addHandler(EventHandler.java:27) ~[classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:38) [classes!/:0.0.1-SNAPSHOT]
    at de.koudingspawn.vault.kubernetes.Watcher$1.eventReceived(Watcher.java:31) [classes!/:0.0.1-SNAPSHOT]
    at io.fabric8.kubernetes.client.utils.WatcherToggle.eventReceived(WatcherToggle.java:49) [kubernetes-client-4.1.0.jar!/:na]
    at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onMessage(WatchConnectionManager.java:232) [kubernetes-client-4.1.0.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:310) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:222) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:101) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:265) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:204) [okhttp-3.9.1.jar!/:na]
    at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) [okhttp-3.9.1.jar!/:na]
    at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.9.1.jar!/:na]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]

I am able to use vault-crd for a non-versioned secrets engine in vault so it is installed correctly and can talk to vault. I am using the same token in vault-crd as I am on the command line.
I'm sure I'm doing something wrong but I'm not sure what.

Regression: vault resources not syncing to k8s secrets

Expected behavior: vault-crd creates and deletes kubernetes secrets in the cluster when Vault resources are created or deleted.

Observed behavior:

vault-crd doesn't sync newly created Vault resources with the vault server to create a kubernetes secret.
On deletion of a Vault resource, its respective kubernetes secret is not deleted.
Output from the modules d.koudingspawn.vault.kubernetes.Watcher and d.k.vault.kubernetes.KubernetesService no longer appear in logs

Versions: 1.3.2, or 1.4.0 (crd update was applied during deployment of 1.4.0). Tested with kubernetes clusters on versions 1.12.8 and 1.15.6. Same behavior occurs in both clusters.

This might be a regression introduced in version 1.3.2, present in 1.4.0 as well.

Logs from vault version 1.4.0 (demonstrates incorrect behavior):

+ vault-crd-5595bf88d-mngb5 › vault-crd
vault-crd-5595bf88d-mngb5 vault-crd 
vault-crd-5595bf88d-mngb5 vault-crd   .   ____          _            __ _ _
vault-crd-5595bf88d-mngb5 vault-crd  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
vault-crd-5595bf88d-mngb5 vault-crd ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
vault-crd-5595bf88d-mngb5 vault-crd  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
vault-crd-5595bf88d-mngb5 vault-crd   '  |____| .__|_| |_|_| |_\__, | / / / /
vault-crd-5595bf88d-mngb5 vault-crd  =========|_|==============|___/=/_/_/_/
vault-crd-5595bf88d-mngb5 vault-crd  :: Spring Boot ::        (v2.2.4.RELEASE)
vault-crd-5595bf88d-mngb5 vault-crd 
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:18.016  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-5595bf88d-mngb5 with PID 1 (/opt/vault-crd.jar started by root in /opt)
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:18.019  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : No active profile set, falling back to default profiles: default
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.181  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.193  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.193  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.30]
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.251  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.252  INFO 1 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1190 ms
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.595  INFO 1 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:20.738  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:20.791  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:20.793  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Started VaultApplication in 3.104 seconds (JVM running for 3.53)
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:30.791  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.413  INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.413  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.418  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 5 ms
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.623  INFO 1 --- [nio-8080-exec-1] o.s.v.a.LifecycleAwareSessionManager     : Scheduling Token renewal
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:30.784  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:54:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:54:30.783  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:55:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:55:30.784  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:56:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:56:30.781  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:57:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:57:30.784  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:58:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:58:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:59:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:59:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:00:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:00:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:01:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:01:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:02:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:02:30.781  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:03:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:03:30.788  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:04:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:04:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:05:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:05:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:06:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:06:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:07:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:07:30.781  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:08:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:08:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:09:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:09:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:10:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:10:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:11:30.774  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:11:30.780  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...

Logs from vault version 1.3.1 (demonstrates correct behavior):

+ vault-crd-586bb99858-kg9zc › vault-crd
vault-crd-586bb99858-kg9zc vault-crd
vault-crd-586bb99858-kg9zc vault-crd   .   ____          _            __ _ _
vault-crd-586bb99858-kg9zc vault-crd  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
vault-crd-586bb99858-kg9zc vault-crd ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
vault-crd-586bb99858-kg9zc vault-crd  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
vault-crd-586bb99858-kg9zc vault-crd   '  |____| .__|_| |_|_| |_\__, | / / / /
vault-crd-586bb99858-kg9zc vault-crd  =========|_|==============|___/=/_/_/_/
vault-crd-586bb99858-kg9zc vault-crd  :: Spring Boot ::        (v2.1.1.RELEASE)
vault-crd-586bb99858-kg9zc vault-crd
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:16.616  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-586bb99858-kg9zc with PID 1 (/opt/vault-crd.jar started by root in /opt)
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:16.619  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : No active profile set, falling back to default profiles: default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.058  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.086  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.087  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/9.0.13
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.099  INFO 1 --- [           main] o.a.catalina.core.AprLifecycleListener   : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.178  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.178  INFO 1 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1519 ms
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.559  INFO 1 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.779  WARN 1 --- [           main] i.f.k.client.internal.VersionUsageUtils  : The client is using resource type 'customresourcedefinitions' with unstable version 'v1beta1'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.529  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.591  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.593  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Started VaultApplication in 3.304 seconds (JVM running for 3.761)
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.693  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret0 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.702  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret1 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.716  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret2 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.721  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret3 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.726  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret4 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.729  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret5 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.738  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret6 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.744  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret7 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.748  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret8 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.758  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret9 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.762  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret10 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.771  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret11 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.777  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret12 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.782  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret13 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.787  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret14 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.794  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret15 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.797  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret16 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.801  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret17 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.805  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret18 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.809  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret19 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.812  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret20 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.820  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret21 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.824  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret22 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.834  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret23 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.838  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret24 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.841  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret25 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:29.566  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:29.695  INFO 1 --- [TaskScheduler-1] o.s.v.a.LifecycleAwareSessionManager     : Scheduling Token renewal
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:30.084  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:35.683  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:35.730  INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService   : Created secret for vault resource secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:02.649  INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:02.650  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:02.658  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 7 ms
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:29.925  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:07:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:07:29.902  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:08:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:08:29.902  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:09:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:34:29.869  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:35:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:35:29.872  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:36:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:36:29.879  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:37:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:37:29.867  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:38:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:38:29.953  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:39:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:39:29.868  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:00.305  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: DELETED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:00.317  INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService   : Deleted secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:29.860  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:18.063  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: ADDED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:18.088  INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService   : Created secret for vault resource secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:29.913  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:42:29.568  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:42:29.853  INFO 1 --- [TaskScheduler-1] d.k.vault.kubernetes.KubernetesService   : Modified secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:42:30.084  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:43:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:43:29.865  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:44:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:44:29.903  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:45:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:45:29.876  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:29.886  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:34.800  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: MODIFIED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:34.827  INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService   : Modified secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:28.099  INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher  : Received action: DELETED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:28.107  INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService   : Deleted secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:29.565  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:29.897  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...

Unable to run application on Kubernetes 1.22.2

Im trying to deploy vault-crd to a new kubernetes cluster with version 1.22.2

I follow the guide at https://vault.koudingspawn.de/install-vault-crd#static-vault-token but when I deploy the rbac.yml to kubernetes the application gets stuck in CrashLooBackOff mode with the error:


  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.4.RELEASE)

2021-10-25 15:34:15.950  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-5f47d4cf75-k9tx6 with PID 1 (/opt/vault-crd.jar started by root in /)
2021-10-25 15:34:15.954  INFO 1 --- [           main] de.koudingspawn.vault.VaultApplication   : No active profile set, falling back to default profiles: default
2021-10-25 15:34:17.615  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2021-10-25 15:34:17.630  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-10-25 15:34:17.630  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.30]
2021-10-25 15:34:17.700  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-10-25 15:34:17.700  INFO 1 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1689 ms
2021-10-25 15:34:18.081  INFO 1 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
2021-10-25 15:34:18.916 ERROR 1 --- [           main] d.k.v.kubernetes.KubernetesConnection    : Please first apply custom resource definition and then restart vault-crd

Have also tried to manually deploy the CRD before running rbac.yml as well without success.

I have validated that the crd is in fact deployed and that the clusterrole has access to it.

kubectl get crd vault.koudingspawn.de
NAME                    CREATED AT
vault.koudingspawn.de   2021-10-25T15:33:16Z

kubectl auth can-i get customresourcedefinitions --as system:serviceaccount:vault-crd:vault-crd-serviceaccount
yes

Run multiple vault on same cluster in different namespaces

Hi There, It's amazing CRD i am using it and working well.

i have a requirement like, want to run this CRD for two vault.

My one vault is in default namespace and secret in the default namespace.

Now i have set up new vault in dev namespace and looking forward to sync secrets in dev namespace it is possible with single deployment or i have to do multiple deployments of CRD.

Dynamic Secrets

Hello - Is there any plans for adding dynamic secret tracking to vault-crd? We currently use this and are liking the model so far but the fact that its limited to static secrets doesn't allow us to unlock the full potential of vault. I work for Under Armour and its possible we could dedicate some engineering effort to adding this feature if you are willing to accept an implementation or have ideas on how you'd like it implement.

Thanks!

if the generated `Secret` of a `Vault` object is accidentally deleted, it causes an Exception during refresh handling

for testing I created and applied

apiVersion: "koudingspawn.de/v1"
kind: Vault
metadata:
  name: jayg
spec:
  type: "KEYVALUE"
  path: "secret/infra/jayg"

and a corresponding Secret was created.

I copied the jayg Vault and changed the name to jayg2 and applied and all was well.

I then deleted the jayg2 Secret (but not the Vault object) from our cluster via
kubectl delete -f jayg2.yaml

on the next refresh pass there was a NullExceptionPointer in the vault-crd logs

2019-03-18 19:46:00.850  INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
2019-03-18 19:46:01.337 ERROR 1 --- [TaskScheduler-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task.

java.lang.NullPointerException: null
   at de.koudingspawn.vault.kubernetes.KubernetesService.modifySecret(KubernetesService.java:58) ~[classes!/:0.0.1-SNAPSHOT]
   at de.koudingspawn.vault.kubernetes.EventHandler.modifyHandler(EventHandler.java:44) ~[classes!/:0.0.1-SNAPSHOT]
   at de.koudingspawn.vault.kubernetes.scheduler.ScheduledRefresh.refreshCertificates(ScheduledRefresh.java:42) ~[classes!/:0.0.1-SNAPSHOT]
   at sun.reflect.GeneratedMethodAccessor65.invoke(Unknown Source) ~[na:na]
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_181]
   at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_181]
   at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
   at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_181]
   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_181]
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_181]
   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_181]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]

I probed a bit by adding more Vault objects with different names. The refresh process continued to work for those that were sorted before the deleted one. (e.g. jayg continued to update from vault as secrets were changed as well as a new ajayg Vault object)
But anything sorted after the deleted secret did not get updated secrets from Vault secret changes. (e.g. jayg2 Secret never gets re-created. jayg3 will get a Secret created based on the value in vault at the time it was applied but will never get updated via the Refresh process after a Vault secret value update.
(I am not sure if it actually processes them alphabetically but I created this scenario to test whether one 'bad' refresh error stops processing for the rest of the Vault objects.)

Too old resource version

Hi :

I'm using vault-crd 1.4.1 version on aws eks cluster right now, when I was used aws eks 1.14.9 with vault-crd 1.4.1 everything works perfectly. but when I upgrade my aws eks to 1.15.10, vault-crd 1.4.1 start crashing and show the following message :

| 2020-03-11T04:42:26.010628826Z 2020-03-11 04:42:26.007 ERROR 1 --- [/172.20.0.1/...] d.koudingspawn.vault.kubernetes.Watcher  : Watch for custom │
│  resource failed                                                                                                                                 │
│ 2020-03-11T04:42:26.010653996Z                                                                                                                   │
│ 2020-03-11T04:42:26.010659185Z io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 47851836 (71336772)             │
│ 2020-03-11T04:42:26.010663896Z     at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onMessage(WatchConnectionManager.java:2 │
│ 63) [kubernetes-client-4.6.4.jar!/:na]                                                                                                           │
│ 2020-03-11T04:42:26.010668776Z     at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:322) [okhttp-3.14.6.jar!/:na]           │
│ 2020-03-11T04:42:26.010674226Z     at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219) [okhttp-3.14.6.jar!/:na]    │
│ 2020-03-11T04:42:26.010678096Z     at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105) [okhttp-3.14.6.jar!/:na]    │
│ 2020-03-11T04:42:26.010682966Z     at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.java:273) [okhttp-3.14.6.jar!/:na]              │
│ 2020-03-11T04:42:26.010686886Z     at okhttp3.internal.ws.RealWebSocket$1.onResponse(RealWebSocket.java:209) [okhttp-3.14.6.jar!/:na]            │
│ 2020-03-11T04:42:26.010691186Z     at okhttp3.RealCall$AsyncCall.execute(RealCall.java:174) [okhttp-3.14.6.jar!/:na]                             │
│ 2020-03-11T04:42:26.010695836Z     at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [okhttp-3.14.6.jar!/:na]                         │
│ 2020-03-11T04:42:26.010708666Z     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]             │
│ 2020-03-11T04:42:26.010712877Z     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]             │
│ 2020-03-11T04:42:26.010716496Z     at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]                                                       │
│ 2020-03-11T04:42:26.010720107Z                                                                                                                   │
│ 2020-03-11T04:42:36.319008222Z 2020-03-11 04:42:36.318  INFO 1 --- [extShutdownHook] o.s.s.c.ThreadPoolTaskScheduler          : Shutting down Ex │
│ ecutorService 'vaultThreadPoolTaskScheduler'

but if I downgrade vault-crd to 1.3.2 with aws eks 1.15.10, and it wouldn't crash, but it doesn't do anything.

can you help to take a look? it looks like #28

Thank you

Support for LOG_LEVEL configuration (other than "info")

We're using vault-crd in both production and nonproduction services. This lead us to creating quite a lot of separate environments, and thus instances where vault-crd is used.

We're currently generating around 100k logs per day with vault-crd, most of which are INFO logs like this:

2023-03-28 16:25:29.422  INFO 1 --- [ool-1-thread-74] d.koudingspawn.vault.kubernetes.Watcher  : Received scheduled refresh for .. in namespace ...

Here's our Datadog logs view:

We're now paying around $78/mo just for these logs... 😅 Yeah, I know, Datadog is expensive... but we're not in a position to easily change it.

Q: Is it possible to switch the LOG_LEVEL to ERROR?

(Helm chart) PodSecurityPolicy was deprecated in Kubernetes v1.21, and removed from Kubernetes in v1.25

Hello,

It seems that there is an issue with your helm chart which installs vault-crd. If the kubernetes cluster is greater then 1.25, the chart will fail to install with the following error message.

❯ helm install vault-crd vault-crd/vault-crd --set vaultCRD.vaultToken=hvs.xxxxxxxxxxxxxxxxxxx --set vaultCRD.vaultUrl=https://vault.example.com:8200
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: resource mapping not found for name: "vault-crd-pod-running-policy" namespace: "default" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"
ensure CRDs are installed first

After doing some research, it seems that they are deprecated in Kubernetes v1.21, and then removed from Kubernetes in v1.25.

I am currently running version kubernetes version 1.28.xx, so it looks like the charts need some upgrades to make it function correctly.

https://kubernetes.io/docs/concepts/security/pod-security-policy/

Enable https scheme for Vault

Hi, I was wondering if it was possible to use a -tls-skip-verify flag or pass the CA certificate for Vault through an environment variable?

If I target an https endpoint for Vault (which is using a self-signed certificate), I get the following error in the stack trace:

org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://x.x.x.x:8200/v1/auth/token/lookup-self": sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The Deployment config is:

env:
        - name: KUBERNETES_VAULT_URL
          value: "https://x.x.x.x:8200/v1/"
        - name: KUBERNETES_VAULT_TOKEN
          value: "45WSJG5RRcu51pgnAa3B59F0"

Cannot login I/O error on POST request for "https://VAULT_HOST:8200/v1/auth/kubernetes/login"

Hello, I'm configuring vault-crd version 1.11.0 with vault 1.12.0-1, and several times I tried to update the certificate used in authentication, and the error below persists.

vault-crd org.springframework.vault.authentication.VaultLoginException: Cannot login using org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://VAULT_HOST:8200/v1/auth/kubernetes/login": PKIX path building failed: sun.security.provider.certpa ││ th.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The process I'm running is the KeyStore import with my certificates that are stored in the k8s secret, as shown in the example below.

apiVersion: v1
kind: Secret
metadata:
  name: root-ca
type: Opaque
data:
  root-ca.pem: valid-base64-certificate

This certificate is self-signed and created using terraform. Below when I extract the certificate with the command

openssl x509 -noout -in vault.crt -text

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            SERIAL_NUMBER
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = BR, ST = SAo-Paulo, L = Campinas, O = CONTOSO, OU = IT, CN = INTERMEDIATE_CN
        Validity
            Not Before: Oct 27 23:59:43 2022 GMT
            Not After : Oct 26 23:59:43 2025 GMT
        Subject: C = BR, ST = Sao-Paulo, L = campinas, O = CONTOSO, OU = IT, CN = ROOT_CN
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:

                Exponent: (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment, Key Agreement, Certificate Sign, CRL Sign
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                keyid:

            X509v3 Subject Alternative Name:
                DNS:PRIVATE_DNS_HOSTNAME, DNS:localhost, IP Address:127.0.0.1, IP Address:INTERNAL_IP
    Signature Algorithm:

The vault service is live and follows the entire process of the tutorial below.

https://vault.koudingspawn.de/install-vault-crd/self-signed-certificates

Even though every process is running successfully, the problem persists. Could you please help me?

Vault Version: vault/bionic,now 1.12.0-1 amd64
SO Version: 18.04.6 LTS (Bionic Beaver)
Vault CRD: 1.11
EKS: v1.22.13-eks-15b7512

The service account, cluster role, cluster role binding, was created from the link below:

https://raw.githubusercontent.com/DaspawnW/vault-crd/master/deploy/rbac.yaml

Deployment YAML:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: vault-crd
  name: vault-crd
  namespace: default
spec:
  selector:
    matchLabels:
      app: vault-crd
  replicas: 1
  template:
    metadata:
      labels:
        app: vault-crd
    spec:
      initContainers:
        - name: cert-import
          image: openjdk:8
          command:
            - /bin/bash
          args:
            - -c
            - keytool -importcert -noprompt -trustcacerts -alias root-ca -file /certs/root-ca.pem -keystore /etc/ssl/certs/java/cacerts -storepass changeit &&
              keytool -importcert -noprompt -trustcacerts -alias int-ca -file /certs/int-ca.pem -keystore /etc/ssl/certs/java/cacerts -storepass changeit
          volumeMounts:
          - mountPath: /certs
            name: vault-certs
            readOnly: true
          - mountPath: /etc/ssl/certs/java
            name: cacerts
      serviceAccountName: vault-auth-develop
      serviceAccount: vault-auth-develop
      containers:
      - name: vault-crd
        image: daspawnw/vault-crd:1.11.0
        env:
        - name: KUBERNETES_VAULT_URL
          value: https://PRIVATE_DNS_HOSTNAME:8200/v1/
        - name: KUBERNETES_VAULT_ROLE
          value: develop
        - name: KUBERNETES_VAULT_AUTH
          value: serviceAccount
        - name: KUBERNETES_VAULT_PATH
          value: kubernetes-develop
        ports:
          - containerPort: 8080
        volumeMounts:
        - mountPath: /certs
          name: vault-certs
          readOnly: true
        - mountPath: /etc/ssl/certs/java
          name: cacerts
      volumes:
        - name: vault-certs
          projected:
            defaultMode: 420
            sources:
            - secret:
                items:
                - key: root-ca.pem
                  path: root-ca.pem
                name: root-ca
            - secret:
                items:
                - key: int-ca.pem
                  path: int-ca.pem
                name: int-ca
        - name: cacerts 
        livenessProbe:
          httpGet:
            port: 8080
            path: "/actuator/health"
            scheme: HTTP
          initialDelaySeconds: 30
          failureThreshold: 3
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 5
      restartPolicy: Always

[Question] How do you actually renew the token?

Hi, this is more a question than an issue itself.

I'm trying to figure out how would you renew the token, it's not clear to me. What happens if the token expires and then pod gets deleted. Do I need to re-create the deployment with a new token?

On the other hand I'd like to know if you would be open to login using AppRole auth method instead of a token-based login.

Thanks!!

Errors on documentation

I couldn't find anywhere to create a merge request for the documentation.

https://vault.koudingspawn.de/install-vault-crd
base64 -D should be base64 -d

KUBERNETES_VAULT_PATH is not described in the documentation but implemented in /src/main/java/de/koudingspawn/vault/vault/VaultConfiguration.java:62.
A simple description could be:
Please specify here the Vault auth path to be used. Defaults to kubernetes.

Support for multiple kubernetes clusters

We are using one Vault for multiple kubernetes clusters.
It is configured by passing path argument to vault auth enable command.

vault auth enable --path="kube-tst" kubernetes

Looks like, it is not supported in vault-crd currently and all requests are always send to /v1/auth/kubernetes/ endpoint.
In this case path is /v1/auth/kube-tst/
It would be nice if we could override "kubernetes" path by providing env variable.

Resource version too old

hello,

even with 1.4.2i get the error.

io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 9055242 (9091399)

Deploy vault-crd per namespace

is it possible to deploy a vault-crd per namespace? The reason I'm looking for this is to limit a deployment from being able to retrieve a secret in vault that they shouldn't.

such as dev deployments using vault-crd shouldn't be able see prod secrets