Today we are creating an entry for every node (SPIRE agent) to register a workload registrar

This would reduce the number of sidecars, but it would require a container to recycle every time the secret has been modified or revoked.

Currently when claims don't match the CA, we return < HTTP/1.1 503 SERVICE UNAVAILABLE, but it should be 403 Forbidden

The utility secret-maker.sh needs to support pods with multiple images. Perhaps sort them alphabetically?

We should at some point figure out what code we can reuse and change it so that each different service is just made out of something like a bash script + dockerfile, and all the python code is all generic

TI-KeyRelease/components/jss/gen-jwt.py

https://github.com/IBM/trusted-service-identity/blob/master/components/jss/gen-jwt.py#L23

 ./gen-jwt.py  --aud foo,bar --claims=email:[email protected]|images:img1,img2 key.pem 

Certain parameters that should be able to be toggled live should be put in configmaps instead of as helm values.

Recreate the test files

The test fails and it prevents the build. Temporarily suppress the error. This needs to be looked at.
https://github.com/IBM/trusted-service-identity/blob/img-repo/webhook_test.go#L393-L394

Consider using /var/tsi-secure

Provide a list of namespaces to support TSI secret injection. [ 'trusted-identity', 'my-secure-namespace',.... ]

As a workaround, the TSI can be installed individually on each namespace, but since they all would be using the same intermediate key per node, we need to investigate potential conflicts (when the public interfaces are open and closed). Alternatively we can add the namespace to the host path.

No tokens are generated.

"The service_account_whitelist configurable is deprecated and will be removed in a future release. Please use service_account_allow_list instead." external=false plugin_name=k8s_psat plugin_type=NodeAttestor subsystem_name=catalog

There are still several references for the old naming conventions TI. They all need to be changed to TSI

This needs to be better organized to guarantee the proper sequence of images. Perhaps put images in a hash map and list them out lexicographically.

Images are sorted here, to be displayed in claims:
https://github.com/IBM/trusted-service-identity/blob/images/components/tsi-util/secret-maker.sh#L189-L203

And here is the secret maker that reads the yaml input files:
https://github.com/IBM/trusted-service-identity/blob/images/components/tsi-util/secret-maker.sh#L189-L203

With #36, intermediate CA subject alt names are checked as part of the claims prefixed by TSI:. This is done for certs part of the chain but which are not the root CA. There may be a usecase where this is useful and can be implemented by extending the claims dictionary population to the root CA as well.

During the trusted bootstrapping process, the values like "region", "cluster-name", "datacenter" should be injected to CSR, so they can be validated during the vault-plugin verification. Throw an error if the claims in JWT don't match the values in intermediate CA

Operator managing the application should not have access to the namespace where TSI components are running. This prevents any malicious or careless modifications to the TSI enviroment. Make sure the appliction can be deployed in a different namespace.

https://github.com/IBM/trusted-service-identity/security/dependabot/go.mod/github.com%2Fhashicorp%2Fvault/open

The tests should fail but still shows success. There are references to old IBM github.

We should spawn both processes and wait on either, if any of them return the sidecar should fail immediately, or try and restart either. Unfortunately, this is not easy in bash.

https://github.com/IBM/trusted-service-identity/blob/master/components/jwt-sidecar/run-sidecar.sh#L3-L4

Documentation needed on running TSI with OpenShift

We make sure the pod identity contains image(s) for all the containers and initContainers, sha256 encoded, however, we don't protect the identity based on the provided commands. Users might add/modify the CMD values without affecting the identity.

We should add also the encoded value that contains the following info for all the containers in the pod:

image: ubuntu@sha256:250cc6f3f3ffc5cdaa9d8f4946ac79821aafb4d3afc93928f0de9336eba21aa4
command: [ "/bin/bash", "-c", "--" ]
args: [ "while true; do ls /tsi-secrets; sleep 5; done;" ]

There are several scripts with instances of running a command and using the output without testing it, these commands usually don't error but should be resilent due to changes in API such as oc ..., the CLI may change in the future and the scripts should be able to report failures and errors to make it easier to catch issues down the road.

https://newreleases.io/project/github/spiffe/spire/release/v1.0.0
SPIRE CLI syntax changed

We should have something that validates the values separately just to make sure if aws_iid, azure_msi, etc are valid and at least gives an error message if it is not set up right rather than continuing silently

During the Cluster setup:
helm install charts/tsi-node-setup --debug --name tsi-setup --set reset.all=true

following errors show up in the container log:

pod tsi-setup-tsi-node-setup-pgnsw connected to host kube-fra02-cr034e871e697e4f2abff69102f2d792b0-w2.cloud.ibm machineid: a00917949efe4757bc4f5eb08bd6d69a
rm: cannot remove '/host/tsi-secure': Device or resource busy
full reset performed
rm: cannot remove '/host/tsi-secure/x5c': No such file or directory
x5c file reset performed
directory /host/tsi-secure/sockets created
Generating RSA private key, 2048 bit long modulus (2 primes)
............+++++
....+++++
e is 65537 (0x010001)
Can't load /root/.rnd into RNG
140476636512704:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/root/.rnd

Since we move the Vault secret retrieval to the sidecar and the main containers does not get the JWT anymore, we need to update/verify all the test cases are still relavant:

e.g. expectedMutateInitialization.json "mountPath": "/usr/share/secrets"

Since it is no longer jwt, we should change the volume name to reflect that instead of jwt-shared -data

Review JSS libraries that use outdated crypto. We probably don't need them anymore

Showing the following error:

Normal SuccessfulDelete 23m daemonset-controller Deleted pod: vtpm2-server-gbgjc Warning FailedCreate 4m27s (x21 over 22m) daemonset-controller Error creating: admission webhook "tsi-mutate.trusted-service-identity.ibm" denied the request: Using hostPath Volume with '/tsi-secure' is not allowed

In order to protect namespaces and hosts, we should also inspect the deployments in addition to pods, and returns the failure on the level of the deployment if violates the rules.

These lines cause problems now and need to be investigated:

webhook_test.go

Cleanup and remove stale code for injecting certificates for each new pod. These are the following components gen-vault-cert and revorker, update the webhook and all the configuration files, configmaps, helm charts and the documentation

The default command suggestions by gen-jwt.py doesn't work because it contains pipes that are not escaped. In addition, --expire doesnt seem to be used anymore and TTL_SEC env variable is used instead. In addition, a few other fields are ignored - like subject, iss, etc..

usage: gen-jwt.py [-h] [-iss ISS] [-aud AUD] [-sub SUB] [-claims CLAIMS]
                  [-jwks JWKS] [-expire EXPIRE]
                  key

Python script generates a JWT signed with custom private key.
issuer(iss) is passed as env. var. (ISS)
token expiration is passed as env. var (TTL_SEC)

Example:
./gen-jwt.py  --aud foo,bar --claims=email:[email protected]|images:img1,img2 key.pem

Suggestions to add quotes:

gen-jwt.py  --aud foo,bar --claims='email:[email protected]|images:img1,img2' key.pem

Since we moved to software signing service, we abandoned the existing vTPM solution. Let's bring it back to make it easier to integrate with hardware TPM once is available.

Currently, our Vault client is using HTTP to communicate with Vault. This should be done via HTTPS

Remove dependency on tiller and set a stricter helm version requirement. Tiller is generally not something that is desired in the cluster, so we should try to remove dependency on it.

in charts/ti-key-release-2/templates/jss-server.yaml

hostPort: 5000
volumeMounts:
- mountPath: /host/tsi-secure
name: tsi-secure

We should have 2 different directories, one for private key and CSR, and another one for x5c. Then we can do the first one always RO for all JSS and keep x5c R/W for JSS PUB
The public need to be R/W because it needs to inject the x5c, but the private one should be RO

Kubernetes jobs require all the container to complete the job in order to get status Completed, however TSI sidecar runs continuously indefinitely. One solution is to run disable the sidecar, by only executing the init container PR #59 however, we should consider notifying the sidecar that container has ended, so it can exit as well.

From Kubernetes 1.18, if all normal containers have reached a terminal state (Succeeded for restartPolicy=OnFailure, or Succeeded/Failed for restartPolicy=Never), then all sidecar containers will be sent a SIGTERM.

Once created, the annotation should not be updated by external configuration updates.

annotations:
    admission.trusted.identity/inject: "true"
    admission.trusted.identity/status: injected
    admission.trusted.identity/ti-cluster-name: ti-test1
    admission.trusted.identity/ti-cluster-region: eu-de
    admission.trusted.identity/ti-images: ubuntu:18.04@sha256:250cc6f3f3ffc5cdaa9d8f4946ac79821aafb4d3afc93928f0de9336eba21aa4
    admission.trusted.identity/ti-secret-key: ti-secret-274e1101-55be-0374-ae4b-5fc47cb34368

Current cryptography library 2.3 is not supported. Upgrade to 3.2 is required. As a result, python must be updated as well, since the cryptography 3.2 cannot be run with python 2.7

/usr/lib/python2.7/site-packages/jwcrypto/jwa.py:8: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.

see PR #73 and #74

Vault plugin is an integral part of the TSI solution so it should be moved into components. Use only the example and demo files in the old location.

Our install on OpenShift should check if the client is using helm version 2. Using helm v3 would cause an error

skipping unknown hook: "crd-install"