containersolutions / registry-tooling Goto Github PK

kube-registry.kube-system.svc.cluster.local:31000 is a bit much to type.

The script should be easily installable.

Probably the simplest solution is to make the script standalone by moving the k8s yaml files into here docs in the script.

When starting

./reg-tool.sh install-k8s-reg

The scripts stays in the 'Waiting for job to complete' phase indefinitely.

The logs of the create-certs-20cm2 pod show:

Error from server (Forbidden): User "system:serviceaccount:default:default" cannot delete secrets in the namespace "default". (delete secrets registry-cert)
Error from server (Forbidden): User "system:serviceaccount:default:default" cannot delete secrets in the namespace "kube-system". (delete secrets registry-cert)
Error from server (Forbidden): User "system:serviceaccount:default:default" cannot delete secrets in the namespace "kube-system". (delete secrets registry-key)
Generating a 4096 bit RSA private key
...................................++
.................++
writing new private key to 'certs/domain.key'
-----
Error from server (Forbidden): User "system:serviceaccount:default:default" cannot create secrets in the namespace "default". (post secrets)

If I could pass my ~/.kube/config file with authentication and certificates to the create_certs and copy_certs containers, the operation would succeed.

The registry listens on port 5000 by default. If we changed this to match the nodeport, it would allow pods to talk to the registry on the same address.

Rather than create our own cert, we should ask kubernetes for one (now that we can do that).

https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/

There seems to be a bug that means the cert isn't being copied to the minikube VM. Not clear why the volume mount isn't working.

The images include a statically linked version of kubectl. This won't work for some people who have different versions of k8s. A better, cleaner, solution would be to use the API.

$ cat /etc/*-release file.
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
[..]

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"2016-12-14T00:57:05Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1+coreos.0", GitCommit:"cc65f5321f9230bf9a3fa171155c1213d6e3480e", GitTreeState:"clean", BuildDate:"2016-12-14T04:08:28Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

$ ./reg-tool.sh install-k8s-reg

Installs a Docker registry in your Kubernetes cluster and configures it
[...]
Do you want to continue? (y/n) Y

Tidying up any old registry jobs

Creating new registry certificate
job "create-certs" created

Waiting for job to complete.............................
Copying certs to nodes
The Job "copy-certs-172.17.4.99" is invalid: spec.template.spec.containers[0].name: Invalid value: "copy-certs-172.17.4.99": must match the regex a-z0-9? (e.g. 'my-name' or '123-abc')

My k8s master had a taint applied that prevented your script from running on this node.

I added to the temporary file you create:

tolerations:
        - key: "node-role.kubernetes.io/master"
          operator: "Exists"
          effect: "NoSchedule"

This allows the pod to be scheduled on the master, as well.

I'm getting these errors in the create-certs pod:

╰ 21:44:49 $ kubectl logs create-certs-f24ng 
Error from server (Forbidden): secrets "registry-cert" is forbidden: User "system:serviceaccount:default:default" cannot delete resource "secrets" in API group "" in the namespace "default"
Error from server (Forbidden): secrets "registry-cert" is forbidden: User "system:serviceaccount:default:default" cannot delete resource "secrets" in API group "" in the namespace "kube-system"
Error from server (Forbidden): secrets "registry-key" is forbidden: User "system:serviceaccount:default:default" cannot delete resource "secrets" in API group "" in the namespace "kube-system"
Generating a 4096 bit RSA private key
.................................................................................++
..............................................................................................................................................++
writing new private key to 'certs/domain.key'
-----
Error from server (Forbidden): secrets is forbidden: User "system:serviceaccount:default:default" cannot create resource "secrets" in API group "" in the namespace "default"

I don't see any mention of "rbac" in the script, so I'm guessing it was written before RBAC was a thing.

To reproduce the error

$ minikube start --kubernetes-version 1.6.0
...

$ ./reg-tool.sh install-k8s-reg
...

$ kubectl get pods --show-all
NAME                                                    READY     STATUS             RESTARTS   AGE
copy-certs-497761da-1a50-11e7-a9d0-080027dc1105-1wlgn   0/1       CrashLoopBackOff   3          1m
create-certs-3s3m3                                      0/1       Completed          0          1m

$ kubectl logs copy-certs-497761da-1a50-11e7-a9d0-080027dc1105-1wlgn
copying certs
error: group map[apps:0xc82038eb60 autoscaling:0xc82038ec40 batch:0xc82038ecb0 extensions:0xc82038ed90 policy:0xc82038ee00 rbac.authorization.k8s.io:0xc82038ee70 :0xc82038eaf0 authorization.k8s.io:0xc82038ebd0 componentconfig:0xc82038ed20 authentication.k8s.io:0xc82038ef50 federation:0xc82038ea80] is already registered

Versions

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.1", GitCommit:"b0b7a323cc5a4a2019b2e9520c21c7830b7f708e", GitTreeState:"clean", BuildDate:"2017-04-03T23:37:53Z", GoVersion:"go1.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.0", GitCommit:"fff5156092b56e6bd60fff75aad4dc9de6b6ef37", GitTreeState:"dirty", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.7", Compiler:"gc", Platform:"linux/amd64"}