Setup and document Minikube for local development about fontbakery-dashboard HOT 7 CLOSED

One of the most annoying things with Minikube is establishing a local docker registry, where k8s can pull images from. We want to have it locally, so that we don't have to push and then pull images through the internet via a e.g. google hosted docker registry server.

This solution creates a registry server within the minikube vm images and persists the registry data between reboots of minikube minikube stop; minikube start:

set up the docker registry

# we'll always have to start minikube with the `--insecure-registry` option
$ minikube start --insecure-registry localhost:5000

# this will make the local `docker` command use the minikube vm as docker host
# thus all commands starting with `docker` will affect the minikube vm not the host computer
$ eval $(minikube docker-env)

# "registry:2" is: https://github.com/docker/distribution
#  `/data` within minikube is persisted, as documented per:
# https://github.com/kubernetes/minikube/blob/master/docs/persistent_volumes.md#persistent-volumes
docker run -d -p 5000:5000 --restart=always --name registry   -v /data/docker-registry:/var/lib/registry registry:2

That's it, now we can use docker to build images, then tag them like in this example:

docker tag rethinkdb-2.3.5 localhost:5000/fontbakery/rethinkdb-2.3.5
docker push localhost:5000/fontbakery/rethinkdb-2.3.5

To pull them in k8s, the yml spec.containers.image key looks like this:

spec:
  containers:
  - image: localhost:5000/fontbakery/rethinkdb-2.3.5
    name: rethinkdb 

After the registry setup, starting minikube always involves:

$ minikube start --insecure-registry localhost:5000
# AND *important*
$ eval $(minikube docker-env)

from fontbakery-dashboard.

For local development, I'm using a k8s namespace "fontbakery".

$ kubectl create namespace fontbakery

Despite the documentation stating:

For clusters with a few to tens of users, you should not need to create or think about namespaces at all.

The reason is that I can easily delete the whole namespace and thus wipe out all state in it. This is really interesting for development, e.g. when working on how to bootstrap the cluster from zero and resetting is needed frequently.

delete with:

$ kubectl delete namespace fontbakery

If you try this with the default namespace you get:

$ kubectl delete namespace default
Error from server (Forbidden): namespaces "default" is forbidden: this namespace may not be deleted

Thus, you'd have to clean up the items of the namespace by hand.

To make kubectl use the namespace:

$  kubectl config set-context $(kubectl config current-context) --namespace=fontbakery

NOTE: It looks like this config setting does not persist across minikube reboots!

Alternatively, kubectl can be used with --namespace=fontbakery on every kubectl command, like:

$ kubectl --namespace=fontbakery get pods

from fontbakery-dashboard.

Here are some more commands from the category good to know

# get a shell in the minikube vm:
$ minikube ssh

# this can be used to inspect the docker registry storage, so you know it works:
# within the minikube vm:
$ ls /data/docker-registry/docker/registry/v2/repositories/fontbakery/
rethinkdb-2.3.5

List all pods:

kubectl get pods

get a shell in a pod

kubectl  exec -it rethinkdb-rc-k1n4d -- /bin/bash

We can also get a shell in any of the running docker containers directly via docker. This requires that the docker host is the minikube vm, i.e. $ eval $(minikube docker-env) was executed in the current shell.

# by name
$ docker exec -it registry /bin/sh
# Instead of "registry", this could also be a docker id.
# Use `docker ps` to get a list of all containers with id and name.

more nicer docker env:

before I was setting up the docker env with:

$ eval $(minikube docker-env)

but I actually think, for bash users this is nicer:

$ source <(minikube docker-env)

or, even shorter:

$ . <(minikube docker-env)

from fontbakery-dashboard.

Get the logs of a pod live:

kubectl  --namespace=fontbakery  logs -f www-fontbakery-3811584483-6mpgg 

from fontbakery-dashboard.

• when developing in minikube
• while using $ . <(minikube docker-env)
• when minikube is up and running

It's sometimes good to run:

$ docker rmi $(docker images -q)

to remove unused docker images and release disk space.

IMPORTANT: minikube must be running, so that we don't delete docker images that are currently used. I wonder at this point if we even need a local docker registry when we build with minikube docker-env.

from fontbakery-dashboard.

I'm reading the comments in this issue history as well as the fontbakery-dashboard/docs/minikube-help.md file on the dragndrop branch in order to re-establish a local setup on my computer for addressing #45

@graphicore Let me know if you are aware of anything here (or there) that may have become obsolete.

from fontbakery-dashboard.

For #45 just don't! Make it an docker image that runs fontbakery without errors, don't care about minikube/kubernetes. Read the docker documentation instead.

from fontbakery-dashboard.