Add support for Sysbox PODs about sysbox HOT 2 CLOSED

Sysbox always uses the Linux user-ns in the containers/pods it creates. It's a must-have for proper functionality & isolation.

In order for K8s to create pods with Sysbox, the user-ns associated with the pod must be created before the network ns (and all other namespaces too) associated with that pod.

This can't be done by Sysbox itself, because per the K8s CRI spec, it's the CRI implementation (e.g., dockershim, containerd, or cri-o) that sequences this.

I've done some research and found that the CRI-O runtime manager has initial support for enabling the user-ns in pods. That is, upstream versions of CRI-O are capable of creating the user-ns for a pod first, then create the remaining ns as required.

At a high level, the way this will works is:

1. The user creates a pod with an annotation indicating use of the "user-ns" and sets runtimeClass to the Sysbox runtime.
2. k8s deploys that pod on a node
3. The k8s kubelet on that node talks to cri-o to create the pod with user-ns and sysbox
4. CRI-O creates the user-ns and network-ns, then tells sysbox to create the containers for the pod
5. Sysbox creates the containers with inside the pod

Other than CRI-O, I don't believe the other CRI implementations (dockershim and containerd) support user-ns functionality. dockershim is in fact out of the question because it does not even support the runtimeClass spec required for K8s to deploy containers with Sysbox. There is work happening in Containerd to support user-ns too, but it's unlikely to come before mid 2021 (per talking to the containerd maintainers at KubeCon).

Thus, it looks like an initial implementation of K8s + Sysbox will require CRI-O on the K8s node(s).

I am working on this right now.

from sysbox.

This task is now complete.

As of Sysbox v0.4.0 (released on 07/14/21), it's now possible to use Kubernetes + Sysbox to deploy secure (rootless) pods capable of running microservices or VM-workloads (e.g., systemd, Docker, even K8s itself inside the pod).

Installation is done via a daemonset called sysbox-deploy-k8s. See here for instructions.

Currently, Sysbox requires CRI-O as the container manager on each worker node (i.e., it does not yet work with containerd, see prior comment).

The sysbox-deploy-k8s daemonset installs CRI-O on the worker node and configures the Kubelet to use CRI-O instead of the existing container manager (typically Docker or containerd).

Once containerd is updated to support pods with user-ns (expected in 2022), Sysbox will be adjusted to work with containerd too (at which point CRI-O won't be a requirement).

Closing!

from sysbox.