nestybox / sysbox-ee Goto Github PK

I'm unsure what I'm doing wrong here, but missing a shiftfs module.

root@system:~# docker run --runtime=sysbox-runc --rm -it --hostname my_cont debian:latest
docker: Error response from daemon: OCI runtime create failed: container requires user-ID shifting but error was found: shiftfs module is not loaded in the kernel. Update your kernel to include shiftfs module or enable Docker with userns-remap. Refer to the Sysbox troubleshooting guide for more info: unknown.
ERRO[0000] error waiting for container: context canceled```

I'm not seeing any license in this repository.

I'm on Ubuntu 19.10 and installed without issue the bionic .deb.
Now, when I run docker run --runtime=sysbox-runc ... I get the following:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:364: starting container process caused "process_linux.go:474: container init caused \"process_linux.go:441: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: time=\\\\\\\"2020-02-01T13:15:25+01:00\\\\\\\" level=fatal msg=\\\\\\\"dial unix /var/run/docker/libnetwork/b1aea14fa6b47a6f800f04a76d29b6fab16f9a9176a9c5a3a7a0259f801457ed.sock: connect: connection refused\\\\\\\"\\\\n\\\"\"": unknown.
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Then docker doesn't work anymore at all (even with the default runtime) and service command neither:

sudo service docker restart
Failed to connect to bus: Connection refused
sudo service apache2 restart
Failed to connect to bus: Connection refused

The only way to get things up is to restart the computer.

Btw, while installing the deb, a sudo service docker restart was required for docker to detect the new runtime. This is missing in your docs.

Hi, I encountered the following issue when attempting to pull the openjdk image within a container using the sysbox-runc runtime,

# docker pull openjdk
Using default tag: latest
latest: Pulling from library/openjdk
977461c90301: Pull complete 
38d4dba4c275: Extracting [==================================================>]   14.8MB/14.8MB
2b949418b5d5: Download complete 
failed to register layer: Error processing tar file(exit status 1): operation not permitted

This was run inside a container started as follows,

docker run --runtime=sysbox-runc --rm --name nestybox-docker -it nestybox/alpine-docker

dockerd was started from another window via docker exec nestybox-docker dockerd -D.

I was able to reproduce the same error message building this smaller Dockerfile,

FROM oraclelinux:7-slim
RUN rm -rf /var/cache/yum

HI, mi distro is ubuntu based Pop!_OS 20.04 not work.

Testing this out, I was trying to limit the resources on the system container and see the effect inside the system container.

I tried passing the docker run option flags like --cpus=1 on launching the system container, but once inside, I still see 2 CPUs.

Same trying to use -m or --memory: this seems to have no effect.

How I can define/limit the resources for the system container?

1. cat docker-compose.yml

version: '3.7'
services:
test-node:
container_name: test
image: nestybox/ubuntu-bionic-systemd-docker:latest
restart: always
tty: true
stdin_open: true
runtime: sysbox-runc
command:
- /bin/bash
- -c
- >
touch /testfile

2. run cd / && ls -a on the container:
   bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var

docker run -it --runtime=sysbox-runc --privileged --restart=always -p 9200:9000 nestybox/ubuntu-bionic-systemd-docker:latest

1. running log:
   [ OK ] Started Dispatch Password Requests to Console Directory Watch.
   [ OK ] Reached target Remote File Systems.
   [ OK ] Started Forward Password Requests to Wall Directory Watch.
   [ OK ] Created slice User and Session Slice.
   [ OK ] Reached target Paths.
   [ OK ] Reached target Swap.
   [ OK ] Created slice System Slice.
   [ OK ] Listening on udev Kernel Socket.
   [ OK ] Listening on udev Control Socket.
   [ OK ] Listening on /dev/initctl Compatibility Named Pipe.
   proc-sys-fs-binfmt_misc.automount: Failed to initialize automounter: Operation not permitted
   [FAILED] Failed to set up automount Arbitrary Executable File Formats File System Automount Point.
   See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details.

2. run systemctl status proc-sys-fs-binfmt_misc.automount on the container:
   ● proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point
   Loaded: loaded (/lib/systemd/system/proc-sys-fs-binfmt_misc.automount; static; vendor preset: enabled)
   Active: failed (Result: resources)
   Where: /proc/sys/fs/binfmt_misc
   Docs: https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
   https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems

3. run docker info on host machine :
   Client:
   Debug Mode: false
   Server:
   Containers: 2
   Running: 2
   Paused: 0
   Stopped: 0
   Images: 57
   Server Version: 19.03.4
   Storage Driver: overlay2
   Backing Filesystem: extfs
   Supports d_type: true
   Native Overlay Diff: true
   Logging Driver: json-file
   Cgroup Driver: cgroupfs
   Plugins:
   Volume: local
   Network: bridge host ipvlan macvlan null overlay
   Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
   Swarm: inactive
   Runtimes: runc sysbox-runc
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
   runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
   init version: fec3683
   Security Options:
   apparmor
   seccomp
   Profile: default
   Kernel Version: 5.4.0-72-generic
   Operating System: Ubuntu 18.04.3 LTS
   OSType: linux
   Architecture: x86_64
   CPUs: 4
   Total Memory: 7.645GiB
   Name: sanzenwin
   ID: 2YUT:QA42:UWRF:X2M7:LTJY:TJN7:SCND:BY24:QRJZ:VJ5X:R4PC:Z5YY
   Docker Root Dir: /var/lib/docker
   Debug Mode: false
   Registry: https://index.docker.io/v1/
   Labels:
   Experimental: false
   Insecure Registries:
   127.0.0.0/8
   Registry Mirrors:
   https://ustc-edu-cn.mirror.aliyuncs.com/
   Live Restore Enabled: false
   WARNING: No swap limit support

4. run uname -a on host machine:
   Linux ***** 5.4.0-72-generic #80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Hello

I tried to make the sysbox runtime work for our Jenkins setup but it ends up crashing (the whole Docker daemon).

System information

• AWS EC2 Instance
• Ubuntu 18.04 Server (ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20200112 (ami-0b418580298265d5c))
• Upgraded kernel as documented via backports (uname -r = 5.3.0-40-generic)

Jenkins Volumes (run directly on host using regular Docker runtime)

• /var/run/docker.sock:/var/run/docker.sock
• jenkins-workspace:/var/jenkins_home

Jenkins Pipeline

pipeline {
    ...
    agent {
        docker {
            image 'censored:latest'
            args  '--runtime=sysbox-runc -v jenkins-maven-repository:/home/jenkins/.m2'
        }
    }
    ...
}

For completness, this is what Jenkins tries to run:

docker run -t -d -u 1000:1000 --runtime=sysbox-runc -v jenkins-maven-repository:/home/jenkins/.m2 -w /var/jenkins_home/jobs/maven-pipeline-ng/workspace --volumes-from 9f65cddc6f6cf5951e78cc7bbd67f64040b5877919c8860d0d550a1c6a3905cf -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** censored:latest cat

So basically the Jenkins container (which is a regular one) should start pipeline agents with sysbox runtime -> Agents will be siblings of Jenkins but be able to have isolated Docker inside

This leads to the following error however:

Failed to run image 'censored:latest'. Error: docker: Error response from daemon: OCI runtime create failed: container_linux.go:364: starting container process caused "process_linux.go:474: container init caused \"process_linux.go:441: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: time=\\\\\\\"2020-03-06T13:50:22Z\\\\\\\" level=fatal msg=\\\\\\\"dial unix /var/run/docker/libnetwork/c0d26181f7af.sock: connect: connection refused\\\\\\\"\\\\n\\\"\"": unknown.

The whole server (not only Docker, literally everything) starts to behave strange afterwards and the only solution is to perform a reboot via AWS console (doesn't really work too). I.e. running reboot -h now results in:

Failed to connect to bus: Connection refused
Failed to open /dev/initctl: No such device or address
Failed to talk to init daemon.

Manually creating censored:latest containers within the Jenkins container works and Docker within such a container works as well when done without -v jenkins-maven-repository:/home/jenkins/.m2 -w /var/jenkins_home/jobs/maven-pipeline-ng/workspace --volumes-from 9f65cddc6f6cf5951e78cc7bbd67f64040b5877919c8860d0d550a1c6a3905cf -e flags - so the problem must be with these.

Currently access to the server is not possible (stuck and un-stoppable) so I cannot further break down which of the flags alone is a problem (maybe --volumes-from?). I will try once I regain access to the server, but hopefully the provided information is enough anyways.

Thanks.

FYI:

I installed on Ubuntu 19.04 server from the live CD on VirtualBox.
The installer gives an option to install Docker during installation of Ubuntu

Docker ends up under /var/snap/docker/384/run and the config is under /var/snap/docker/384/config

I am guessing this is why the daemon.json config file wasn't properly updated to include the runtime.

after updating the daemon.json config file with the path to the sysbox-runc runtime, the issue was resolved.