Comments (5)
ctalledo
commented on August 22, 2024
2
Thanks @kylecarbs for filing the issue, I was able to easily reproduce it.
Looks like the problem is that the rpm package manager running inside the inner redhat container is trying to chown /proc to root:root. I found this by stracing the bash shell inside the redhat container:
18385 chown("/proc", 0, 0) = -1 EPERM (Operation not permitted)
Doing a chown of /proc to root:root should normally be fine (though it seems unnecessary). However, when running inside a sysbox container the operation fails with EPERM:
root@86940b46acc8:/home/admin# chown 0:0 /proc
chown: changing ownership of '/proc': Operation not permitted
The reason for this failure is that inside the sysbox container, /proc is owned by the host's root user rather than by the container's root user (recall sysbox containers use the linux user-namespace, so root in container != root in host). This is why it shows up as nobody:nogroup inside the container:
root@86940b46acc8:/home/admin# ls -l / | grep proc
dr-xr-xr-x 261 nobody nogroup 0 Nov 21 21:19 proc
Ideally, inside the sysbox container /proc would show up as root:root (and chown should work), but it's a hard problem to solve when using the linux user-namespace inside the container. We need to think carefully here, we don't have a solution right now.
Workaround
There is a work-around, which tells the rpm package manager inside the redhat container to not change the ownership of "/proc". It uses the rpm's %_netsharedpath config, described here.
For example:
# this tells rpm to not touch /proc and /sys
[root@0080cb671377 /]# echo "%_netsharedpath /sys:/proc" >> /etc/rpm/macros.dist
# this now works ...
[root@0080cb671377 /]# rpm -Uvh /var/cache/dnf/ubi-8-baseos-53c30a88cff3796c/packages/filesystem-3.8-3.el8.x86_64.rpm --force
Verifying... ################################# [100%]
Preparing... ################################# [100%]
Updating / installing...
1:filesystem-3.8-3.el8 ################################# [ 50%]
Cleaning up / removing...
2:filesystem-3.8-2.el8 ################################# [100%]from sysbox.
ctalledo
commented on August 22, 2024
2
FYI, this same issue has been seen in unprivileged LXC containers, with no solution to date:
https://bugzilla.redhat.com/show_bug.cgi?id=1589968
I am confident Sysbox can solve it, change is out for code-review right now.
from sysbox.
ctalledo
commented on August 22, 2024
1
Modified the title of this issue to better describe the problem.
from sysbox.
ctalledo
commented on August 22, 2024
1
FYI: Rodny confirmed the same problem occurs in a Podman unprivileged container:
[vagrant@fedora-31-vm wsp]$ podman run -it registry.access.redhat.com/ubi8/ubi:8.2-347 /bin/bash
[root@aa64e996386d /]# dnf install filesystem -y --downloadonly
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Red Hat Universal Base Image 8 (RPMs) - BaseOS 2.0 MB/s | 771 kB 00:00
Red Hat Universal Base Image 8 (RPMs) - AppStream 6.4 MB/s | 4.8 MB 00:00
Red Hat Universal Base Image 8 (RPMs) - CodeReady Builder 45 kB/s | 13 kB 00:00
Package filesystem-3.8-2.el8.x86_64 is already installed.
Dependencies resolved.
===========================================================================================================================================================================================================================
Package Architecture Version Repository Size
===========================================================================================================================================================================================================================
Upgrading:
filesystem x86_64 3.8-3.el8 ubi-8-baseos 1.1 M
Transaction Summary
===========================================================================================================================================================================================================================
Upgrade 1 Package
Total download size: 1.1 M
DNF will only download packages for the transaction.
Downloading Packages:
filesystem-3.8-3.el8.x86_64.rpm 4.7 MB/s | 1.1 MB 00:00
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 4.6 MB/s | 1.1 MB 00:00
Complete!
The downloaded packages were saved in cache until the next successful transaction.
You can remove cached packages by executing 'dnf clean packages'.
[root@aa64e996386d /]#
[root@aa64e996386d /]# rpm -Uvh /var/cache/dnf/ubi-8-baseos-53c30a88cff3796c/packages/filesystem-3.8-3.el8.x86_64.rpm --force
Verifying... ################################# [100%]
Preparing... ################################# [100%]
Updating / installing...
1:filesystem-3.8-3.el8 ################################# [ 50%]
error: unpacking of archive failed on file /proc: cpio: chown failed - No such file or directory
error: filesystem-3.8-3.el8.x86_64: install failed
error: filesystem-3.8-2.el8.x86_64: erase skipped
from sysbox.
ctalledo
commented on August 22, 2024
1
Hi @kylecarbs,
We were able to fix this, the change has been committed.
sysbox-runc PR 19
sysbox-fs PR 13
Closing!
Please re-open if you hit any issues.
from sysbox.
Related Issues (20)
- Add custom CRI-O configuration for k8s installation HOT 3
- K8s projected volumes don't work when using systemd inside the container HOT 18
- File change inside container will be reset after host reboot. HOT 2
- podman/buildah cannot list nestybox+cri-o containers HOT 2
- AKS sysbox nodes sporadically enter bad state (`stat failed on /var/lib/containers/storage/overlay/...`) HOT 16
- docker: Error response from daemon: unknown or invalid runtime name: sysbox-runc. HOT 1
- AKS sysbox container missing /proc/sys/net/ipv4/neigh/default HOT 8
- No such file or directory - unprivileged_userns_clone HOT 8
- Sysbox doesn't work with estargz compressed images
- Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? HOT 6
- EKS 1.28 won't install sysbox through sysbox-deploy-k8s daemonset HOT 5
- Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? HOT 3
- content of /proc/sys/kernel/random/uuid is always the same HOT 2
- Error in supervisord.conf of jenkins exemplary docker file HOT 2
- Kubernetes 1.29 support HOT 1
- failed to create sysbox-mgr: ID-mapping check failed on ZFS HOT 3
- can't build sysbox-in-docker image. `sysbox-runc` binary not available HOT 2
- writing /etc/resolv.conf caused: open /etc/resolv.conf: permission denied: unknown without "userns-remap":"sysbox"; kernel version is fine HOT 4
- sysbox k8s directory mounted as nobody HOT 9
- ingress networking in docker swarm in dind not working.. it's working fine on the docker host HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sysbox.