Git Product home page Git Product logo

oci-seccomp-bpf-hook's Introduction

Build Status

oci-seccomp-bpf-hook

This project provides an OCI hook to generate seccomp profiles by tracing the syscalls made by the container. The generated profile would allow all the syscalls made and deny every other syscall.

The syscalls are traced by launching a binary by using the prestart OCI hook. The binary started spawns a child process which attaches function enter_trace to the raw_syscalls:sys_enter tracepoint using eBPF. The function looks at all the syscalls made on the system and writes the syscalls which have the same PID namespace as the container to the perf buffer. The perf buffer is read by the process in the userspace and generates a seccomp profile when the container exits.

There are a few limitations to this approach:

  • Needs CAP_SYS_ADMIN to run
  • Compiles C code on the fly
  • Cannot use podman run --rm along with this ability

To build it, we need extra dependencies namely bcc-devel and kernel-headers for Fedora and bcc-tools and linux-headers-[..] for Ubuntu.

Interface:

sudo podman run --annotation io.containers.trace-syscall="if:[absolute path to the input file];of:[absolute path to the output file]" IMAGE COMMAND

The profile will be created at the output path provided to the annotation. Providing of: is mandatory, while if: is optional. An input file can be used to create a baseline and newly recorded syscalls will be added to the set and written to the output. If a syscall is blocked in the base profile, then it will remain blocked in the output file even if it is recorded while tracing.

Please refer to an article on Enable Sysadmin for more details.

Copyright {2018-2022} {containers/oci-seccomp-bpf-hook maintainers}

oci-seccomp-bpf-hook's People

Contributors

cavokz avatar cevich avatar edsantiago avatar fgiloux avatar giuseppe avatar jnovy avatar lawndoc avatar lsm5 avatar martinetd avatar renovate[bot] avatar rhafer avatar rhatdan avatar saschagrunert avatar tomsweeneyredhat avatar vrothberg avatar weirdwiz avatar wt-l00 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

oci-seccomp-bpf-hook's Issues

Start versioning

We should start versioning to have some kind of release engineering but also to cover maintenance cases.

  • Add a --version flag to hook to allow for inspecting the version.
  • Add a VERSION file to the repository to ease packaging and compiling in the version.
  • Use semantic versioning.

Tool needs to support input and output files.

Perhaps

sudo podman run --annotation io.podman.trace-syscall=if:/tmp/seccomp-old.json,of:/tmp/seccomp.json  IMAGE COMMAND

Annotation becomes a comma separate list of options to the plugin.

sudo podman run --annotation io.podman.trace-syscall=if:/tmp/seccomp.json,of:/tmp/seccomp.json  IMAGE COMMAND

Would update the existing seccomp.json file.

sudo podman run --annotation io.podman.trace-syscall=of:/tmp/seccomp.json  IMAGE COMMAND

Should fail if the /tmp/seccomp.json file exists.

Better names for the env variables?

Hi

I was trying to compile on Fedora 32 silverblue and had this funny thing: the compilation was failing and complaining on the link command having wrong arguments. Looking at the Makefile there are a few env variables that get defined if not already set, especially:
VERSION ?= $(shell cat ./VERSION)
The thing is that this variable is defined per default with "32 (Container Image)" on my system, replacing the value with the content of the VERSION file solved the issue.
Would it make sense to use variables with a prefix, something like OCISBH_VERSION for avoiding this?

Regards,

Frédéric

Consider using libbpf

If we utilize libbpf, then we can produce a smaller binary which also runs faster and minimizes the runtime dependencies. The overall architecture of the hook could be simplified as well. I created a syscall recorder project for demonstration purposes: https://github.com/saschagrunert/syscall-recorder

Building the main application (the syscall-recorder) requires bpftool, clang, llvm, libbpf, libelf, libz and libseccomp (for converting the syscall IDs to names). Statically linking is now also possible.

For my demo and to keep things simple, I decided to not fork within the recorder and build a small wrapper around systemd-run: https://github.com/saschagrunert/syscall-recorder/blob/main/hack/oci-hook/hook.go. Right now the recorder is not able to produce a full seccomp profile, but writes a list of syscalls to the target location.

demo

What are your thoughts on that?

Can't generate a seccomp profile

I used this blog and followed the steps in it to generate a container seccomp profile, and I don't get a file, ls.json, output from running this sudo podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:32 ls / > /dev/null

I installed a fresh x86 fedora 32 server using this Fedora-Server-netinst-x86_64-32-1.6.iso on a vm

  • then installed like this
sudo dnf install -y podman
sudo dnf install -y bcc-devel bcc-tools git golang libseccomp-devel golang-github-cpuguy83-md2man make
git clone https://github.com/containers/oci-seccomp-bpf-hook.git
cd oci-seccomp-bpf-hook
make binary
sudo make install

I've tried this on several other fedora 32 vm's I had running, and I don't seem to be able to generate the profile. I might be missing something simple, but I don't know where else to look. Sorry if I'm doing something obviously wrong. Thanks.

Here is the command and the output of the journal

sudo podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:32 ls / > /dev/null

root@seccomp ~]# journalctl --since '2 minutes ago' | grep seccomp                                                                                                                           
Sep 01 17:31:12 seccomp audit[32794]: USER_ACCT pid=32794 uid=0 auid=1013 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix,pam_localu
ser acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'                                                                                                        
Sep 01 17:31:12 seccomp sudo[32794]:     root : TTY=pts/0 ; PWD=/root ; USER=root ; COMMAND=/usr/bin/podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:32 ls         
Sep 01 17:31:12 seccomp audit[32794]: USER_CMD pid=32794 uid=0 auid=1013 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/root" cmd=706F646D616E2072756E202D2D616E6
E6F746174696F6E20696F2E636F6E7461696E6572732E74726163652D73797363616C6C3D6F663A2F746D702F6C732E6A736F6E206665646F72613A3332206C73 exe="/usr/bin/sudo" terminal=pts/0 res=success'             
Sep 01 17:31:12 seccomp audit[32794]: CRED_REFR pid=32794 uid=0 auid=1013 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_fprintd ac
ct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'                                                                                                              
Sep 01 17:31:12 seccomp sudo[32794]: pam_unix(sudo:session): session opened for user root by jkl92(uid=0)                                                                                     
Sep 01 17:31:12 seccomp audit[32794]: USER_START pid=32794 uid=0 auid=1013 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_
limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'                                                         
Sep 01 17:31:12 seccomp kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth109c450f: link becomes ready                                                                                               
Sep 01 17:31:12 seccomp kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready                                                                                                       
Sep 01 17:31:12 seccomp NetworkManager[764]: <info>  [1598995872.6598] device (veth109c450f): carrier: link connected                                                                         
Sep 01 17:31:12 seccomp NetworkManager[764]: <info>  [1598995872.6606] manager: (veth109c450f): new Veth device (/org/freedesktop/NetworkManager/Devices/9)
Sep 01 17:31:12 seccomp kernel: cni-podman0: port 2(veth109c450f) entered blocking state
Sep 01 17:31:12 seccomp kernel: cni-podman0: port 2(veth109c450f) entered disabled state
Sep 01 17:31:12 seccomp kernel: device veth109c450f entered promiscuous mode
Sep 01 17:31:12 seccomp kernel: cni-podman0: port 2(veth109c450f) entered blocking state
Sep 01 17:31:12 seccomp kernel: cni-podman0: port 2(veth109c450f) entered forwarding state
Sep 01 17:31:12 seccomp audit: ANOM_PROMISCUOUS dev=veth109c450f prom=256 old_prom=0 auid=1013 uid=0 gid=0 ses=1
Sep 01 17:31:12 seccomp systemd-udevd[32838]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Sep 01 17:31:12 seccomp systemd-udevd[32838]: Using default interface naming scheme 'v245'.
Sep 01 17:31:12 seccomp audit[32847]: NETFILTER_CFG table=nat family=2 entries=10 op=replace pid=32847 subj=unconfined_u:system_r:iptables_t:s0-s0:c0.c1023 comm="iptables"
Sep 01 17:31:12 seccomp audit[32849]: NETFILTER_CFG table=nat family=2 entries=12 op=replace pid=32849 subj=unconfined_u:system_r:iptables_t:s0-s0:c0.c1023 comm="iptables"
Sep 01 17:31:12 seccomp audit[32851]: NETFILTER_CFG table=nat family=2 entries=13 op=replace pid=32851 subj=unconfined_u:system_r:iptables_t:s0-s0:c0.c1023 comm="iptables"
Sep 01 17:31:12 seccomp audit[32853]: NETFILTER_CFG table=nat family=2 entries=14 op=replace pid=32853 subj=unconfined_u:system_r:iptables_t:s0-s0:c0.c1023 comm="iptables"
Sep 01 17:31:12 seccomp systemd[1]: Started libpod-conmon-3e6bc02ff2fed640a673bbafb814fa8588b2988e9a4a5fbd0ebc00a0aa91de28.scope.
Sep 01 17:31:12 seccomp systemd[1]: Started libcrun container.
Sep 01 17:31:12 seccomp systemd[1]: tmp-crun.Awfg20.mount: Succeeded.
Sep 01 17:31:12 seccomp systemd[961]: tmp-crun.Awfg20.mount: Succeeded.
Sep 01 17:31:12 seccomp systemd[1]: libpod-3e6bc02ff2fed640a673bbafb814fa8588b2988e9a4a5fbd0ebc00a0aa91de28.scope: Succeeded.
Sep 01 17:31:12 seccomp sudo[32794]: pam_unix(sudo:session): session closed for user root
Sep 01 17:31:12 seccomp audit[32794]: USER_END pid=32794 uid=0 auid=1013 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Sep 01 17:31:12 seccomp audit[32794]: CRED_DISP pid=32794 uid=0 auid=1013 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_env,pam_fprintd acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
Sep 01 17:31:12 seccomp audit[32931]: NETFILTER_CFG table=nat family=2 entries=15 op=replace pid=32931 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp audit[32933]: NETFILTER_CFG table=nat family=2 entries=17 op=replace pid=32933 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp audit[32934]: NETFILTER_CFG table=nat family=2 entries=15 op=replace pid=32934 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp audit[32936]: NETFILTER_CFG table=nat family=2 entries=17 op=replace pid=32936 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp audit[32937]: NETFILTER_CFG table=nat family=10 entries=5 op=replace pid=32937 subj=unconfined_u:system_r:iptables_t:s0 comm="ip6tables"
Sep 01 17:31:13 seccomp audit[32939]: NETFILTER_CFG table=nat family=10 entries=7 op=replace pid=32939 subj=unconfined_u:system_r:iptables_t:s0 comm="ip6tables"
Sep 01 17:31:13 seccomp audit[32940]: NETFILTER_CFG table=nat family=10 entries=5 op=replace pid=32940 subj=unconfined_u:system_r:iptables_t:s0 comm="ip6tables"
Sep 01 17:31:13 seccomp audit[32942]: NETFILTER_CFG table=nat family=10 entries=7 op=replace pid=32942 subj=unconfined_u:system_r:iptables_t:s0 comm="ip6tables"
Sep 01 17:31:13 seccomp kernel: cni-podman0: port 2(veth109c450f) entered disabled state
Sep 01 17:31:13 seccomp kernel: device veth109c450f left promiscuous mode
Sep 01 17:31:13 seccomp kernel: cni-podman0: port 2(veth109c450f) entered disabled state
Sep 01 17:31:13 seccomp audit: ANOM_PROMISCUOUS dev=veth109c450f prom=0 old_prom=256 auid=1013 uid=0 gid=0 ses=1
Sep 01 17:31:13 seccomp NetworkManager[764]: <info>  [1598995873.0462] device (veth109c450f): released from master device cni-podman0
Sep 01 17:31:13 seccomp audit[32955]: NETFILTER_CFG table=nat family=2 entries=15 op=replace pid=32955 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp audit[32958]: NETFILTER_CFG table=nat family=2 entries=14 op=replace pid=32958 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp audit[32959]: NETFILTER_CFG table=nat family=2 entries=12 op=replace pid=32959 subj=unconfined_u:system_r:iptables_t:s0 comm="iptables"
Sep 01 17:31:13 seccomp systemd[961]: run-netns-cni\x2d7b8121a1\x2dfd92\x2da6f5\x2d44d1\x2dd090204332da.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[1]: run-netns-cni\x2d7b8121a1\x2dfd92\x2da6f5\x2d44d1\x2dd090204332da.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[1]: tmp-crun.Ap3I5u.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[961]: tmp-crun.Ap3I5u.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[961]: var-lib-containers-storage-overlay\x2dcontainers-3e6bc02ff2fed640a673bbafb814fa8588b2988e9a4a5fbd0ebc00a0aa91de28-userdata-shm.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[1]: var-lib-containers-storage-overlay\x2dcontainers-3e6bc02ff2fed640a673bbafb814fa8588b2988e9a4a5fbd0ebc00a0aa91de28-userdata-shm.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[961]: var-lib-containers-storage-overlay-1217e0819dff93f275a2718ea0adfbb9c17c54f9cc383aca3c724e2d2530db75-merged.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[1]: var-lib-containers-storage-overlay-1217e0819dff93f275a2718ea0adfbb9c17c54f9cc383aca3c724e2d2530db75-merged.mount: Succeeded.
Sep 01 17:31:13 seccomp systemd[1]: libpod-conmon-3e6bc02ff2fed640a673bbafb814fa8588b2988e9a4a5fbd0ebc00a0aa91de28.scope: Succeeded.

[root@seccomp ~]# ls /tmp/
systemd-private-0241899183c7408eabd29f70cc3adde0-chronyd.service-GGFMLf      systemd-private-0241899183c7408eabd29f70cc3adde0-ModemManager.service-H1tlri
systemd-private-0241899183c7408eabd29f70cc3adde0-dbus-broker.service-XSxsAh  systemd-private-0241899183c7408eabd29f70cc3adde0-systemd-logind.service-o51Szh
[root@seccomp ~]# find / -name ls.json
[root@seccomp ~]# 

OCI runtime error when executing hook

I tried using this tool with a container that I made and it fails to run the hook. The container was originally made for Docker and runs Apache Guacamole as well as Firefox. If you need more information on the container, see my repo or ask me.

Steps to reproduce the error: (Fedora 32 server)
$ sudo dnf install oci-seccomp-bpf-hook podman git
$ git clone https://github.com/doctormay6/foxception.git
$ cd foxception
$ sudo podman build --tag foxception:seccomp -f ./Dockerfile
$ sudo podman run -d --shm-size=1G --annotation io.containers.trace-syscall=of:foxception.json foxception:seccomp
Error message at this point:
Error: error executing hook '/usr/libexec/oci/hooks.d/oci-seccomp-bpf-hook' (exit code: 1): OCI runtime error

I'll run podman in debug log level and see if I can pinpoint exactly where it fails or if there is a more informative error message.

Request: Cut v1.2.0

Hey 👋, may I ask you to cut a new release that we can get the architecture feature in? I'm on my way packaging it for openSUSE distributions and it would be cool to have the feature landed. 😇

Build failure on RHEL 8.2

Hi,
trying to make binary fails on my RHEL8.2:

$ make binary
GO111MODULE=on go build -mod=vendor -o bin/oci-seccomp-bpf-hook -ldflags "-X main.version=1.0.0" github.com/containers/oci-seccomp-bpf-hook

github.com/iovisor/gobpf/bcc

vendor/github.com/iovisor/gobpf/bcc/module.go:32:10: fatal error: bcc/bcc_common.h: No such file or directory
#include <bcc/bcc_common.h>
^~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:49: binary] Fehler 2

Did anyone succeed to install it on RHEL8.2?
If so, could you kindly post the steps?
I'm concerned a bit due to "No match for argument: bcc-devel" on RHEL. How to mitigate?

Cheers.

man pages

It would be great to have man pages for the hook explaining how it can be used etc.

Apache 2.0 license seems to be incorrectly applied

At bottom of LICENSE is explained how to apply the Apache 2.0 license to some work. The following text should be enclosed in source files:

   Copyright {yyyy} {name of copyright owner}

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.

I cannot find this statement in any file of this tree and I'm blocked from packaging it for Debian.

Actually no file has any copyright notice at all.

RHEL8 - make install issue with oci-seccomp-bpf-hook.json

Hi there,
I was following https://podman.io/blogs/2019/10/15/generate-seccomp-profiles.html on my bit older test system and run into issues with the make install.

It copies oci-seccomp-bpf-hook.json to /usr/local/share/containers/oci/hooks.d/ instead of /usr/share/containers/oci/hooks.d/
Probably wrong PREFIX in the Makefile.

Sadly there is no error in the profile generation that indicates the lack of this file.

Long version:

# Old system, sorry about that...
cat /etc/redhat-release 
Rocky Linux release 8.5 (Green Obsidian)

podman -v
podman version 3.4.2

sudo yum config-manager --set-enabled powertools

# following this guide: https://podman.io/blogs/2019/10/15/generate-seccomp-profiles.html
sudo yum install bcc-devel bcc-tools git golang libseccomp-devel golang-github-cpuguy83-md2man make
git clone https://github.com/containers/oci-seccomp-bpf-hook.git
cd oci-seccomp-bpf-hook
make binary
sudo make install

sudo podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:30 ls / > /dev/null
ls -la /tmp/ls.json
ls: cannot access '/tmp/ls.json': No such file or directory

# Digging around a bit
# from "man oci-seccomp-bpf-hook"
FILES
       OCI hook configuration file for prestart
       /usr/share/containers/oci/hooks.d/oci-seccomp-bpf-hook.json
       /etc/containers/oci/hooks.d/oci-seccomp-bpf-hook.json (Override)

ls -la        /usr/share/containers/oci/hooks.d/oci-seccomp-bpf-hook.json
ls: cannot access '/usr/share/containers/oci/hooks.d/oci-seccomp-bpf-hook.json': No such file or directory

# This did the trick:
sudo mkdir -p /usr/share/containers/oci/hooks.d/
sudo ln -s /usr/local/share/containers/oci/hooks.d/oci-seccomp-bpf-hook.json /usr/share/containers/oci/hooks.d/oci-seccomp-bpf-hook.json
sudo podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:30 ls / > /dev/null
ls -la /tmp/ls.json
-rw-r--r--. 1 root root 695 Mar 10 00:58 /tmp/ls.json

Need to hook up cirrus testing.

  • Create new GCE Project
  • Grant stake-holders 'owner' IAM role (including James) in new project
  • Ask James Russell to move billing account
  • Create 'imgts-' IAM service account in new project
    - Use description: "Account for access to update VM Image metadata"
    - Create and store key (JSON) file locally for use later
  • Grant 'imgts-' IAM roles in new project
    - Service Account User
  • Grant 'imgts-' IAM roles in libpod project
    - Compute Image Label Writer
    - Service Account User
  • Create 'cirrus-ci' IAM service account in new project
    - Use description: "Account for access to manage resources for Cirrus-CI"
    - Create and store key (JSON) file locally for use later
  • Grant 'cirrus-ci' IAM roles in new project
    - Storage Object Admin
    - Storage Admin
    - Compute Instance Admin (v1)
    - Compute OS Admin Login
    - Compute OS Login
    - Service Account User
  • Grant 'cirrus-ci' IAM roles in libpod project
    - Compute Image User
    - Service Account User
  • In Github Org. settings, enable Cirrus-CI App. access to new repo.
  • Go to https://cirrus-ci.com/github/containers/<new repo name> settings
    - Disable "Require approval for builds from users without write permissions"
    - Enable "Decrypt Secured Environment Variables for builds initialized by:"
    - Use the encryption box to generate ENCRYPTED[abc123] database keys for both service accounts (imgts-<number> and cirrus-ci) JSON file contents. Note the strings temporarily.
    - Use the encryption box to generate database keys for the libpod project ID and the complete (fake) e-mail name of the imgts-<number> service account.
  • Add the new google project ID to the Google Cloud Platform projects list under the Monitored accounts section of the Stackdriver workspace settings page.
  • Clone and create working branch for new repo.
  • Open PR for working branch
  • Port .cirrus.ci over, based on another project, like buildah.
  • Port over hack/get_ci_vm.sh, edit as needed until it works.
  • Implement simple tasks/scripts, pushing to PR, and updating until Cirrus-CI working properly.
  • Finalize PR based on project needs.

build failure on armv7hl

First off, not sure if armv7hl is supported at all. If it's not, please close this right away.

I tried building the latest master commit for fedora rawhide armv7hl and I noticed this:

In file included from _cgo_export.c:4:
cgo-gcc-export-header-prolog:50:14: error: conflicting types for 'lostCallback'
In file included from _cgo_export.c:4:
perf.go:37:13: note: previous declaration of 'lostCallback' was here
_cgo_export.c:45:7: error: conflicting types for 'lostCallback'
   45 |  void lostCallback(void* cbCookie, long unsigned int lost)
      |       ^~~~~~~~~~~~
In file included from _cgo_export.c:4:
perf.go:37:13: note: previous declaration of 'lostCallback' was here

Seems similar to golang/go#43830 but in this case it occurs even without specifying any CGO_CFLAGS and only for armv7hl.

Hook hangs on "Loading enter tracepoint"

Alright, this might be tricky to debug, but let me first summarize my observations:

Podman command:

> sudo podman --log-level=debug --hooks-dir $PWD/hooks run --annotation io.containers.trace-syscall="of:/tmp/test.json" -it alpine echo hi

Systemd outputs that the hook gets loaded successfully, but timed out:

Aug 21 12:49:29 nixos oci-seccomp-bpf-hook[21953]: time="2020-08-21T12:49:29+02:00" level=info msg="Started OCI seccomp hook version v1.2.1-dev"
Aug 21 12:49:29 nixos oci-seccomp-bpf-hook[21953]: time="2020-08-21T12:49:29+02:00" level=info msg="Trying to load `kheaders` module"
Aug 21 12:49:29 nixos oci-seccomp-bpf-hook[21953]: time="2020-08-21T12:49:29+02:00" level=info msg="Loading `kheaders` failed, continuing in hope kernel headers reside on disk: fork/exec /usr/sbin/modprobe: no such file or directory"
Aug 21 12:49:29 nixos oci-seccomp-bpf-hook[21962]: time="2020-08-21T12:49:29+02:00" level=info msg="Running floating process PID to attach: 21947"
Aug 21 12:49:29 nixos oci-seccomp-bpf-hook[21962]: time="2020-08-21T12:49:29+02:00" level=info msg="Loading enter tracepoint"
Aug 21 12:49:39 nixos oci-seccomp-bpf-hook[21953]: time="2020-08-21T12:49:39+02:00" level=fatal msg="BPF program didn't compile and attach within 10 seconds: please refer to the syslog (e.g., journalctl(1)) for more details"
Aug 21 12:49:39 nixos systemd[1]: Stopping libcontainer container 120de2b30c06daabfab5409b7abf75a9687a6a129813e9fde286cad550bec028.

I think we can ignore the kheaders message for now, because they're loaded:

> lsmod | rg kheaders
kheaders             3645440  0

Might it be possible that there is a library incompatibility with the hook?

The linker says:

> ldd bin/oci-seccomp-bpf-hook
        linux-vdso.so.1 (0x00007fff945f0000)
        libbcc.so.0 => /nix/store/1qn9mgcq87rm37dvr23arr20m62izg8y-bcc-0.12.0/lib/libbcc.so.0 (0x00007faba4644000)
        libseccomp.so.2 => /nix/store/y9r40mvhgi7hka0xhpml9dq3z1k9g0f5-libseccomp-2.4.2-lib/lib/libseccomp.so.2 (0x00007faba45f7000)
        libpthread.so.0 => /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib/libpthread.so.0 (0x00007faba45d6000)
        libc.so.6 => /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib/libc.so.6 (0x00007faba4417000)
        libelf.so.1 => /nix/store/sv6f05ngaarba50ybr6fdfc7cciv6nbv-elfutils-0.176/lib/libelf.so.1 (0x00007faba43fa000)
        libz.so.1 => /nix/store/msp4hm62a75pdidlc3s2ymma2g5hsjjk-zlib-1.2.11/lib/libz.so.1 (0x00007faba43dd000)
        librt.so.1 => /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib/librt.so.1 (0x00007faba43d3000)
        libdl.so.2 => /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib/libdl.so.2 (0x00007faba43ce000)
        libncursesw.so.6 => /nix/store/xhhkr936b9q5sz88jp4l29wljbbcg39k-ncurses-6.1-20190112/lib/libncursesw.so.6 (0x00007faba435d000)
        libstdc++.so.6 => /nix/store/danv012gh0aakh8xnk2b35vahklz72mk-gcc-9.2.0-lib/lib/libstdc++.so.6 (0x00007faba417d000)
        libm.so.6 => /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib/libm.so.6 (0x00007faba403b000)
        libgcc_s.so.1 => /nix/store/danv012gh0aakh8xnk2b35vahklz72mk-gcc-9.2.0-lib/lib/libgcc_s.so.1 (0x00007faba4021000)
        /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib/ld-linux-x86-64.so.2 => /nix/store/xg6ilb9g9zhi2zg1dpi4zcp288rhnvns-glibc-2.30/lib64/ld-linux-x86-64.so.2 (0x00007faba8352000)

🤔

build fails on CentOS

Trying to build 1.1.0 rpm on OBS for CentOS, and it fails with:

[  168s] _build/src/github.com/containers/oci-seccomp-bpf-hook/vendor/github.com/iovisor/gobpf/bcc/module.go:98:40: too many arguments in call to _Cfunc_bpf_module_create_c_from_string
[  168s] 	have (*_Ctype_char, number, **_Ctype_char, _Ctype_int, _Ctype__Bool, nil)
[  168s] 	want (*_Ctype_char, _Ctype_uint, **_Ctype_char, _Ctype_int, _Ctype__Bool)

Fails to build with current bcc

bcc added two additional parameters to bcc_func_load which aren't passed here:

[  141s] GO111MODULE=on go build -mod=vendor -mod=vendor -o bin/oci-seccomp-bpf-hook -ldflags "-X main.version=v1.2.6" github.com/containers/oci-seccomp-bpf-hook
[  141s] # github.com/iovisor/gobpf/bcc
[  141s] vendor/github.com/iovisor/gobpf/bcc/module.go:230:28: not enough arguments in call to _C2func_bcc_func_load
[  141s]        have (unsafe.Pointer, _Ctype_int, *_Ctype_char, *_Ctype_struct_bpf_insn, _Ctype_int, *_Ctype_char, _Ctype_uint, _Ctype_int, *_Ctype_char, _Ctype_uint, nil)
[  141s]        want (unsafe.Pointer, _Ctype_int, *_Ctype_char, *_Ctype_struct_bpf_insn, _Ctype_int, *_Ctype_char, _Ctype_uint, _Ctype_int, *_Ctype_char, _Ctype_uint, *_Ctype_char, _Ctype_int)
[  141s] make: *** [Makefile:49: binary] Error 2

Unfortunately it looks like gobpf is dead: iovisor/gobpf#311

oci-seccomp-bpf-hook exits with OCI runtime error Code 127

Hello , Sorry if this is the wrong place to raise this issue or its something too basic.

I am following the same blog and same procedure as above.
Fresh install of Fedora 33 from Fedora-Workstation-Live-x86-64-33-1.2.iso
Then dnf install oci-seccomp-bpf-hook

The images were created
[root@fedora labs]# sudo podman run fedora:33 cat /etc/fedora-release
Fedora release 33 (Thirty Three)

However,
root@fedora labs]# sudo podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:33 ls / > /dev/null

Error: error executing hook /usr/libexec/oci/hooks.d/oci-seccomp-bpf-hook (exit code: 127): OCI runtime error

journalctl is empty
root@fedora labs]# journalctl --since "10 minutes ago" | grep seccomp

Any pointers on something I might have missed?

Originally posted by @Kvlbs in #64 (comment)

consistent integration test failures on Fedora rawhide and 38

Seeing this consistently on fedora rawhide and 38.

$ rpm -q podman
podman-4.4.2-3.fc39.x86_64


$ sudo make test-integration

==> Running integration tests (must be run as root)
./hack/check_root.sh
bats  test/
00-simple.bats
 ✓ Podman available
 ✓ Version check
 ✗ Trace and check size of generated profile
   (in test file test/00-simple.bats, line 42)
     `[ "${size}" -gt 0 ]' failed
   Temporary file: /tmp/tmp.8BtbxR9AcP
   Podman output: bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var
   Size of generated file: 0
 ✗ Trace and use generated profile
   (in test file test/00-simple.bats, line 60)
     `[ "${size}" -gt 0 ]' failed
   Temporary file: /tmp/tmp.1sOORIrrPe
   Podman output: bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var
   Size of generated file: 0
 ✗ Containers fails to run blocked syscall
   (in test file test/00-simple.bats, line 82)
     `[ "${size}" -gt 0 ]' failed
   Temporary file: /tmp/tmp.ikcxhS8Iy6
   Podman output: bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var
   Size of generated file: 0
 ✗ Extend existing seccomp profile
   (in test file test/00-simple.bats, line 110)
     `[ "${size}" -gt 0 ]' failed
   Temporary file 1: /tmp/tmp.CkIbpn1Nqs
   Temporary file 2: /tmp/tmp.Auh0OmTjQr
   Size of the first generated file: 0
 ✗ Syscall blocked in input profile remains blocked in output profile
   (in test file test/00-simple.bats, line 149)
     `[ "${size}" -gt 0 ]' failed
   Temporary file : /tmp/tmp.M5Q6ydWWyi
   Podman output:
   Size of the first generated file: 0
01-syslog.bats
 ✓ Trace and look for syslogs

8 tests, 5 failures

make: *** [Makefile:66: test-integration] Error 1

not enough arguments in call to _C2func_bpf_attach_uprobe

$ go get -u github.com/containers/oci-seccomp-bpf-hook

github.com/containers/oci-seccomp-bpf-hook/vendor/github.com/iovisor/gobpf/bcc

github.com/containers/oci-seccomp-bpf-hook/vendor/github.com/iovisor/gobpf/bcc/module.go:261:33: not enough arguments in call to _C2func_bpf_attach_uprobe
have (_Ctype_int, uint32, *_Ctype_char, *_Ctype_char, _Ctype_ulong, _Ctype_int)
want (_Ctype_int, uint32, *_Ctype_char, *_Ctype_char, _Ctype_ulong, _Ctype_int, _Ctype_uint)

Same error with:
oci-seccomp-bpf-hook$ make binary
GO111MODULE=on go build -mod=vendor -mod=vendor -o bin/oci-seccomp-bpf-hook -ldflags "-X main.version=v1.2.1-dev" github.com/containers/oci-seccomp-bpf-hook

github.com/iovisor/gobpf/bcc

vendor/github.com/iovisor/gobpf/bcc/module.go:261:33: not enough arguments in call to _C2func_bpf_attach_uprobe
have (_Ctype_int, uint32, *_Ctype_char, *_Ctype_char, _Ctype_ulong, _Ctype_int)
want (_Ctype_int, uint32, *_Ctype_char, *_Ctype_char, _Ctype_ulong, _Ctype_int, _Ctype_uint)
make: *** [Makefile:49: binary] Error 2

I compiled bcc from source. My system is Ubuntu 20.04

oci hook processes never terminate

When trying to make a seccomp profile with the oci-hook and mpirun (lsdyna). The lsdyna job (24 containers on one host) successfully terminates, but the oci-hook processes continue to run.
I end up with a system that is under heavy load but do not get the expected profile.

Could be that this is because 24 containers try to write the same file?

This is how I start the containers:

/usr/local/platform-mpi/09.01.04.03r/bin/mpirun \
   -TCP -prot \
   -hostfile hostfile \
   podman run --runtime /usr/bin/runc \
      --rm \
      --annotation io.containers.trace-syscall=of:/chome/poduser/lsdyna.json \
      --env-host \
           --userns=keep-id \
           --net=host --pid=host --ipc=host \
           --volume /chome/poduser:/chome/poduser \
                  localhost/ls-dyna /usr/local/dyna3d/mpp_R9.3_dm_134916/mpp_R9.3_dm_134916 \
                   i=Caravan-V03c-2400k-main-shell16-120ms.k \
                   memory=600m \
                   memory2=60m \
                   endtime=0.02

This is what I see hours after the lsdyna job finishes:

# ps aux | grep oci-seccomp-bpf-hook

root     36141  141  0.1 1447468 145756 ?      Sl   Apr15 972:07 oci-seccomp-bpf-hook -r 36123 -o /chome/poduser/lsdyna.json -i
root     36209  152  0.1 1595444 144112 ?      Sl   Apr15 1047:44 oci-seccomp-bpf-hook -r 36190 -o /chome/poduser/lsdyna.json -i
root     36278  151  0.1 1447724 144652 ?      Sl   Apr15 1041:24 oci-seccomp-bpf-hook -r 36253 -o /chome/poduser/lsdyna.json -i
root     36329  149  0.1 1447724 144104 ?      Sl   Apr15 1027:21 oci-seccomp-bpf-hook -r 36311 -o /chome/poduser/lsdyna.json -i
root     36402  140  0.1 1447724 143588 ?      Sl   Apr15 968:47 oci-seccomp-bpf-hook -r 36374 -o /chome/poduser/lsdyna.json -i
root     36483  146  0.1 1447724 144112 ?      Sl   Apr15 1009:38 oci-seccomp-bpf-hook -r 36461 -o /chome/poduser/lsdyna.json -i
root     36520  139  0.1 1447980 143432 ?      Sl   Apr15 962:05 oci-seccomp-bpf-hook -r 36504 -o /chome/poduser/lsdyna.json -i
root     36636 21.9  0.1 1447724 144092 ?      Sl   Apr15 151:21 oci-seccomp-bpf-hook -r 36589 -o /chome/poduser/lsdyna.json -i
root     36693  141  0.1 1595252 146128 ?      Rl   Apr15 972:11 oci-seccomp-bpf-hook -r 36624 -o /chome/poduser/lsdyna.json -i
root     36715  152  0.1 1595700 143904 ?      Sl   Apr15 1052:28 oci-seccomp-bpf-hook -r 36666 -o /chome/poduser/lsdyna.json -i
root     36742  141  0.1 1373992 143504 ?      Sl   Apr15 973:23 oci-seccomp-bpf-hook -r 36695 -o /chome/poduser/lsdyna.json -i
root     36810  148  0.1 1595188 144200 ?      Sl   Apr15 1021:10 oci-seccomp-bpf-hook -r 36746 -o /chome/poduser/lsdyna.json -i
root     36845  141  0.1 1447724 143648 ?      Rl   Apr15 972:05 oci-seccomp-bpf-hook -r 36788 -o /chome/poduser/lsdyna.json -i
root     36875  144  0.1 1595188 144252 ?      Sl   Apr15 996:41 oci-seccomp-bpf-hook -r 36813 -o /chome/poduser/lsdyna.json -i
root     36889  152  0.1 1448236 143320 ?      Sl   Apr15 1049:39 oci-seccomp-bpf-hook -r 36858 -o /chome/poduser/lsdyna.json -i
root     36970  139  0.1 1448236 143692 ?      Sl   Apr15 960:22 oci-seccomp-bpf-hook -r 36924 -o /chome/poduser/lsdyna.json -i
root     36977  139  0.1 1521456 141660 ?      Rl   Apr15 963:03 oci-seccomp-bpf-hook -r 36938 -o /chome/poduser/lsdyna.json -i
root     37045  149  0.1 1521712 143752 ?      Rl   Apr15 1030:00 oci-seccomp-bpf-hook -r 36983 -o /chome/poduser/lsdyna.json -i
root     37087  141  0.1 1447468 144620 ?      Sl   Apr15 976:36 oci-seccomp-bpf-hook -r 37019 -o /chome/poduser/lsdyna.json -i
root     37111  144  0.1 1595700 144260 ?      Sl   Apr15 995:09 oci-seccomp-bpf-hook -r 37054 -o /chome/poduser/lsdyna.json -i
root     37139  152  0.1 1595444 144244 ?      Sl   Apr15 1047:31 oci-seccomp-bpf-hook -r 37093 -o /chome/poduser/lsdyna.json -i
root     37202  141  0.1 1521712 143320 ?      Rl   Apr15 971:41 oci-seccomp-bpf-hook -r 37142 -o /chome/poduser/lsdyna.json -i
root     37253  150  0.1 1595700 144180 ?      Rl   Apr15 1039:10 oci-seccomp-bpf-hook -r 37215 -o /chome/poduser/lsdyna.json -i
root     37260  147  0.1 1447980 151920 ?      Sl   Apr15 1016:15 oci-seccomp-bpf-hook -r 37235 -o /chome/poduser/lsdyna.json -i

Any clues on what might go wrong or how to fix it?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

dockerfile
contrib/cirrus/Dockerfile
gomod
go.mod
  • go 1.21
  • github.com/containers/common v0.59.0
  • github.com/containers/storage v1.54.0
  • github.com/iovisor/gobpf v0.2.1-0.20221005153822-16120a1bf4d4@16120a1bf4d4
  • github.com/opencontainers/runtime-spec v1.2.0
  • github.com/seccomp/libseccomp-golang v0.10.0
  • github.com/sirupsen/logrus v1.9.3
  • github.com/stretchr/testify v1.9.0
regex
.cirrus.yml
  • containers/automation_images 20240529t141726z-f40f39d13

  • Check this box to trigger a request for Renovate to run again on this repository

Allow recording multiple processes into the same file

We can have multiple containers within a single pod in Kubernetes, but we cannot have annotations on a container level. This means if we run the hook in k8s with multiple containers, we will overwrite the output file depending on who is the last writer:

if err := ioutil.WriteFile(profilePath, sJSON, 0644); err != nil {
return errors.Wrap(err, "error writing seccomp profile")
}

My idea is now to specify an option to write and append a list of profiles to a file instead of overwriting it. WDYT?

It wouldn't be easy to preserve the JSON format of the file. We would have to track how much other processes are running and will probably run into races. We could just add the profiles line by line but this would not be valid JSON any more.

No secomp filter output file created

Steps to reproduce

sudo dnf install -y podman
sudo dnf install -y bcc-devel bcc-tools git golang libseccomp-devel golang-github-cpuguy83-md2man make
git clone https://github.com/containers/oci-seccomp-bpf-hook.git
cd oci-seccomp-bpf-hook
make binary
sudo make install

$ sudo podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:30 ls / > /dev/null

Expected result
secomp filter output file be created at /tmp/ls.json

Current behavior
No secomp filter output file is created at /tmp/ls.json

Env
I am trying this on a Vagrant system

cat /etc/os-release

NAME=Fedora
VERSION="34 (Cloud Edition)"
ID=fedora
VERSION_ID=34
VERSION_CODENAME=""
PLATFORM_ID="platform:f34"
PRETTY_NAME="Fedora 34 (Cloud Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:34"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/34/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=34
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=34
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Cloud Edition"
VARIANT_ID=cloud
podman version
Version:      3.3.1
API Version:  3.3.1
Go Version:   go1.16.6
Built:        Mon Aug 30 20:46:36 2021
OS/Arch:      linux/amd64
sudo  journalctl --no-pager | grep oci-seccomp
Sep 16 22:43:40 fedora audit[32951]: USER_CMD pid=32951 uid=1000 auid=1000 ses=8 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook" cmd=6D616B6520696E7374616C6C exe="/usr/bin/sudo" terminal=pts/0 res=success'
Sep 16 22:43:40 fedora sudo[32951]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook ; USER=root ; COMMAND=/usr/bin/make install
Sep 16 22:44:01 fedora audit[32989]: USER_CMD pid=32989 uid=1000 auid=1000 ses=8 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook" cmd=706F646D616E2072756E202D2D616E6E6F746174696F6E20696F2E636F6E7461696E6572732E74726163652D73797363616C6C3D6F663A2F746D702F6C732E6A736F6E206665646F72613A3330206C73202F exe="/usr/bin/sudo" terminal=pts/0 res=success'
Sep 16 22:44:01 fedora sudo[32989]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook ; USER=root ; COMMAND=/usr/bin/podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:30 ls /
Sep 16 22:49:12 fedora audit[33470]: USER_CMD pid=33470 uid=1000 auid=1000 ses=8 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook" cmd=706F646D616E2072756E202D2D616E6E6F746174696F6E20696F2E636F6E7461696E6572732E74726163652D73797363616C6C3D6F663A2F746D702F6C732E6A736F6E206665646F72613A3330206C73202F exe="/usr/bin/sudo" terminal=pts/0 res=success'
Sep 16 22:49:12 fedora sudo[33470]:  vagrant : TTY=pts/0 ; PWD=/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook ; USER=root ; COMMAND=/usr/bin/podman run --annotation io.containers.trace-syscall=of:/tmp/ls.json fedora:30 ls /
Sep 16 22:55:55 fedora audit[33719]: USER_CMD pid=33719 uid=1000 auid=1000 ses=8 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/vagrant/go/src/github.com/oci-seccomp-bpf-hook" cmd=6A6F75726E616C63746C202D2D6E6F2D7061676572 exe="/usr/bin/sudo" terminal=pts/0 res=success'

Use one hook instead of two

We can combine the two hooks into one by inspecting the inspecting the container's state in the runtime state json that is passed via stdin. @wking gives a great overview of how that can be done here: cri-o/cri-o#1360

Thanks to @wking for the guidance!

Disable Dependabot after Renovate trial run

This repository is migrating over to Renovate for dependency/security automated updates, given it's enhanced configuration/capabilities and centralized management. For the time being, both tools will be enabled so that we may catch any "missing" update problems. After 30-ish days, Dependabot should be disabled in the repo. settings - leaving the security-scanning feature enabled (renovate uses it).

Ref: #111

Out of memory kill on Fedora 34 when used with CRI-O

In the security-profiles-operator, we're using Fedora 33 for our recording and SELinux e2e tests, which uses the OCI seccomp BPF hook 1.2.1-2.fc33.

Upgrading the Vagrant machine to Fedora 34 will break the hook (1.2.2-0.1.git4e42394.fc34), but only when using CRI-O's cgroupfs cgroup driver. It works with systemd.

Running the hook with CRI-O (cgroupfs) will cause the following OOM kill on Fedora 34:

[root@fedora vagrant]# journalctl --no-pager | grep oci-seccomp
May 05 10:24:03 fedora oci-seccomp-bpf-hook[17864]: time="2021-05-05T10:24:03Z" level=info msg="Started OCI seccomp hook version 1.2.2"
May 05 10:24:03 fedora oci-seccomp-bpf-hook[17864]: time="2021-05-05T10:24:03Z" level=info msg="Trying to load `kheaders` module"
May 05 10:24:03 fedora oci-seccomp-bpf-hook[17873]: time="2021-05-05T10:24:03Z" level=info msg="Running floating process PID to attach: 17863"
May 05 10:24:05 fedora oci-seccomp-bpf-hook[17873]: time="2021-05-05T10:24:05Z" level=info msg="Loading enter tracepoint"
May 05 10:24:05 fedora oci-seccomp-bpf-hook[17873]: time="2021-05-05T10:24:05Z" level=info msg="Loading exit tracepoint"
May 05 10:24:05 fedora oci-seccomp-bpf-hook[17873]: time="2021-05-05T10:24:05Z" level=info msg="Loaded tracepoints"
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="unsafe use of perf" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora oci-seccomp-bpf-hook[17864]: time="2021-05-05T10:24:05Z" level=info msg="Received signal (presumably from child): user defined signal 1"
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="unsafe use of perf" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17864]: AVC avc:  denied  { confidentiality } for  pid=17864 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="unsafe use of perf" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
May 05 10:24:05 fedora audit[17873]: AVC avc:  denied  { confidentiality } for  pid=17873 comm="oci-seccomp-bpf" lockdown_reason="use of bpf to read kernel RAM" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:container_runtime_t:s0 tclass=lockdown permissive=0
jMay 05 10:24:05 fedora oci-seccomp-bpf-hook[17873]: time="2021-05-05T10:24:05Z" level=info msg="PerfMap Start"
May 05 10:24:31 fedora kernel: [  17873]     0 17873   515841    54011   925696        0          1000 oci-seccomp-bpf
May 05 10:24:31 fedora kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/kubepods/besteffort/pod680ec8b1-a1ec-4f9d-b3e6-ecb416b60fce/crio-conmon-7ff9d0b6352d6eb2ee1f6ac851727ae5ebf3e14d177e5adb0dc868588aed0cf5,task=oci-seccomp-bpf,pid=17873,uid=0
May 05 10:24:31 fedora kernel: Out of memory: Killed process 17873 (oci-seccomp-bpf) total-vm:2063364kB, anon-rss:216044kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:904kB oom_score_adj:1000
May 05 10:24:31 fedora kernel: oom_reaper: reaped process 17873 (oci-seccomp-bpf), now anon-rss:0kB, file-rss:144kB, shmem-rss:0kB

cc @JAORMX @jhrozek

hook should detect when running in a rootless context and give a meaningful error

# podman run --rm --log-level=info --hooks-dir /usr/share/containers/oci/hooks.d --security-opt label=disable --annotation io.containers.trace-syscall='of:/tmp/foo.json'  -it  bash sh -c 'ls -al'
INFO[0000] podman filtering at log level info           
INFO[0000] Found CNI network podman (type=bridge) at /home/bernhard/.config/cni/net.d/87-podman.conflist 
INFO[0000] Setting parallel job count to 25             
INFO[0000] Running conmon under slice user.slice and unitName libpod-conmon-ff907b2f6f7c482f5a49a6823c9fbf468b7f3041f3e48db2aad9ca0bfeacb16c.scope 
Error: OCI runtime error: error executing hook `/usr/libexec/oci/hooks.d/oci-seccomp-bpf-hook` (exit code: 1)

Note that the last lane is delayed by roughly 5~10 seconds

# /usr/share/containers/oci/hooks.d/oci-seccomp-bpf-hook.json
{
    "version": "1.0.0",
    "hook": {
        "path": "/usr/libexec/oci/hooks.d/oci-seccomp-bpf-hook",
        "args": [
            "oci-seccomp-bpf-hook",
            "-s"
        ]
    },
    "when": {
        "annotations": {
            "^io\\.containers\\.trace-syscall$": ".*"
        }
    },
    "stages": [
        "prestart"
    ]
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.