Comments (15)
daemon1024
commented on June 10, 2024
1
As per my understanding, The operator wouldn't try to mount /usr/src unless there's no BTF present.
Kernel version is very recent so I am not sure why would BTF be not present.
Can you share annotations added to nodes? (kubectl describe nodes)
// @rksharma95
from kubearmor.
daemon1024
commented on June 10, 2024
1
We will still note this, KubeArmor would fail to run if both /usr/src or BTF is not present.
Even if we ignore /usr/src here, nothing would work. We will think of a way to inform the users that either of these are needed.
from kubearmor.
daemon1024
commented on June 10, 2024
1
@aberfeldy agreed, we had an internal discussion as well. We are trying to explore what might be a better approach here, especially also considering how to inform user at the same time that things might not run.
We will keep you posted, Good to know KubeArmor is working as expected. Let me know if we can help with anything else 🙌🏽
from kubearmor.
aberfeldy
commented on June 10, 2024
Sure thing. those are all the annotations nodes have.
alpha.kubernetes.io/provided-node-ip: x.x.x.x
csi.volume.kubernetes.io/nodeid: {"rook-ceph.cephfs.csi.ceph.com":"nodename","rook-ceph.rbd.csi.ceph.com":"nodename"}
k3s.io/node-args: ["server","--server","https://cp-lb:443","--token","********","--node-ip","x.x.x.x","--node-external-ip","x.x.x.x","--...
k3s.io/node-config-hash: ...
k3s.io/node-env: {"K3S_DATA_DIR":"/var/lib/rancher/k3s/data/28f7e87eba734b7f7731dc900e2c84e0e98ce869f3dcf57f65dc7bbb80e12e56"}
kubevirt.io/heartbeat: 2024-04-02T14:29:31Z
kubevirt.io/ksm-handler-managed: false
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: truefrom kubearmor.
rksharma95
commented on June 10, 2024
@aberfeldy can you please share node labels?
from kubearmor.
aberfeldy
commented on June 10, 2024
sure:
beta.kubernetes.io/arch: amd64
beta.kubernetes.io/os: linux
cpu-feature.node.kubevirt.io/3dnowprefetch: true
cpu-feature.node.kubevirt.io/abm: true
cpu-feature.node.kubevirt.io/adx: true
cpu-feature.node.kubevirt.io/aes: true
cpu-feature.node.kubevirt.io/amd-ssbd: true
cpu-feature.node.kubevirt.io/amd-stibp: true
cpu-feature.node.kubevirt.io/apic: true
cpu-feature.node.kubevirt.io/arat: true
cpu-feature.node.kubevirt.io/arch-capabilities: true
cpu-feature.node.kubevirt.io/avx: true
cpu-feature.node.kubevirt.io/avx2: true
cpu-feature.node.kubevirt.io/bmi1: true
cpu-feature.node.kubevirt.io/bmi2: true
cpu-feature.node.kubevirt.io/clflush: true
cpu-feature.node.kubevirt.io/clflushopt: true
cpu-feature.node.kubevirt.io/clwb: true
cpu-feature.node.kubevirt.io/clzero: true
cpu-feature.node.kubevirt.io/cmov: true
cpu-feature.node.kubevirt.io/cmp_legacy: true
cpu-feature.node.kubevirt.io/cr8legacy: true
cpu-feature.node.kubevirt.io/cx16: true
cpu-feature.node.kubevirt.io/cx8: true
cpu-feature.node.kubevirt.io/de: true
cpu-feature.node.kubevirt.io/f16c: true
cpu-feature.node.kubevirt.io/fma: true
cpu-feature.node.kubevirt.io/fpu: true
cpu-feature.node.kubevirt.io/fsgsbase: true
cpu-feature.node.kubevirt.io/fxsr: true
cpu-feature.node.kubevirt.io/fxsr_opt: true
cpu-feature.node.kubevirt.io/hypervisor: true
cpu-feature.node.kubevirt.io/ibpb: true
cpu-feature.node.kubevirt.io/ibrs: true
cpu-feature.node.kubevirt.io/invtsc: true
cpu-feature.node.kubevirt.io/lahf_lm: true
cpu-feature.node.kubevirt.io/lbrv: true
cpu-feature.node.kubevirt.io/lfence-always-serializing: true
cpu-feature.node.kubevirt.io/lm: true
cpu-feature.node.kubevirt.io/mca: true
cpu-feature.node.kubevirt.io/mce: true
cpu-feature.node.kubevirt.io/mds-no: true
cpu-feature.node.kubevirt.io/misalignsse: true
cpu-feature.node.kubevirt.io/mmx: true
cpu-feature.node.kubevirt.io/mmxext: true
cpu-feature.node.kubevirt.io/movbe: true
cpu-feature.node.kubevirt.io/msr: true
cpu-feature.node.kubevirt.io/mtrr: true
cpu-feature.node.kubevirt.io/npt: true
cpu-feature.node.kubevirt.io/nrip-save: true
cpu-feature.node.kubevirt.io/null-sel-clr-base: true
cpu-feature.node.kubevirt.io/nx: true
cpu-feature.node.kubevirt.io/osvw: true
cpu-feature.node.kubevirt.io/pae: true
cpu-feature.node.kubevirt.io/pat: true
cpu-feature.node.kubevirt.io/pause-filter: true
cpu-feature.node.kubevirt.io/pclmuldq: true
cpu-feature.node.kubevirt.io/pdpe1gb: true
cpu-feature.node.kubevirt.io/perfctr_core: true
cpu-feature.node.kubevirt.io/pfthreshold: true
cpu-feature.node.kubevirt.io/pge: true
cpu-feature.node.kubevirt.io/pni: true
cpu-feature.node.kubevirt.io/popcnt: true
cpu-feature.node.kubevirt.io/pschange-mc-no: true
cpu-feature.node.kubevirt.io/pse: true
cpu-feature.node.kubevirt.io/pse36: true
cpu-feature.node.kubevirt.io/rdctl-no: true
cpu-feature.node.kubevirt.io/rdpid: true
cpu-feature.node.kubevirt.io/rdrand: true
cpu-feature.node.kubevirt.io/rdseed: true
cpu-feature.node.kubevirt.io/rdtscp: true
cpu-feature.node.kubevirt.io/sep: true
cpu-feature.node.kubevirt.io/sha-ni: true
cpu-feature.node.kubevirt.io/skip-l1dfl-vmentry: true
cpu-feature.node.kubevirt.io/smap: true
cpu-feature.node.kubevirt.io/smep: true
cpu-feature.node.kubevirt.io/spec-ctrl: true
cpu-feature.node.kubevirt.io/ssbd: true
cpu-feature.node.kubevirt.io/sse: true
cpu-feature.node.kubevirt.io/sse2: true
cpu-feature.node.kubevirt.io/sse4.1: true
cpu-feature.node.kubevirt.io/sse4.2: true
cpu-feature.node.kubevirt.io/sse4a: true
cpu-feature.node.kubevirt.io/ssse3: true
cpu-feature.node.kubevirt.io/stibp: true
cpu-feature.node.kubevirt.io/svm: true
cpu-feature.node.kubevirt.io/svme-addr-chk: true
cpu-feature.node.kubevirt.io/syscall: true
cpu-feature.node.kubevirt.io/tsc: true
cpu-feature.node.kubevirt.io/tsc-deadline: true
cpu-feature.node.kubevirt.io/tsc-scale: true
cpu-feature.node.kubevirt.io/tsc_adjust: true
cpu-feature.node.kubevirt.io/umip: true
cpu-feature.node.kubevirt.io/v-vmsave-vmload: true
cpu-feature.node.kubevirt.io/vgif: true
cpu-feature.node.kubevirt.io/virt-ssbd: true
cpu-feature.node.kubevirt.io/vmcb-clean: true
cpu-feature.node.kubevirt.io/vme: true
cpu-feature.node.kubevirt.io/wbnoinvd: true
cpu-feature.node.kubevirt.io/x2apic: true
cpu-feature.node.kubevirt.io/xgetbv1: true
cpu-feature.node.kubevirt.io/xsave: true
cpu-feature.node.kubevirt.io/xsavec: true
cpu-feature.node.kubevirt.io/xsaveerptr: true
cpu-feature.node.kubevirt.io/xsaveopt: true
cpu-model-migration.node.kubevirt.io/EPYC-Rome: true
cpu-model-migration.node.kubevirt.io/Nehalem: true
cpu-model-migration.node.kubevirt.io/Nehalem-IBRS: true
cpu-model-migration.node.kubevirt.io/Opteron_G1: true
cpu-model-migration.node.kubevirt.io/Opteron_G2: true
cpu-model-migration.node.kubevirt.io/Penryn: true
cpu-model-migration.node.kubevirt.io/SandyBridge: true
cpu-model-migration.node.kubevirt.io/SandyBridge-IBRS: true
cpu-model-migration.node.kubevirt.io/Westmere: true
cpu-model-migration.node.kubevirt.io/Westmere-IBRS: true
cpu-model.node.kubevirt.io/Nehalem: true
cpu-model.node.kubevirt.io/Nehalem-IBRS: true
cpu-model.node.kubevirt.io/Opteron_G1: true
cpu-model.node.kubevirt.io/Opteron_G2: true
cpu-model.node.kubevirt.io/Penryn: true
cpu-model.node.kubevirt.io/SandyBridge: true
cpu-model.node.kubevirt.io/SandyBridge-IBRS: true
cpu-model.node.kubevirt.io/Westmere: true
cpu-model.node.kubevirt.io/Westmere-IBRS: true
cpu-timer.node.kubevirt.io/tsc-frequency: 2495312000
cpu-timer.node.kubevirt.io/tsc-scalable: true
cpu-vendor.node.kubevirt.io/AMD: true
cpumanager: false
host-model-cpu.node.kubevirt.io/EPYC-Rome: true
host-model-required-features.node.kubevirt.io/amd-ssbd: true
host-model-required-features.node.kubevirt.io/arch-capabilities: true
host-model-required-features.node.kubevirt.io/cmp_legacy: true
host-model-required-features.node.kubevirt.io/hypervisor: true
host-model-required-features.node.kubevirt.io/ibrs: true
host-model-required-features.node.kubevirt.io/invtsc: true
host-model-required-features.node.kubevirt.io/lbrv: true
host-model-required-features.node.kubevirt.io/lfence-always-serializing: true
host-model-required-features.node.kubevirt.io/mds-no: true
host-model-required-features.node.kubevirt.io/null-sel-clr-base: true
host-model-required-features.node.kubevirt.io/pause-filter: true
host-model-required-features.node.kubevirt.io/pfthreshold: true
host-model-required-features.node.kubevirt.io/pschange-mc-no: true
host-model-required-features.node.kubevirt.io/rdctl-no: true
host-model-required-features.node.kubevirt.io/skip-l1dfl-vmentry: true
host-model-required-features.node.kubevirt.io/spec-ctrl: true
host-model-required-features.node.kubevirt.io/ssbd: true
host-model-required-features.node.kubevirt.io/stibp: true
host-model-required-features.node.kubevirt.io/svme-addr-chk: true
host-model-required-features.node.kubevirt.io/tsc-deadline: true
host-model-required-features.node.kubevirt.io/tsc-scale: true
host-model-required-features.node.kubevirt.io/tsc_adjust: true
host-model-required-features.node.kubevirt.io/v-vmsave-vmload: true
host-model-required-features.node.kubevirt.io/vgif: true
host-model-required-features.node.kubevirt.io/virt-ssbd: true
host-model-required-features.node.kubevirt.io/vmcb-clean: true
host-model-required-features.node.kubevirt.io/x2apic: true
hyperv.node.kubevirt.io/base: true
hyperv.node.kubevirt.io/frequencies: true
hyperv.node.kubevirt.io/ipi: true
hyperv.node.kubevirt.io/reenlightenment: true
hyperv.node.kubevirt.io/reset: true
hyperv.node.kubevirt.io/runtime: true
hyperv.node.kubevirt.io/synic: true
hyperv.node.kubevirt.io/synic2: true
hyperv.node.kubevirt.io/synictimer: true
hyperv.node.kubevirt.io/time: true
hyperv.node.kubevirt.io/tlbflush: true
hyperv.node.kubevirt.io/vpindex: true
instance.hetzner.cloud/is-root-server: true
k8slens-edit-resource-version: v1
kubearmor.io/apparmorfs: no
kubearmor.io/btf: no
kubearmor.io/enforcer: bpf
kubearmor.io/rand: 7s7j
kubearmor.io/runtime: cri-o
kubearmor.io/seccomp: yes
kubearmor.io/securityfs: yes
kubearmor.io/socket: var_run_crio_crio.sock
kubernetes.io/arch: amd64
kubernetes.io/hostname: NODENAME
kubernetes.io/os: linux
kubevirt.io/ksm-enabled: false
kubevirt.io/schedulable: true
kubevirt.io/sev:
node-access: protected
node-role.kubernetes.io/control-plane: true
node-role.kubernetes.io/master: true
scheduling.node.kubevirt.io/tsc-frequency-2495310000: true
scheduling.node.kubevirt.io/tsc-frequency-2495312000: true
topology.kubernetes.io/region: REGION
topology.rook.io/datacenter: DC
topology.rook.io/rack: RACKfrom kubearmor.
rksharma95
commented on June 10, 2024
kubearmor.io/btf: no
for some reasons it detected that btf is not present or simply /sys/kernel/btf/vmlinux file is not present on the node. can you verify it by accessing the node?
from kubearmor.
aberfeldy
commented on June 10, 2024
no it's not on the machine
# ls /sys/kernel/btf/vmlinux
ls: /sys/kernel/btf/vmlinux: No such file or directory
from kubearmor.
daemon1024
commented on June 10, 2024
@aberfeldy is it a custom kernel build? Is it possible for you to enable BTF in your kernel?
Ref flatcar-archive/coreos-overlay#753
from kubearmor.
aberfeldy
commented on June 10, 2024
will have a look into that
from kubearmor.
aberfeldy
commented on June 10, 2024
compiled kernel with BTF, works like a charm.
just one minor thing, whenever I install via helm the service-account for the snitch jobs is missing. only after applying the KubeArmorConfig it was created. until then the jobs were hanging in pending since the SA wasn't present.
thanks for the support!
from kubearmor.
daemon1024
commented on June 10, 2024
Great to hear @aberfeldy
whenever I install via helm the service-account for the snitch jobs is missing. only after applying the KubeArmorConfig it was created. until then the jobs were hanging in pending since the SA wasn't present.
@rksharma95 is that intended, or do we need to fix this?
from kubearmor.
rksharma95
commented on June 10, 2024
Great to hear @aberfeldy
whenever I install via helm the service-account for the snitch jobs is missing. only after applying the KubeArmorConfig it was created. until then the jobs were hanging in pending since the SA wasn't present.
@rksharma95 is that intended, or do we need to fix this?
Yes it's expected behavior, we only deploy resources after kubeconfig has been created.
from kubearmor.
aberfeldy
commented on June 10, 2024
I get the intention, but I'd suggest moving the snitch creation so that the jobs are created after the config was applied as well. Otherwise you have jobs with no SA and they can never be fulfilled or can be seen as erroneous or failed by monitoring tools
from kubearmor.
aberfeldy
commented on June 10, 2024
Thank you so much, at the moment there is nothing else to report, I'm a happy engineer with a new toy. Keep up the great work, really love how it works.
Will close this for now, since everything is fine
from kubearmor.
Related Issues (20)
- Container Security Capabilities for Graviton Instances HOT 4
- KubeArmor daemonset pods failing to deploy successfully on x86 Ubuntu EKS-A Baremetal Env. HOT 5
- makefile go build && go mod tidy behaves differently starting with Go 1.21, leading to unknown directive toolchain errors HOT 1
- Snitch should not mount entire host rootfs HOT 2
- Request: support for taints and tolerations to KubeArmor deployments in Helm charts. HOT 4
- Update Helm values to reflect sample-config.yml HOT 3
- the "/usr/bin/sleep 1" example policy does not work when installation on a ubuntu22.04.4 VM using systemd mode HOT 8
- Way to constrain only specific selectors HOT 5
- Kubearmor does not enforce anything on pod's entrypoint or execs to kubernetes pods
- kubearmor helm charts on artifacthub HOT 7
- Improve system test coverage and practices for KubeArmor HOT 3
- Alert Policy for KubeArmor HOT 1
- `make run` fails in Ubuntu 24.02 HOT 1
- CI test suite does not generate coverage report
- Host Security Policy not enforced HOT 2
- KubeArmor Operator Enhancements
- Installing the KubeArmorOperator throws an error when the "kubarmor" namespace is not created in the cluster
- Cluster Scoped KubeArmor Security Policy
- Refactor KubeArmor Operator
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kubearmor.