kata-containers / shim Goto Github PK

View Code? Open in Web Editor NEW

67.0 67.0 49.0 14.33 MB

Kata Containers version 1.x shim (for version 2.x see https://github.com/kata-containers/kata-containers).

Home Page: https://katacontainers.io/

License: Apache License 2.0

Makefile 3.05% Go 92.40% Shell 4.55%

shim's Introduction

Kata Containers

Welcome to Kata Containers!

This repository is the home of the Kata Containers code for the 2.0 and newer releases.

If you want to learn about Kata Containers, visit the main Kata Containers website.

Introduction

Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workload isolation and security advantages of VMs.

License

The code is licensed under the Apache 2.0 license. See the license file for further details.

Platform support

Kata Containers currently runs on 64-bit systems supporting the following technologies:

Architecture	Virtualization technology
`x86_64`, `amd64`	Intel VT-x, AMD SVM
`aarch64` ("`arm64`")	ARM Hyp
`ppc64le`	IBM Power
`s390x`	IBM Z & LinuxONE SIE

Hardware requirements

The Kata Containers runtime provides a command to determine if your host system is capable of running and creating a Kata Container:

$ kata-runtime check

Notes:

This command runs a number of checks including connecting to the network to determine if a newer release of Kata Containers is available on GitHub. If you do not wish this to check to run, add the --no-network-checks option.

By default, only a brief success / failure message is printed. If more details are needed, the --verbose flag can be used to display the list of all the checks performed.

If the command is run as the root user additional checks are run (including checking if another incompatible hypervisor is running). When running as root, network checks are automatically disabled.

Getting started

See the installation documentation.

Documentation

See the official documentation including:

Configuration

Kata Containers uses a single configuration file which contains a number of sections for various parts of the Kata Containers system including the runtime, the agent and the hypervisor.

Hypervisors

See the hypervisors document and the Hypervisor specific configuration details.

Community

To learn more about the project, its community and governance, see the community repository. This is the first place to go if you wish to contribute to the project.

Getting help

See the community section for ways to contact us.

Raising issues

Please raise an issue in this repository.

Note: If you are reporting a security issue, please follow the vulnerability reporting process

Developers

See the developer guide.

Components

Main components

The table below lists the core parts of the project:

Component	Type	Description
runtime	core	Main component run by a container manager and providing a containerd shimv2 runtime implementation.
runtime-rs	core	The Rust version runtime.
agent	core	Management process running inside the virtual machine / POD that sets up the container environment.
`dragonball`	core	An optional built-in VMM brings out-of-the-box Kata Containers experience with optimizations on container workloads
documentation	documentation	Documentation common to all components (such as design and install documentation).
tests	tests	Excludes unit tests which live with the main code.

Additional components

The table below lists the remaining parts of the project:

Component	Type	Description
packaging	infrastructure	Scripts and metadata for producing packaged binaries (components, hypervisors, kernel and rootfs).
kernel	kernel	Linux kernel used by the hypervisor to boot the guest image. Patches are stored here.
osbuilder	infrastructure	Tool to create "mini O/S" rootfs and initrd images and kernel for the hypervisor.
kata-debug	infrastructure	Utility tool to gather Kata Containers debug information from Kubernetes clusters.
`agent-ctl`	utility	Tool that provides low-level access for testing the agent.
`kata-ctl`	utility	Tool that provides advanced commands and debug facilities.
`trace-forwarder`	utility	Agent tracing helper.
`runk`	utility	Standard OCI container runtime based on the agent.
`ci`	CI	Continuous Integration configuration files and scripts.
`ocp-ci`	CI	Continuous Integration configuration for the OpenShift pipelines.
`katacontainers.io`	Source for the `katacontainers.io` site.
`Webhook`	utility	Example of a simple admission controller webhook to annotate pods with the Kata runtime class

Packaging and releases

Kata Containers is now available natively for most distributions.

General tests

See the tests documentation.

Metrics tests

See the metrics documentation.

Glossary of Terms

See the glossary of terms related to Kata Containers.

shim's People

Contributors

Stargazers

Watchers

shim's Issues

add contribution guidelines

terminal: Fix setup of raw terminal

We should be able to prevent the import of github.com/moby/moby/pkg/term package since we can program the raw terminal with some simple functions coming from the more generic package golang.org/x/sys/unix.
Also, we need to ensure we are setting the raw terminal only if STDIN is a terminal, otherwise the function call will fail.

Update agent vendoring

We need to update the agent vendoring to allow for future changes (such as #123).

Clean up logging calls

The shim code should call logger() rather than reference shimLog where at all possible.

use exec id instead of pid and wait on SIGUSR1

As we discussed in kata-containers/agent#72, we will use runtime generated exec id to uniquely identify a process, replacing current pid returned by agent.

And in order to support runtime create --pid-file option, we need to start shim before starting the container. So pause shim on SIGUSR1 if it is an init process shim, in which case container ID equals to exec ID. Runtime should wait up shim after it calls StartContainer().

Needs 2 approval to be able to merge a pull request

It would be better to rely on 2 approvals to consider a PR as mergeable.

travis: Add support for ppc64le

travis: Fix static-checks

Require doc team sign-off for docs

We should require an "ack" from the documentation team before merging any doc changes on PRs.

ci: lib: allow override of tests_repo

Only use the default tests_repo if one is not already set

shim unit tests fail on CI

Taking a look at the CI jobs for the shim repository, I see that there are failures in the unit tests.

01:06:46 INFO: Running 'go test' as root user on package 'github.com/kata-containers/shim' with flags '-v -race -timeout 10s'
01:06:53 === RUN   TestNewShimAgent
01:06:53 --- PASS: TestNewShimAgent (0.01s)
01:06:53 === RUN   TestAddContainer
01:06:53 --- PASS: TestAddContainer (0.01s)
01:06:53 === RUN   TestInitLogger
01:06:53 --- PASS: TestInitLogger (0.00s)
01:06:53 === RUN   TestPipe
01:06:53 --- PASS: TestPipe (0.01s)
01:06:53 === RUN   TestNewShim
01:06:53 --- PASS: TestNewShim (0.01s)
01:06:53 === RUN   TestShimOps
01:06:53 --- PASS: TestShimOps (0.02s)
01:06:53 === RUN   TestSetupTerminalOnNonTerminalFailure
01:06:53 --- PASS: TestSetupTerminalOnNonTerminalFailure (0.00s)
01:06:53 === RUN   TestSetupTerminalSuccess
01:06:53 --- FAIL: TestSetupTerminalSuccess (0.00s)
01:06:53 	<autogenerated>:1: 
                          
	Error Trace:	terminal_linux_test.go:45
01:06:53 		
	Error:      	Expected nil, but got: &os.PathError{Op:"open", Path:"/dev/tty", Err:0x6}
01:06:53 		
	Test:       	TestSetupTerminalSuccess
01:06:53 		
	Messages:   	Failed to create terminal
01:06:53 	<autogenerated>:1: 
                          
	Error Trace:	terminal_linux_test.go:49
01:06:53 		
	Error:      	Expected nil, but got: 0x9
01:06:53 		
	Test:       	TestSetupTerminalSuccess
01:06:53 		
	Messages:   	Should not fail because the file is a terminal
01:06:53 	<autogenerated>:1: 
                          
	Error Trace:	terminal_linux_test.go:52
01:06:53 		
	Error:      	Expected nil, but got: 0x9
01:06:53 		
	Test:       	TestSetupTerminalSuccess
01:06:53 		
	Messages:   	Failed to get terminal information
01:06:53 	<autogenerated>:1: 
                          
	Error Trace:	terminal_linux_test.go:57
01:06:53 		
	Error:      	Should be true
01:06:53 		
	Test:       	TestSetupTerminalSuccess
01:06:53 		
	Messages:   	Termios C flag should be properly set in raw mode
01:06:53 	<autogenerated>:1: 
                          
	Error Trace:	terminal_linux_test.go:58
01:06:53 		
	Error:      	Should be true
01:06:53 		
	Test:       	TestSetupTerminalSuccess
01:06:53 		
	Messages:   	Termios CC VMIN value should be properly set in raw mode
01:06:53 	<autogenerated>:1: 
                          
	Error Trace:	terminal_linux_test.go:62
01:06:53 		
	Error:      	Expected nil, but got: 0x9
01:06:53 		
	Test:       	TestSetupTerminalSuccess
01:06:53 		
	Messages:   	Terminal should be properly restored
01:06:53 FAIL
01:06:53 coverage: 39.9% of statements
01:06:53 exit status 1
01:06:53 FAIL	github.com/kata-containers/shim	0.072s
01:06:53 Makefile:23: recipe for target 'test' failed
01:06:53 make: *** [test] Error 1

Complete Logs:
http://kata-jenkins-ci.westus2.cloudapp.azure.com/job/kata-containers-shim-ubuntu-16-04-master/15/console
http://kata-jenkins-ci.westus2.cloudapp.azure.com/job/kata-containers-shim-ubuntu-17-10-master/1/console

Build shim as position independent executable (PIE)

This is a follow up of kata-containers/runtime#875, refer to that issues for details and motivations.

Allow SIGWINCH sent to shim to be forwarded to container

PR #100 is fixing a problem, but not the one documented on #74...

docker kill -s WINCH now works, but kill -WINCH $shim_pid does not.

coredump on internal error

When a C program crashes, it dumps core. This doesn't happen with golang programs. From https://golang.org/pkg/runtime/:

By default, a failure prints a stack trace for the current goroutine, eliding functions internal to the run-time system, and then exits with exit code 2.

This behaviour is sub-optimal in our scenario for 2 reasons:

No full goroutine backtrace.
No core dump.

Modify the code to always coredump on internal failure so that automatic crash handling systems have an opportunity to run and capture details of the crash.

Use central go test script

Use the central go-test.sh in the tests repo for consistency.

CI: update static-checks.sh in travis.yml

Remove unused test code

agent_test.go contains some unused code.

Enable checkcommits

Enable static checks via kata-containers/tests#7

log: Enable syslog logging

It's important for debug that we centralize all logs to the same place. The easiest way for that is to use syslog. We will be able to check the logs by running the following command:

journalctl -f

Also, the shim has to ensure it does print anything to stdout, otherwise the user will get some traces coming from the shim while he would expect traces from the workload only.

Fix initial log call

The initial shim log call only contains fields - there is no standard msg field.

Improve the messages logged when failing to resize window

shim.resizeTty() could do with some minor tweaks to improve logging on error.

Make sure the output will reach the shim

Sometimes, when the CloseStdin() call is invoked before the shim got all the output from the corresponding container, we're actually missing some of this output.
We should make sure we close stdin only when we got everything from stdout and stderr.

Add CI scripts for the environment setup and integration tests execution

Add .ci/setup.sh and .ci/run.sh to install the environment for launching Kata Containers and to execute the integration tests.

Fix logger identifier

The syslog hooks needs to be passed the name of the application to avoid it defaulting to the path to the current application.

dep broken?

$ go get -u github.com/golang/dep/cmd/dep
$ go get github.com/kata-containers/shim
$ cd $GOPATH/src/github.com/kata-containers/shim
Solving failure: No versions of github.com/kata-containers/agent met constraints:
        8f22514ae5790f3b6953cf93692e663fa29469e3: Could not introduce github.com/kata-containers/agent@8f22514ae5790f3b6953cf93692e663fa29469e3, as it has a dependency on google.golang.org/grpc with constraint 5a9f7b402fe85096d2e1d0383435ee1876e863d0, which has no overlap with existing constraint ^1.8.0 from (root)                                                                 
        1.4.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.3.1: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.3.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.2.2: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.2.1: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.2.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.1.1: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.1.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.0.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        0.3.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        0.2.0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        0.0.1: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.3.0-rc1: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        1.3.0-rc0: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        master: Could not introduce github.com/kata-containers/agent@master, as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        stable-1.1: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        stable-1.2: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.
        stable-1.3: Could not introduce github.com/kata-containers/[email protected], as it is not allowed by constraint 8f22514ae5790f3b6953cf93692e663fa29469e3 from project github.com/kata-containers/shim.

Update bash on Travis osx for hash support

As shown on kata-containers/proxy#90, the osx environment provided by Travis provides a version of bash that does not support typeset -A. As of kata-containers/tests#471, that is now required for the static analysis CI phase, so we need to install a newer bash using homebrew.

Support darwin build for kata shim

Description of problem

Opening up to track building the Kata Shim on macos. Needed to support Kata Containers over xhyve

Expected result

$ cd $GOPATH/src/github.com/kata-containers/shim && make
go build -o kata-shim -ldflags "-X main.version=1.0.0-a69326b63802952b14203ea9c1533d4edb8c1d64"
# Builds the kata-shim exec on darwin

Actual result

$ cd $GOPATH/src/github.com/kata-containers/shim && make
go build -o kata-proxy -ldflags "-X main.version=1.0.0-a69326b63802952b14203ea9c1533d4edb8c1d64"
# github.com/kata-containers/shim
./signals.go:27:2: undefined: syscall.SIGSTKFLT
make: *** [kata-proxy] Error 2

nsenter support

In docker case, a container can join the network namespace of another container. In runv, we solved it by letting runv shim own the network namespace and if a new container needs to enter the netns, we make its shim join the netns.

@sameo @sboeuf how does cc-runtime support the case?

Make sure output is in right order

We need to make sure the shim is not calling into ReadStderr(), expecting some stderr output, when the terminal is set. In that case, we should only expect stdout output, and that's why we are getting wrong ordered output.

Duplicates kata-containers/agent#125

Reduce footprint by implementing shim with other language

The shim represents the container process on the host side. And for this reason, we have one shim per process, meaning we'll have one shim for the container process, but we'll also have one for every exec on an existing container.
If you look at this from a k8s perspective, we might have a lot of containers and associated processes inside a pod, raising the density per pod.

In order to improve density, we should consider re-writing the shim in a lighter weight language leveraging some dynamically loaded libraries by writing it in C or Rust.

The constraint is to make sure we would rely on a solid gRPC library as we don't want to maintain our own, and try to catch up with changes along the time.

Restrict the max number of golang threads that will be used

Description of problem

It has been seen that the proxy will, over time under certain circumstances, grow more and more processor threads (possibly via the go runtime garbage collector), which then predominantly sit idle. They do however consume host PIDs, which then limits the number of containers we can run on a given host (there is a finite, if somewhat large, limit to the number of active PIDs).
Although this symptom has not been seen on the shim yet, it makes sense to set a limit here as well, as it will reduce some golang internal resource utilisation and guard against any future similar issues.

See:
kata-containers/runtime#807
for more details and diagnostics.

One fix is to set the GOMAXPROCS setting in the golang runtime to restrict/reduce the number of threads that will be consumed.

Fix lint errors

Fix the outstanding gometalinter issues.

Fix race condition with WaitProcess()

Because WaitProcess() deletes everything related to the process from the agent, we should make sure any further call involving the same process are issued after WaitProcess() returned.

Fix staticcheck lint errors

terminal_linux_test.go:36:2:warning: should check returned error before deferring file.Close() (SA5001) (staticcheck)
terminal_linux_test.go:45:2:warning: should check returned error before deferring file.Close() (SA5001) (staticcheck)

add execution permissions to setup.sh and run.sh CI scripts.

Add opentracing support.

See: kata-containers/kata-containers#27.

Don't log I/O errors unless there is one

In a couple of places, shim.proxyStdio() is unconditionally logging errors, even when there isn't one.

make: add suport for DESTDIR

Add support for DESTDIR, to be used when installing kata-shim in a root filesystem other than '/'.

DESTDIR is already supported in the other kata projects, so it makes sense to have it here too :)

Improve logger

There is one shim / container process so add container + process details to all log calls.

Apply user, group and perms to install files

Rather than just calling install -D in Makefile, call:

INSTALL_USER = root
INSTALL_GROUP = root
INSTALL_MODE = 0644

    :

install:
    install --owner="$(INSTALL_USER)" --group="$(INSTALL_GROUP)" --mode="$(INSTALL_MODE)" -D ...

The minimum set of permissions should be applied in all cases. This issue applies to shim, runtime, proxy and agent.

See: #24 (review)

Rely on SIGUSR1 signal to signal the shim to connect

This might be a temporary solution, but this is the easiest way to make sure we can start the shim any time in order to get the PID, and that we can really control when to make him connect the proxy/agent.

Add github issue template

Copy https://github.com/kata-containers/runtime/blob/master/.github/ISSUE_TEMPLATE.md into this repo.

SIGKILL handling

I'm leaving the SIGKILL handling out of the first PR, so that we can find a long term solution.

In short, we need a monitor process somewhere to see if the shim is killed, and send SIGKILL to container/process as well.

Possible candidates are:

shim spawns a monitor process
runtime monitors shim
higher level handling (is it possible?)

OTOH, we need to decide if we want to support shim-reconnection in some cases (like upgrade?) and how to distinguish that from shim being properly stopped.

WDYT? @laijs @sameo @sboeuf @WeiZhang555

vendor: Bump logrus to latest commit

Logrus brought some improvements, particularly for syslog support. We should revendor to the latest version.

Make faketty pass return code back to caller

.ci/faketty.sh doesn't work as expected:

$ .ci/faketty.sh true; echo $?
0
$ .ci/faketty.sh false; echo $?
0

docker kill SIGWINCH is not forwarded to container

The following issue clearcontainers/runtime#768 is still present in kata-containers.

shim info messages going to stderr when not using a terminal and to stdout when using a terminal

When running kata containers with docker without using a terminal, the shim messages are being redirected to stderr and when using a terminal they are send to stdout

Without terminal and without redirecting stderr:

fuentess@kata-vm:~$ sudo docker run --runtime=kata-runtime -i --rm -u root:postgres postgres id
time="2018-02-13T17:45:37.611156562Z" level=info name=kata-shim pid=108755 version=0.0.1-b8f7336791818eea7f9e756a070b3e95eed801d2
time="2018-02-13T17:45:37.611581865Z" level=info msg="Error getting size" error="inappropriate ioctl for device"
time="2018-02-13T17:45:37.863835289Z" level=info msg="copy stderr failed" error="<nil>"
uid=0(root) gid=999(postgres) groups=999(postgres)
time="2018-02-13T17:45:37.86657301Z" level=info msg="copy stdout failed" error="<nil>"
time="2018-02-13T17:45:37.882742234Z" level=info msg="using shim to proxy exit code" exitcode=0 name=kata-shim pid=108755

Without terminal and redirecting stderr to /dev/null:

fuentess@kata-vm:~$ sudo docker run --runtime=kata-runtime -i --rm -u root:postgres postgres id 2> /dev/null 
uid=0(root) gid=999(postgres) groups=999(postgres)

With terminal and redirecting stdout to /dev/null:

fuentess@kata-vm:~$ sudo docker run --runtime=kata-runtime -it --rm -u root:postgres postgres id 1> /dev/null 
fuentess@kata-vm:~$

makefile: Add "install" case

We need to be able to install kata-shim from the Makefile by using:

sudo make install

Backtrace on SIGUSR1

Implement same logic as on kata-containers/runtime#241.

Fix logging

Currently, the shim sends log output to stderr and doesn't show any timestamps in the syslog entries.

kata-containers / shim Goto Github PK

shim's Introduction

Kata Containers

Introduction

License

Platform support

Hardware requirements

Getting started

Documentation

Configuration

Hypervisors

Community

Getting help

Raising issues

Developers

Components

Main components

Additional components

Packaging and releases

General tests

Metrics tests

Glossary of Terms

shim's People

Contributors

Stargazers

Watchers

Forkers

shim's Issues

Description of problem

Expected result

Actual result

Description of problem

Recommend Projects

Recommend Topics

Recommend Org