Add `OCI` image support about lxd HOT 18 CLOSED

I published the PR for OCI support if anyone wants to follow along :)

Since last time, I've basically added:

• Basic env variable support (needs to be moved to image unpack)
• Usage of liblxc's execute rather than start method (needed a go-lxc fix)
• Proper handling of entry point containing spaces or being relative
• Fixed the console handling
• DHCP client (IPv4 now working)

So the main bits to do at this point are:

• Make it possible to distinguish app vs system containers in the API and CLI
• Cleanup DHCP client to avoid failures on less common network setups
• Move environment handling to image unpack (set environment config keys)
• Add port handling during image unpack (create proxy devices)
• Add config keys for auto-restart of the instance
• Improve performance of image conversion and add progress reporting

from lxd.

So I've basically got support for OCI images throughout the codebase and rely on skopeo and umoci to access image servers and convert things into image files we can consume.

stgraber@dakara:~$ incus image info 266b1
Fingerprint: 266b191e926f65542fa8daaec01a192c4d292bff79426f47300a046e1bc576fd
Size: 0.00MiB
Architecture: x86_64
Type: container
Public: no
Timestamps:
    Created: 2023/05/02 12:49 EDT
    Uploaded: 2024/06/05 01:14 EDT
    Expires: never
    Last used: 2024/06/05 01:20 EDT
Properties:
Aliases:
Cached: yes
Auto update: enabled
Source:
    Server: https://docker.io
    Protocol: oci
    Alias: hello-world
Profiles:
    - default

This is all working fine and will even get automatic background updates on images and everything, same as what we have with simplestreams.

It's missing a lot of optimizations though, the current process downloads a ton of stuff to disk just to repack them and send them back over the network. Ideally we should be able to do all of that as data is streaming through.

But this basically unblocks getting the rest of the work done, which is having the LXC driver detect application containers and automatically run them with all the right settings.

from lxd.

Yeah, I'm currently working on the network configuration side of things and already have a minimal DHCP client written in Go, just sorting out the hooks to have it called and be able to safely configure the network in the container.

from lxd.

@cyphar @hallyn @tych0 so I'm guessing the way to go for the OCI image retrieval and conversion into a format we can handle (just merging all layers together, we'll do something more optimized later) would be to use umoci's Go packages?

from lxd.

umoci doesn't support retrieval, for that I used containers/image, which is a redhat library that backs skopeo. It is a bit annoying because it brings in a bunch of shared object dependencies, most of which you can disabel.

For extraction and manipulation once you have them downloaded, though, umoci is the way to go.

from lxd.

I think it is also worth exporting the exit code of these app containers somehow, as it's a thing people care about. I was intending to do that, but have gotten distracted by other things :)

from lxd.

Ah yeah indeed, I remember us talking about the exit code and adding some liblxc logic to expose that to the stop/post-stop hooks.

from lxd.

stgraber@dakara:~$ incus launch docker:hello-world demo-oci -c raw.lxc=lxc.init.cmd=/hello
Launching demo-oci
stgraber@dakara:~$ incus console --show-log demo-oci

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

from lxd.

A few more things I've noticed that we'll need to sort out:

• Network configuration. Containers don't do that, so we need our init shim to handle it.
• --console on incus launch or incus start doesn't work on very short lived containers like hello-world as the container is stopped by the time we try to connect to it. To fix this we should both have the CLI handle the stopped case by pulling the log instead AND have the init shim wait a couple of seconds before executing the entry point, giving us time to connect.
• We need to do smarter parsing/escaping of lxc.init_cmd to handle commands with arguments containing spaces
• Definitely need to have GetImageFile return at least some kind of progress information given how slow the process can be

from lxd.

stgraber@dakara:~$ incus launch docker:hello-world --ephemeral --console
Launching the instance
Instance name is: primary-fawn

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

from lxd.

Woah! That was fast!

from lxd.

The devil's always in the details, so while we have something fun and flashy right now, it's likely going to take a bit longer to sort out all those small issues...

from lxd.

I have questions. :)

How would docker-compose files be handled? Or would the user define the container launch parameters via command line? Will this handle .env files, otherwise potentially passwords are going to be sloshing about in clear text?

As a general comment, the ability to launch containers/VM from a config file held in a git would be great. Then a hook to trigger a redeployment of the container after a change to the config would be chef's kiss

incus launch docker:hello-world --ephemeral --console --configfile git:hello-world --poll 5m

from lxd.

How would docker-compose files be handled? Or would the user define the container launch parameters via command line? Will this handle .env files, otherwise potentially passwords are going to be sloshing about in clear text?

I don't expect we will be handling docker-compose files, at least not initially.
Our focus is really on running OCI images natively and not on trying to achieve parity with Docker itself or its tools ecosystem.

For the .env, we support feeding YAML directly to incus create or incus launch so that's likely how you'd avoid passing your environment variables through the globally visible command line.

Our ability to do incus launch docker:hello-world --ephemeral --console < config.yaml is also how someone would likely implement your second comment. Having us directly understand how to interact and monitor git feels out of scope, but the fact that we support reading a full YAML config from stdin already makes scripting something like that pretty easy.

from lxd.

Nice! @stgraber you've probably looked into it already, but we hacked a LXC OCI template back in the days: https://github.com/lxc/lxc/blob/main/templates/lxc-oci.in
Not sure if some of the logic could be reused here.

from lxd.

Yep, that's what I used to figure out most of the logic and tools needed.

Incus doesn't use LXC templates and works very differently in the way it handles images and creates the storage part of the container so we can't straight up use the script but it made for a good starting point and a good way to make sure I didn't miss anything.

from lxd.

Ok yeah makes sense, just checking.

Depending on your kernel requirements you could also avoid umoci altogether with unpriv overlay (i.e. parallel aufs2ovlfs + tar / mksquash the mount), this should provide significant speedup.

The DHCPv4 hook was kind of a pain, but I guess for this one you can bundle a small Go dhcpclient.

from lxd.

This is what I need. Thank you!

from lxd.