[RFC] Return Config from a manifest (as an Image property) about image HOT 15 CLOSED

I don’t know; my basic question is “to what extent is the “config” a stable concept likely to exist in the future, and to what extent is this likely to be true for individual fields in that config?", and I mostly have no idea.

Implementation-wise, IIRC there already are Docker→runc config conversion tools, and it would be Nice™ to maintain that code once instead of having separate reimplementations, each supporting varying subsets of the formats and versions.

Though that can mean either that containers/image will just return a config blob + MIME type [do the inlined configs in old Docker manifests even have a MIME type?], and some other library will do the parsing and interpretation, or that containers/image will become the universal interface for accessing and converting configs, calling third-party code to do the actual conversions.

Considering the various discussions about skopeo copy etc. doing conversions between image formats, both packed (manifest+tarballs→manifest+tarballs) and unpacked (manifest+tarballs→config+prepared FS in a graph driver, and perhaps the opposite as well), I suspect some level of parsing/understanding the configs within containers/image will become necessary; and at that point giving all callers a format-independent inspect-like interface for the configs would be natural.

Still, the more interfaces we expose, and the more formats we support, the harder it will be to keep API compatibility and to keep the semantics consistent across different image formats. (In this sense, the existing Image.Inspect is nice in that it hides the existence of configs from callers; everything is just an Image object.) And if we start with inspect, soon will be talking about editing the config…

I guess, overall, sure, why not; to prevent overbuilding, let’s focus on APIs with clear use cases (ideally replacing existing private-use code with calls into the shared containers/image library).

from image.

I don’t know; my basic question is “to what extent is the “config” a stable concept likely to exist in the future, and to what extent is this likely to be true for individual fields in that config?", and I mostly have no idea.

I understand your concern, however I'm sure it's a stable concept, used in the early days as well and apart from moving from inline config in a manifest to blob nothing has changed substantially.

Implementation-wise, IIRC there already are Docker→runc config conversion tools, and it would be Nice™ to maintain that code once instead of having separate reimplementations, each supporting varying subsets of the formats and versions.

Docker -> runc config conversion is sadly just done from containers (as opposed to images). The tool you're probably talking about is riddler which can output a config.json from a docker inspect CONTAINER - not an image :(

I believe we can tweak that tool to work with image config as well - and use runc config.json default values for the fields not populated in an image config (like Devices - they are only populated at run time, the image has no notion of these).
I believe it's valuable to do "image" -> "runc config.json" using default values for unknown fields - and also I expect this should work for unknown fields. @mrunalp what do you think about this? see also my view of ocid+graphtool+containers/image below

Though that can mean either that containers/image will just return a config blob + MIME type [do the inlined configs in old Docker manifests even have a MIME type?],

no MIME type :(

and some other library will do the parsing and interpretation, or that containers/image will become the universal interface for accessing and converting configs, calling third-party code to do the actual conversions.

Considering the various discussions about skopeo copy etc. doing conversions between image formats, both packed (manifest+tarballs→manifest+tarballs) and unpacked (manifest+tarballs→config+prepared FS in a graph driver,

I'm still not sure about containers/image doing "prepared FS in a graph driver" but you probably have a better view of this than I have :/

and perhaps the opposite as well), I suspect some level of parsing/understanding the configs within containers/image will become necessary; and at that point giving all callers a format-independent inspect-like interface for the configs would be natural.

Still, the more interfaces we expose, and the more formats we support, the harder it will be to keep API compatibility and to keep the semantics consistent across different image formats. (In this sense, the existing Image.Inspect is nice in that it hides the existence of configs from callers; everything is just an Image object.) And if we start with inspect, soon will be talking about editing the config…

I guess, overall, sure, why not; to prevent overbuilding, let’s focus on APIs with clear use cases (ideally replacing existing private-use code with calls into the shared containers/image library).

ACK on a stable API of course.

Btw, I would like to reiterate on why we could need this and feel free to tell me I haven't understood anything till now:

ocid <- manage oci runc containers
---> call into containers/image to download images (let's say blob+manifest and a blob can be a config in case it's an oci image or a v2s2 image)
---> call graphtool to setup the rootfs
---> spawn a container with runc

Now, the last point is not well defined because to spawn a container it needs a config.json.and it can be generated from an image config + use default value for config.json OR config provided to ocid when starting the container.

The question is: is ocid (or any other parser/tool) in charge of interpreting the raw config of an image? I think yes, then, it seems needed that containers/image should provide a clean config interface across all different manifest schemas and versions because I don't expect it or anything else to parse the config in a manifest (or in a blob).

@mtrmac @mrunalp @nalind WDYT?

from image.

For reference:

• config from an image (busybox:latest): https://paste.fedoraproject.org/388878/
• runc default config.json from "runc spec": https://paste.fedoraproject.org/388880/

I believe something from the image's config can go into runc config.json (that was my discussion above somehow) or do you guys think generating a default config.json + arguments from ocid it's sufficient here and we don't need the image config at all?

from image.

@runcom We'll have to check what are the minimum fields in the podspec. After consuming those, if there are any defaults in image's config, they could be set in the runtime config.json.

from image.

I'm still not sure about containers/image doing "prepared FS in a graph driver" but you probably have a better view of this than I have :/

Nah. I understand that something somewhere needs to do this in order to get a runnable container, but I have no idea whether containers/image is the right place for that integration. The graph driver library, or a common caller of both, could just as easily be the right place for all I know.

ocid <- manage oci runc containers
---> call into containers/image to download images (let's say blob+manifest and a blob can be a config in case it's an oci image or a v2s2 image)
---> call graphtool to setup the rootfs
---> spawn a container with runc

Sure, it can work that way from my POV. The only advantage of a more direct integration I can think of is: with an in-process integration we could unpack tarballs as they are being streamed over HTTP, without needing to store them into temporary local files and reading them back. I have no idea how much this would impact performance (presumably the tarballs could be big enough so that we don’t want them in cache, OTOH streaming big files is fairly trivial compared to all the seeking needed to actually unpack a FS).

And ultimately, to do a pull with signature verification, we need those temporary files anyway, so I guess there will be no difference in the common case.

from image.

The graph driver library, or a common caller of both, could just as easily be the right place for all I know.

that's what I meant, yes!

from image.

Yeah, common caller could be ocid for tying all this together.

from image.

@mrunalp went through the k8s API and it seems that the following fields are necessary when running containers and the admin does not specify them at runtime:

User         string              `json:"User"`                              
Memory       int                 `json:"Memory"`                            
MemorySwap   int                 `json:"MemorySwap"`                        
CpuShares    int                 `json:"CpuShares"`                         
ExposedPorts map[string]struct{} `json:"ExposedPorts"`                      
Env          []string            `json:"Env"`                               
Entrypoint   []string            `json:"Entrypoint"`                        
Cmd          []string            `json:"Cmd"`                               
Volumes      map[string]struct{} `json:"Volumes"`                           
WorkingDir   string              `json:"WorkingDir"`

The only exception is User - I opened mrunalp/ocid#5

from image.

I'm not sure anymore we need this - let me explain:

When the CRI is going to use those fields the image is downloaded and stored on the node - I don't see how we should use this library to parse the config blob and return the struct above.

IOW, if an image is already stored on disk somehow (by tools like graphtool or such) - who should provide those info to the Container Runtime Interface?

from image.

@runcom I see your point.

Does it make sense to add this to the ImageService. This then implies that kubelet will make the call the then populate the right fields in CreateSandbox/CreateContainer requests.

from image.

@runcom Another option would be to have enough information in the ImageSpec to be able to query that information. No right answer here; decision needs to be made on who handles this: kubelet or the runtime.

from image.

@mrunalp right, I thought about having an ImageConfig in ImageService but I don't really like querying (remotely) an image just to grab those configs.

from image.

@mrunalp it's also pretty clear from this https://github.com/kubernetes/kubernetes/pull/17048/files#diff-822f0e081c10d8b83d7c2ad1391d55f7R66 that the image is already stored on disk at the point we create the container (and need this config info)

from image.

@runcom Yeah, that makes it easier for us. We can write a client that implements the same flow for testing ocid.

from image.

@runcom #259 has added types.Image.OCIConfig. Is there anything else we should do for this issue?

from image.