pluggable transports? about image HOT 11 CLOSED

transports.ImageName would be trivial to separate, but the transport.KnownTransports[key] reference in signature/policy_config.go is not so easy (we could get rid of it in principle, at the cost of noticeably worse diagnostics; not an easy decision). And the Docker code uses the signature package…

We could perhaps split it to have KnownTransports in one subpackage, always available, but empty by default (this would be used by Docker, which would explicitly register the single needed transport into KnownTransports), and a separate subpackage which would register all transports (perhaps a package with the current transports.ParseImageName would contain the list of all transports, assuming that any code generic enough to use transports.ParseImageName wants all transports).

That would still require policy_config.go changes to silently allow configuration which contains unrecognized transports (but to refuse to accept any image from unrecognized transports, I suppose). That’s still a bit risky and pretty ugly, but doable in principle.

from image.

Something like transports.Register should do the job I guess.

from image.

And each transport registers itself when imported? This seems cleaner.

from image.

That’s elegant in a way, (and non-obvious in a way), but there should still be a single thing to import for decoupled consumers like skopeo.

When we add a new transport to containers/image, not a single line should need to be updated in skopeo (beyond revendoring).

That can of course be done by having each transport register itself, and then having a subpackage which just includes all of them without doing anything else with them.

from image.

There's no need to have another package doing that (ofc you need to change skopeo to add a new transport and I'm fine with that). It's go idiomatic to have a package imported with _ (the underscore) to just call its init function (to register the transport in this case), another package to register everything isn't good to me instead since we need to leave to users the option and adding a single line in skopeo when a new transport is added isn't a big deal to me.

from image.

It's go idiomatic to have a package imported with _ (the underscore) to just call its init function

ACK.

ofc you need to change skopeo to add a new transport and I'm fine with that

I’d rather not. It is fine to give callers flexibility, but there should be a default and easy way to consume everything without having to keep up and risking divergence. We don’t want to end up with 3 different CLIs which all accept docker://busybox but only two accept dir: and only one accepts oci:. … I guess it makes sense to me to tie this to transports.ParseImageName; callers of an individual somepackage.NewReference can do whatever they want, but anything which exposes the name syntax should support everything.

from image.

(Of course anyone can still explicitly opt out, or, well, import everything and then edit KnownTransports to drop something, or just do string filtering on input, or… But the easy way to use this package should consume everything I think.)

from image.

I’d rather not. It is fine to give callers flexibility, but there should be a default and easy way to consume everything without having to keep up and risking divergence. We don’t want to end up with 3 different CLIs which all accept docker://busybox but only two accept dir: and only one accepts oci:. …

Isn't this the whole point of this being a library and that we have subpackages? Docker being the example here were it only needs the docker transport.
Maybe we need to refactor the library to be more flexible in this scenario?

from image.

I was thinking something like this

• containers/image/registeredtransports contains KnownTransports, which starts empty
• containers/image/docker usable independently, registers itself into registeredtransports
• containers/image/openshift usable independently, registers itself into registeredtransports
• containers/image/image usable with any ImageSource
• containers/image/signature uses KnownTransports
• containers/image/copy gets ImageReference instances, uses signature and its KnownTransports
• containers/image/transports imports all transports to register them, provides ParseImageName

Then,

• Callers have perfect flexibility to import a single transport and use it to read/store blobs or whatever, starting with sometransport.{New,Parse}Reference.
• Callers can import containers/image/image + a single transport and read information about the image.
• Callers can import copy + any two transports (only) and copy stuff, again starting with sometransport.{New,Parse}Reference.
• A caller like skopeo, which uses ParseImageName, will always support all transfers.

(I didn’t actually check that they packages are that independent right now. I think they are, or can be.)

Does this make sense? It seems quite flexible to me, except for the ParseImageName bit which is all or nothing.

from image.

marking this as enhancement - I plan to work on this after projectatomic/docker#200 since this is just nice to have when I'll open that same PR upstream in docker/docker

from image.

@mtrmac will look further at what you suggested tomorrow

from image.