Comments (14)
@runcom I'm pretty interested in something like this but curious what kind of UX you had in mind? would the user have to supply a from
[edit: is this simply a dupe of #312?]
from image.
[edit: is this simply a dupe of #312?]
yup 😢
from image.
@runcom I guess my use case is slightly different - I'd like a client to avoid copying blobs from a non-docker-registry source to a docker-registry destination, if it knows they're already there in a different namespace... thoughts? (so far I'm using skopeo for all registry communication and it'd be nice to keep it that way instead of manually hitting the API, but I accept it might not map sensibly to the CLI)
from image.
I'd like a client to avoid copying blobs from a non-docker-registry source to a docker-registry destination, if it knows they're already there in a different namespace
the issue here is what the registry can say about a particular blob (e.g. if it's on the registry basically). I'm not sure the docker registry API exposes this info. Otherwise, as said in the other issue there should be some cache about already pushed layers from a client but that won't, of course, know about all blobs. I'm not sure, @mtrmac do you have other ideas here?
from image.
the issue here is what the registry can say about a particular blob (e.g. if it's on the registry basically). I'm not sure the docker registry API exposes this info.
(We do ask by a HEAD /v2/$repo/blobs/$digest
; a “sufficiently smart” (i.e. probably backed by a stateful database with some locking and indexes, instead of the current IIRC ~stateless implementation) registry could already maintain an index of $digest independent from $repo and tell us that the blob is available; but there is no “mount by $digest without knowing $repo” API.)
Otherwise, as said in the other issue there should be some cache about already pushed layers from a client but that won't, of course, know about all blobs.
Yeah. There’s one awkward aspect about that: containers/image is a library which can be used by any user with any privilege, and it’s not completely obvious where the cache should live and how should concurrency be managed. But, this being a cache, we can probably use ~/.cache
or something like that—in the worst case we would do a full copy and things would work. As for concurrency, I’d be tempted to farm that out to SQLite, except that there don’t seem to be bindings in the standard library 😞
It seems quite possible overall, just a bit of work.
from image.
surfacing this issue, after realizing this morning that an operation like skopeo copy ...
was re-pushing every layer. 🌊
from image.
Tackling this now
from image.
@vbatts you in the ocean?
from image.
@vbatts Was it actually copying them, or “only” spending a few roundtrips to notice that they already exist?
from image.
@vbatts Was it actually copying them, or “only” spending a few roundtrips to notice that they already exist?
Never mind, the existence check does not work cross-repo. Sorry.
from image.
@jonboulle
🎱 Reply hazy try again
🎱 As I see it, yes
from image.
@mtrmac you arrived at the answer, but yes, it does a full push for every layer regardless
from image.
@runcom @vbatts @mtrmac @vrothberg I think we have fixes for this, can we close this issue?
from image.
#536 can take advantage of most (but not quite all) opportunities for mounts automatically.
from image.
Related Issues (20)
- Support for structured logging (using `log/slog`) HOT 5
- proposal: Support append images into docker archive HOT 1
- Make a new release HOT 2
- Docker client code can no longer talk to the latest verson of the docker daemon 25.0.0 HOT 5
- Allow empty OCI configs for artifacts HOT 9
- policy.json overwrite not honouring $XDG_CONFIG_HOME HOT 3
- Podman cannot pull image from local registry HOT 4
- copy.Options.EnsureCompressionVariantsExist doesn’t detect existing variants with zstd:chunked
- support multiple sigstore keys HOT 6
- How can I copy from a tar file stream HOT 7
- "slices" module only in go 1.21 HOT 1
- Cannot pull sigstore signed image with podman HOT 4
- Error inspecting local manifest-lists HOT 6
- platform.WantedPlatforms is noisy on macOS HOT 7
- Incorrect syntax highlighting in containers-transports.5
- Why do we get the whole image when inspect with docker daemon? HOT 2
- Support sigstore BYO PKI verification HOT 1
- Support more arbitrary credential helper executable names? HOT 4
- OCI image index loose the artifactType property on copy HOT 4
- zstd:chunked and layer encryption don’t make sense together HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from image.