yodle / docker-registry-client Goto Github PK
View Code? Open in Web Editor NEWA Python REST client for the Docker Registry
License: Apache License 2.0
A Python REST client for the Docker Registry
License: Apache License 2.0
I have a private docker registry using the docker contrainer registry registry:2
. Let's call it myregistry1
I want to create a second identical registry, myregistry2
but without disturbing the original running one. So no shutting down, cloning drive etc.
I can create a new, empty myregistry2
on a second machine using the same docker container method registry registry:2
Is there a script I can run that uses the REST API or ssh to copy all the docker images and associated metadata from myregistry1
to myregistry2
over the network?
I was thinking of using this library or something similar to loop over all the images of myregistry1
and push them to myregistry2
but don't know how.
File "/usr/local/lib/python2.7/dist-packages/docker_registry_client/_BaseClient.py", line 88, in _http_call
return response.json()
This should probably handle the error and return something sensible.
Amazon ECR supports registry V2 with the V1 schema.
Actually, so far implementation of CAS (Central Authorization Service) integration should work in many cases. There WERE some bugs however.
Authorization
header will be always overwritten by a Basic auth part. The result is that even if proper token were obtained, requests were not authorized at all./v2/token
part is not always correct assumption. Address of token service should be discovered from v2 request.Fixing thise two problems will bring token authentication to work. But it still does not follow 'protocol' described here. Implementation is rather straightforward:
401
response code, look for WWW-Authenticate
header, extracting realm
and scope
parameters.GET
request:realm
,scope
along with parameters,token
from the responseAuthentication: Bearer <...>
header.There also can be some caching performed on tokens to improve performance.
Update version number
Update changelog
Publish
The GET .../repository/manifests/tagname request should be using
Accept: application/vnd.docker.distribution.manifest.v2+json header do get correct manifest and digest
I have the newest docker registry.
When get_manifest() (https://github.com/yodle/docker-registry-client/blob/master/docker_registry_client/_BaseClient.py#L186) uses schema1 the registry sends wrong digest and manifest deletion responds 404 MANIFEST_UNKNOWN.
When request uses schema2, gives good digest and manifest deletion works fine.
This library doesn't seem to work with Docker Hub?
I also noticed the auth service for GitLab is very different from Docker Hub... so perhaps it's hard to support all the different ways auth services work?
The following doesn't seem to work:
docker-registry-show.py -v https://registry.hub.docker.com honestbee/fluentd-kubernetes logentries --username so0k --password **** --authorization-service https://auth.docker.io --api-version 2
Full log of error (after removing the v2
path of the auth.docker.io/v2/token
call:
python docker-registry-show.py -v https://registry.hub.docker.com honestbee/fluentd-kubernetes logentries --username so0k --password **** --authorization-service https://auth.docker.io --api-version 2
DEBUG:docker_registry_client._BaseClient:Getting new token for scope: repository:honestbee/fluentd-kubernetes:*
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): auth.docker.io
DEBUG:requests.packages.urllib3.connectionpool:https://auth.docker.io:443 "GET /token?service=registry.hub.docker.com/v2&scope=repository:honestbee/fluentd-kubernetes:* HTTP/1.1" 200 None
DEBUG:docker_registry_client._BaseClient:GET /v2/honestbee/fluentd-kubernetes/manifests/logentries
DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): registry.hub.docker.com
DEBUG:requests.packages.urllib3.connectionpool:https://registry.hub.docker.com:443 "GET /v2/honestbee/fluentd-kubernetes/manifests/logentries HTTP/1.1" 401 171
DEBUG:docker_registry_client._BaseClient:401 Unauthorized
ERROR:docker_registry_client._BaseClient:Error response: '{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"honestbee/fluentd-kubernetes","Action":"pull"}]}]}\n'
Traceback (most recent call last):
File "docker-registry-show.py", line 138, in <module>
cli.run()
File "docker-registry-show.py", line 80, in run
self.show_manifest(client, args.repository, args.ref)
File "docker-registry-show.py", line 123, in show_manifest
manifest, digest = repo.manifest(ref)
File "/usr/local/lib/python3.6/site-packages/docker_registry_client/Repository.py", line 83, in manifest
return self._client.get_manifest_and_digest(self.name, tag)
File "/usr/local/lib/python3.6/site-packages/docker_registry_client/_BaseClient.py", line 186, in get_manifest_and_digest
m = self.get_manifest(name, reference)
File "/usr/local/lib/python3.6/site-packages/docker_registry_client/_BaseClient.py", line 193, in get_manifest
schema=self.schema_1_signed,
File "/usr/local/lib/python3.6/site-packages/docker_registry_client/_BaseClient.py", line 272, in _http_response
response.raise_for_status()
File "/usr/local/lib/python3.6/site-packages/requests/models.py", line 909, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://registry.hub.docker.com/v2/honestbee/fluentd-kubernetes/manifests/logentries
However, I'm able to get image digest without password using a script roughly like this:
import requests
auth_url = 'https://auth.docker.io/token'
auth_service = 'registry.docker.io'
reg_url = 'https://registry.hub.docker.com'
repo = "honestbee/fluentd-kubernetes"
ref = "logentries"
tag_url = {
'reg_url': reg_url,
'repo': repo,
'ref': ref,
}
querystring = {
'service': auth_service,
'scope': 'repository:%s:pull' % repo,
}
auth_response = requests.request('GET',auth_url, params=querystring)
auth_data = auth_response.json()
reg_headers = {
'accept': "application/vnd.docker.distribution.manifest.v2+json",
'Authorization': 'Bearer %s' % auth_data['access_token']
}
get_manifests_v2 = "{reg_url}/v2/{repo}/manifests/{ref}".format(**tag_url)
reg_response = requests.request('GET', get_manifests_v2 , headers=reg_headers)
print(reg_response.json()['config']['digest'])
The v2 API (https://docs.docker.com/registry/spec/api/) is not supported.
I'm looking for a Python client for the Docker v2 registry, and this seems like the best fit. The only part missing is v2 support. :-)
Would you consider accepting this if I started work on it?
good tools,but i think it need a user guide
It would be nice to have more documentation on setting up, types of calls, and in general.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.