packagemgmt / repositorytools Goto Github PK

View Code? Open in Web Editor NEW

58.0 10.0 18.0 120 KB

Python API and command-line interface for working with artifact repositories like Sonatype Nexus

License: Apache License 2.0

Makefile 8.16% Python 91.31% Shell 0.53%

sonatype-nexus sonatype nexus nexus-repository

repositorytools's People

Contributors

Stargazers

Watchers

Forkers

grokzen gdm tantale nilkonto whyjustin sambrightman laurentribot diacone kcvivek jonafidel alkuzad joerg-wille visitbethel maciejkucia bpd1069 s-oravec gitiles folpindo

repositorytools's Issues

with the REST api uploads, the uploads aren't streamed anymore

tried requests-toolbelt, didn't work. Will try poster.

scripts don't work on windows and venv well - use console_scripts in setup.py

Fixed in PR #17 and #20

[cli] upload: allow user to switch between rest and plain old PUT

upload artifacts as bundle

If you try to load 2 artifacts with the same groupId and artifactId you will get bad request:

<html>
  <body>
     <error>Repository with ID= does not allow updating artifacts.</error>
  </body>
</html>

The lib try to upload them one by one so the second one will try to update the first.
this should be uploaded as one.
as describe in this: How-can-I-programatically-upload-an-artifact-into-Nexus

Upload artifact : manage classifier

When uploading artifact with classifier upload_artifacts (class NexusRepositoryClient) returns exceptions.

Sample code :

import repositorytools

client = repositorytools.repository_client_factory(repository_url='http://myrepo.com/nexus')
local_artifact = repositorytools.LocalArtifact(local_path='my_artifact-1.0-my_classifier.jar',
                                               group='my_group',
                                               artifact='my_artifact',
                                               extension='jar',
                                               version='1.0',
                                               classifier='my_classifier')

client.upload_artifacts(local_artifacts=[local_artifact], repo_id='my_repo')

Results :

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/virtualenv/local/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 127, in upload_artifacts
    use_direct_put=use_direct_put)
  File "/virtualenv/local/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 174, in _upload_artifact
    self.resolve_artifact(result)
  File "/virtualenv/local/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 90, in resolve_artifact
    e=remote_artifact.extension))['data']
  File "/virtualenv/local/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 235, in _send_json
    r = self._send(path, data=data, headers=headers, method=method, params=params)
  File "/virtualenv/local/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 225, in _send
    r.raise_for_status()
  File "/virtualenv/local/lib/python2.7/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://myrepo.com/nexus/service/local/artifact/maven/resolve?a=my_artifact&c=my_classifier&r=my_repo&e=jar&g=my_group&v=1.0

automatically release to pypi

http://docs.travis-ci.com/user/deployment/pypi/

repo close: fail if it doesn't manage to close it

Wrong number of arguments potentially passed to CLI.run method

There is a bug in repositorytools.cli.common module in passing arguments to CLI.run method in CLI.__call__. The run method takes only one argument, which is expected to be a list, but the __call__ method passes in expanded variable list of arguments, which can result in wrong argument number error.

class CLI(collections.Callable):
    def run(self, args=None):
        pass  # some stuff done here

    def __call__(self, *args):
        self.run(*args)    # line 58; should be self.run(args)
        return 0

The above code can result, for example, in TypeError: run() takes at most 2 arguments (3 given) error with the following code:

class MyCLI(CLI):
    def _get_parser(self):
        return argparse.ArgumentParser()

MyCLI()("arg1", "arg2")

regression: all commands return exit code 1

caused by #17

How to reproduce (unix):
echo foo > foo
artifact upload foo test com.example
echo $?

expected result:
upload succeeds
return code 0

current result:
upload succeeds
return code 1

staging: allow specifying staging profile name

maybe only docs need to be fixed?

recommand by sonatype official site

congratulation!

the link is:
https://books.sonatype.com/nexus-book/reference/_nexus_integrations.html

the other related repo build in ruby:
https://github.com/RiotGamesMinions/nexus_cli

which is used by nexus cookbook:
https://github.com/RiotGamesCookbooks/nexus-cookbook

I suppose there are some works need to be done, especially for paper work( more detail document)

Best regards
Oliver

cli: Add options --user and --server-url

So people don't need to set variables and can see that param in help

artifact upload fails when uploaded file's name does not match GAV specified in arguments

Call to artifact upload fails when name of the file being uploaded does not match version specified in the command's arguments.

When running this command

artifact upload --use-direct-put --artifact my-artifact --version 2.0.8-1.el6.noarch  /tmp/my-artifact-2.0.8-1.noarch.rpm packages-el6 com.mycomp

the artifact is uploaded, but the command fails with this error:

INFO:/usr/local/bin/artifact:Started /usr/local/bin/artifact, with arguments ['upload', '--use-direct-put', '--artifact', 'my-artifact', '--version', '2.0.8-1.el6.noarch', '/tmp/my-artifact-2.0.8-1.noarch.rpm', 'packages-el6', 'com.mycomp']
INFO:repositorytools.lib.repository:-> Uploading my-artifact-2.0.8-1.noarch.rpm
Traceback (most recent call last):
  File "/usr/local/bin/artifact", line 9, in <module>
    load_entry_point('repositorytools==4.2.1', 'console_scripts', 'artifact')()
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/cli/common.py", line 58, in __call__
    self.run(*args)
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/cli/common.py", line 55, in run
    return args_namespace.func(args_namespace)
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/cli/commands/artifact.py", line 94, in upload
    return self.repository.upload_artifacts([artifact], args.repo_id, use_direct_put=args.use_direct_put)
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 109, in upload_artifacts
    use_direct_put=use_direct_put)
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 170, in _upload_artifact
    maven_metadata = self._send_json(path)['data']
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 217, in _send_json
    r = self._send(path, data=data, headers=headers, method=method, params=params)
  File "/opt/repositorytools/lib/python2.7/site-packages/repositorytools/lib/repository.py", line 207, in _send
    r.raise_for_status()
  File "/opt/repositorytools/lib/python2.7/site-packages/requests/models.py", line 935, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://repository.mycomp.com/service/local/repositories/packages-el6/content/com/mycomp/my-artifact/2.0.8-1.el6.noarch/my-artifact-2.0.8-1.noarch.rpm?describe=maven2

The repository returns HTTP 404, because the URL contains local file name (without el6) and not the file name used on the server. The correct URL should be https://repository.mycomp.com/service/local/repositories/packages-el6/content/com/mycomp/my-artifact/2.0.8-1.el6.noarch/my-artifact-2.0.8-1.el6.noarch.rpm?describe=maven2.

lib, cli: upload_artifacts should have an option to use REST API instead of direct POST

Without this, maven metadata don't work and resolving to latest artifact doesn't work.

generate the same traffic as
curl -v -F r=releases -F hasPom=false -F e=jar -F g=com.test -F a=project -F v=1.0 -F p=jar -F [email protected] -u admin:*** http://localhost:8081/nexus/service/local/artifact/maven/content
does

Add description of env variables to help

AttributeError: 'Namespace' object has no attribute 'func'

Bug reproduction:

artifact --help

artifact upload: add option --classifier

Publish on pypi

Hi,

As I will need to implement an easy to use library for downloading and uploading artifacts it seems that your approach does match very well what I wall looking for.

I would like to join your effort and get this library published to PyPy and to production quality ASAP.

Version regexp match fails on artifacts with numbers

It's rare but can happen, for example:

https://mvnrepository.com/artifact/org.neo4j/neo4j-cypher-compiler-2.1/2.1.2

Has jar named neo4j-cypher-compiler-2.1-2.1.2.jar.

I wanted to made PR for that but I couldn't find any naming convention that could specify what is correct jar naming. So I made subclass LocalArtifactWithPomDetection which loads this data from pom file, if exists (almost every time, there is one)

document basic operations: upload, resolve, and delete of artifacts

It seems that at this moment the documentation is empty and I do think that it would include at least this:

how to install, PyPi seems like the best method
how to download an artefact
how to upload an aterfact

Run system tests on CI (travis)

some tests (for example resolve) can be run against oss.sonatype.org

http://repositorytools.readthedocs.org/ is empty

custom metadata: new nexus (>=2.12) returns 403 if extension is missing in coordinates

print warning in that case or raise an exception

add download_artifact method to NexusRepositoryClient

by default download to current working directory, or to directory specified by optional arg

upload: failing when using REST and specifying version explicitly

What kind of repositories are supported?

This lib seems to work with Maven repos only. Would be nice to have this explicitly stated in the docs.

try to download artifact.

Hello,

I upload a foo-bar-1.0.jar file to my local machine repo, and write these script:

import repositorytools
from repositorytools.lib.artifact import Artifact, RemoteArtifact

url = 'http://localhost:8081/nexus/content/repositories/releases'

client = repositorytools.NexusRepositoryClient(repository_url=url, user='admin',password='admin123')

artifact = RemoteArtifact.from_repo_id_and_coordinates('Release', 'foo:bar:1.0:demo:jar')
print(artifact)
client.resolve_artifact(artifact)
print(client)

got the result of this statement:

print(artifact)

foo:bar:1.0:demo:jar

it seems not right, and the later statement

client.resolve_artifact(artifact)

report _send_json function failed.

it seems that the file in lib/artifact.py RemoteArtifact.from_repo_id_and_coordinates() can not resolve a json format . I change my code:

artifact = RemoteArtifact.from_repo_id_and_coordinates('Release', 'foo:bar:1.0')

but get the result:

foo:bar:1.0::

and still failed,

I'm not sure the root cause, i'm not quite familiar with requests module, Could you post some demo code about upload, download and delete?

$ wget https://pypi.python.org/packages/c8/22/9460e311f340cb62d26a38c419b1381b8593b0bb6b5d1f056938b086d362/lockfile-0.12.2-py2.py3-none-any.whl

$ artifact upload lockfile-0.12.2-py2.py3-none-any.whl pip com.vsct.pip --artifact lockfile --version 0.12.2
INFO:/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/bin/artifact:Started /home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/bin/artifact, with arguments ['upload', 'lockfile-0.12.2-py2.py3-none-any.whl', 'pip', 'com.vsct.pip', '--artifact', 'lockfile', '--version', '0.12.2']
INFO:repositorytools.lib.repository:-> Uploading lockfile-0.12.2-py2.py3-none-any.whl
Traceback (most recent call last):
  File "/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/bin/artifact", line 11, in <module>
    sys.exit(artifact_cli())
  File "/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/lib/python3.4/site-packages/repositorytools/cli/common.py", line 58, in __call__
    self.run(*args)
  File "/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/lib/python3.4/site-packages/repositorytools/cli/common.py", line 55, in run
    return args_namespace.func(args_namespace)
  File "/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/lib/python3.4/site-packages/repositorytools/cli/commands/artifact.py", line 94, in upload
    return self.repository.upload_artifacts([artifact], args.repo_id, use_direct_put=args.use_direct_put)
  File "/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/lib/python3.4/site-packages/repositorytools/lib/repository.py", line 109, in upload_artifacts
    use_direct_put=use_direct_put)
  File "/home/lucas_cimon/.local/share/virtualenvs/nexus_uploader/lib/python3.4/site-packages/repositorytools/lib/repository.py", line 143, in _upload_artifact
    data_list.append( ('file', (filename, f, 'text/plain') ))
AttributeError: 'dict_items' object has no attribute 'append'