blue-yonder / devpi-builder Goto Github PK

View Code? Open in Web Editor NEW

76.0 17.0 30.0 303 KB

Automatically build and upload python packages to devpi

License: Other

Python 100.00%

python devpi packaging wheels

devpi-builder's People

Contributors

Stargazers

Watchers

devpi-builder's Issues

Devpi builder incompatible with pip 8.0

Pip 8.0 removed the deprecated option --download-cache. However, release 1.0.0 of Brandon still uses it and thus builds fail.

Read requirements from setup.py

Hello! Awesome project!

I'd love it if I could pass a setup.py file to devpi-builder. I have a project where I specify the dependencies in the install_requires attribute of the setuptools dict. It should be possible to get the output via python setup.py --requires, but in a quick test that didn't work for me.

I'll try to submit a PR for this.

Improve error output

I tried using this tool to dump a virtualenv's content and upload it to our companies devpi. It started building one package and then quit, no errors or any debug output. This is probably due to me being on Windows which cannot build some of the wheels because it is missing a compiler.

It would be great to at least get some sort of error output, similar to what we can see when dumping wheels manually using pip wheel -r requirements.txt --wheel-dir=.wheelhouse. This will tell me which wheel failed, so I can skip it for the moment.

Can that information be caught and piped to the console?

Add support for client certs

We use a devpi server which requires a client certificate to connect. I'd like to add support for using client certs to devpi-builder. I submitted pull requests in both devpi-builder and devpi-plumber to add support for this a while back. Please note the failing build on the patch here is just due to lack of support in plumber. Let me know what I can do to help get those patches included.

Pull request is #69

Improve test performance

As a developer, I would like the unit/integration tests to run faster.

Our current test infrastructure has several issues:

Many tests use a running devpi server. However, some of them might be formulated using mocks / fake servers.
By design, we start one devpi server per integration test. Devpi performs a lengthy synchronization with pypi.python.org on startup. If we could patch devpi to run this in an asynchronous fashion, our tests will be significantly faster
All calls to the devpi client use a costly subprocess call. This problem will be addressed by the (currently unfinished) devpi-plumber

release to pypi?

Would like to do "pip install devpi-builder" to try it out :)

Exception when version is not specified in requirements.txt

It would be nice if devpi-builder could install the latest version of requred package, when no version is specified for it ...

Traceback (most recent call last):
  File "/usr/local/bin/devpi-builder", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/devpi_builder/cli.py", line 105, in main
    packages = requirements.read(args.requirements)
  File "/usr/local/lib/python2.7/dist-packages/devpi_builder/requirements.py", line 41, in read
    _extract_project_version(requirement) for requirement in pkg_resources.parse_requirements(input_file)
  File "/usr/local/lib/python2.7/dist-packages/devpi_builder/requirements.py", line 27, in _extract_project_version
    raise ValueError('Version specification is missing for "{}".'.format(requirement))
ValueError: Version specification is missing for "websocket-client".

Include stderr in junit.xml output

If the build fails, message written from pip to stderr, like the trusted-host warning, are not included in the junit.xml. Thus an extra step is required to look for the reason of the build failure.

wheel >=0.32.0 breaks devpi-builder

See this issue in wheel:
pypa/wheel#255

HTTP Auth does not work

We protect our devpi server with HTTP Auth. When trying to use devpi-builder to build and upload packages run into the following issue:

$ devpi-builder dev-requirements.txt 'https://foo:[email protected]/user/prod/+simple/' 'user' pass 
Traceback (most recent call last):
  File "/home/aaditya/.virtualenvs/c/bin/devpi-builder", line 11, in <module>
    sys.exit(main())
  File "/home/aaditya/.virtualenvs/c/local/lib/python2.7/site-packages/devpi_builder/cli.py", line 106, in main
    with wheeler.Builder() as builder, DevpiClient(args.index, args.user, args.password) as devpi_client:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/aaditya/.virtualenvs/c/local/lib/python2.7/site-packages/devpi_plumber/client.py", line 33, in DevpiClient
    wrapper.login(user, password)
  File "/home/aaditya/.virtualenvs/c/local/lib/python2.7/site-packages/devpi_plumber/client.py", line 71, in login
    return self._execute('login', user, '--password', password)
  File "/home/aaditya/.virtualenvs/c/local/lib/python2.7/site-packages/devpi_plumber/client.py", line 62, in _execute
    raise DevpiClientError(output.getvalue())
devpi_plumber.client.DevpiClientError: WARN: devpi-client-2.5.0 got an unversioned reply, assuming API-VERSION 2 (as implemented by devpi-server-2.0)
POST http://pypi.example.com/+login
401 Unauthorized: <html>
<head><title>401 Authorization Required</title></head>
<body bgcolor="white">
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.2.1</center>
</body>
</html>

The other issue about using devpi login would solve this I believe.

Use "devpi login" and "devpi use" data instead of command line args

Whenever I'm using devpi-builder, I've probably already used "devpi login" and "devpi use" to authenticate and set my target index. Could devpi-builder (perhaps optionally) use that information in place of the index, user, and password command line arguments?

Remove deprecated download cache

The used pip download cache option is deprecated. We can remove it once we want to rely on pip>= 6.0

https://pip.pypa.io/en/latest/news.html

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Drop pytest runner and test_requires as per pyup security recommendation

pkg_resources.RequirementParseError with vcs requirement

With a requirements.txt with a vcs dependency like:

git+https://github.com/kennethreitz/requests@master#egg=requests==2.10.0

I get this traceback:

Traceback (most recent call last):
  File "/Users/simonweber/platform/venv-builder/bin/devpi-builder", line 11, in <module>
    sys.exit(main())
  File "/Users/simonweber/platform/venv-builder/lib/python2.7/site-packages/devpi_builder/cli.py", line 106, in main
    packages = requirements.read(args.requirements)
  File "/Users/simonweber/platform/venv-builder/lib/python2.7/site-packages/devpi_builder/requirements.py", line 41, in read
    _extract_project_version(requirement) for requirement in pkg_resources.parse_requirements(input_file)
  File "/Users/simonweber/platform/venv-builder/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2920, in parse_requirements
    "version spec")
  File "/Users/simonweber/platform/venv-builder/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2885, in scan_list
    raise RequirementParseError(msg, line, "at", line[p:])
pkg_resources.RequirementParseError: Expected version spec in git+https://github.com/kennethreitz/requests@master#egg=requests==2.10.0 at +https://github.com/kennethreitz/requests@master#egg=requests==2.10.0

But, pip can accept this format without trouble:

$ pip install -r requirements.txt
Collecting requests==2.10.0 from git+https://github.com/kennethreitz/requests@master#egg=requests==2.10.0 (from -r requirements.txt (line 1))
  Cloning https://github.com/kennethreitz/requests (to master) to /private/var/folders/4x/dfcw90sj1wg4q3q_r01tl81m0000gp/T/pip-build-aXfoLP/requests
Installing collected packages: requests
  Running setup.py install for requests
Successfully installed requests-2.10.0

Here's the output of pip freeze. It's from a fresh install of devpi-builder and devpi, so I don't expect it to be the culprit:

$ pip freeze
You are using pip version 7.0.1, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
apipkg==1.4
beautifulsoup4==4.4.1
bleach==1.4.3
Chameleon==2.24
check-manifest==0.31
defusedxml==0.4.1
devpi==2.2.0
devpi-builder==2.2.0
devpi-client==2.6.3
devpi-common==3.0.0
devpi-plumber==0.2.15
devpi-server==4.0.0
devpi-web==3.1.0
docutils==0.12
execnet==1.4.1
html5lib==0.9999999
itsdangerous==0.24
junit-xml==1.6
PasteDeploy==1.5.2
pkginfo==1.3.2
pluggy==0.3.1
py==1.4.31
Pygments==2.1.3
pyramid==1.7
pyramid-chameleon==0.3
readme-renderer==0.7.0
repoze.lru==0.6
requests==2.10.0
six==1.10.0
tox==2.3.1
translationstring==1.3
twitter.common.contextutil==0.3.5
twitter.common.dirutil==0.3.5
twitter.common.lang==0.3.5
venusian==1.0
virtualenv==15.0.2
waitress==0.9.0
WebOb==1.6.1
wheel==0.24.0
Whoosh==2.7.4
zope.deprecation==4.1.2
zope.interface==4.1.3

For this example, I can get around it by installing, freezing, and then using the resulting requirements.txt, but I don't expect that to work if the package isn't available on pypi.

Report build results as junit.xml

The devpi builder should optionally report on successful, skipped and failed build in a junit compatible XML file. This would allow to use standard Jenkins mechanisms to monitor the state of the automatic build.

manylinux wheels

Hi,

have you ever looked into building manylinux wheels?

blue-yonder / devpi-builder Goto Github PK

devpi-builder's People

Contributors

Stargazers

Watchers

Forkers

devpi-builder's Issues

Recommend Projects

Recommend Topics

Recommend Org