Git Product home page Git Product logo

biopython-wheels's Introduction

This repository on GitHub is to automate the compilation of binary wheel packages of Biopython for later upload to PyPI.

It now uses the cibuildwheel system running on GitHub Actions.

This produces lots and lots of wheel files, zipped up as a ~100MB artifact.zip available from the footer of the relevant GitHub Actions run.

We currently manually download, unzip, and then upload to PyPI with twine as part of making a release.

This repository originally used the multibuild system developed by Matthew Brett and the MacPython project, running on TravisCI and AppVeyor.

There is a basic pre-commit configuration provided to validate the YAML files as a git pre-commit hook. Install this as follows:

$ pip install pre-commit
$ pre-commit install

biopython-wheels's People

Contributors

chebizarro avatar joaorodrigues avatar matthew-brett avatar odidev avatar peterjc avatar

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

matthew-brett rlhelinski enformatik chebizarro philloidin prakritichauhanpuresoftware prabhat181998 prakriti07 odidev joaorodrigues clintonkildepstein

biopython-wheels's Issues

64-bit ARM wheels failing (ARM64)

See e.g. https://travis-ci.org/github/biopython/biopython-wheels/builds/773144347 from the Biopython 1.79 release,

Forcing en_US.UTF-8 as workaround for encoding issues in Biopython 1.70 tests
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  language-pack-en
The following NEW packages will be installed:
  language-pack-en language-pack-en-base locales sudo
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 4160 kB of archives.
After this operation, 20.9 MB of additional disk space will be used.
Ign:1 http://ports.ubuntu.com/ubuntu-ports xenial-security/main arm64 locales all 2.23-0ubuntu11.2
Get:2 http://ports.ubuntu.com/ubuntu-ports xenial-updates/main arm64 language-pack-en-base all 1:16.04+20160627 [451 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports xenial-updates/main arm64 language-pack-en all 1:16.04+20161009 [145 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports xenial-updates/main arm64 sudo arm64 1.8.16-0ubuntu1.10 [347 kB]
Err:1 http://ports.ubuntu.com/ubuntu-ports xenial-security/main arm64 locales all 2.23-0ubuntu11.2
  404  Not Found [IP: 91.189.91.39 80]
Fetched 943 kB in 0s (2266 kB/s)
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/g/glibc/locales_2.23-0ubuntu11.2_all.deb  404  Not Found [IP: 91.189.91.39 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

i.e. appears to be failing here in config.sh,

        echo "Forcing en_US.UTF-8 as workaround for encoding issues in Biopython 1.70 tests"
        # There is likely a more consise method to do this, but this works:
        if [ "$(uname -m)" == "aarch64" ]; then
            apt-get install -y sudo locales language-pack-en-base
        fi

As noted on #12 there is indeed no locales_2.23-0ubuntu11.2_all.deb but there was locales_2.23-0ubuntu11.3_all.deb.

This looks like a Debian ARM64 package interdependency problem... @odidev are you able to reproduce this locally and/or know immediately where to look for any relevant Debian issues?

Add support to release aarch64 wheels

Problem

On aarch64, ‘pip install biopython’ builds the wheels from source code and then installs it. It requires the user to have a development environment installed on his system. Also, it takes some time to build the wheels than downloading and extracting the wheels from pypi.

Resolution

On aarch64, ‘pip install biopython’ should download the wheels from pypi

@peterjc Please let me know your interest in releasing aarch64 wheels. I can help in this.

ERROR: No matching distribution found for anaconda-project>=0.9.1

I just tried added Python 3.9 to the build target, both AppVeyor and Travis are failing with:

python -m pip install git+https://github.com/Anaconda-Server/anaconda-client.git
Collecting git+https://github.com/Anaconda-Server/anaconda-client.git
  Cloning https://github.com/Anaconda-Server/anaconda-client.git to c:\users\appveyor\appdata\local\temp\1\pip-req-build-ik1c6f0i
  Running command git clone -q https://github.com/Anaconda-Server/anaconda-client.git 'C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-ik1c6f0i'
Collecting clyent>=1.2.0
  Downloading clyent-1.2.1.tar.gz (20 kB)
Collecting nbformat>=4.4.0
  Downloading nbformat-5.1.3-py3-none-any.whl (178 kB)
Collecting python-dateutil>=2.6.1
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
ERROR: Could not find a version that satisfies the requirement anaconda-project>=0.9.1 (from anaconda-client) (from versions: none)
ERROR: No matching distribution found for anaconda-project>=0.9.1

This is the post-build step to push the wheels to https://anaconda.org/multibuild-wheels-staging

Windows wheels not appearing on multibuild-wheels-staging

e.g. Only mac and Linux:

https://anaconda.org/multibuild-wheels-staging/biopython/files?version=1.78

They built, and AppVeyor has the artifacts from where we can manually download them to post on PyPI, but why are they not uploaded to the staging area?

e.g. https://ci.appveyor.com/project/biopython/biopython-wheels/builds/38983166/job/hcrqmb4hurd3r0m2

Ends with:

Collecting artifacts...
Found artifact 'biopython\dist\biopython-1.78-cp39-cp39-win32.whl' matching 'biopython\dist\*' path
Uploading artifacts...
[1/1] biopython\dist\biopython-1.78-cp39-cp39-win32.whl (2,234,622 bytes)...100%
python -m pip install git+https://github.com/Anaconda-Platform/anaconda-project.git git+https://github.com/Anaconda-Platform/anaconda-client.git
...<snip>...
IF NOT "%BIOPYTHON_STAGING_UPLOAD_TOKEN%" == "" anaconda -t %BIOPYTHON_STAGING_UPLOAD_TOKEN% upload --force --no-progress -u "multibuild-wheels-staging" "dist\*.whl"
Using Anaconda API: https://api.anaconda.org
Using "multibuild-wheels-staging" as upload username
Build success

Is the token wrong?

Biopython 1.71 windows 64bit Python 3.4 wheel failing

Logged as https://github.com/matthew-brett/multibuild/issues/157

It took a couple of tweaks to refresh the multibuild configuration for our repository https://github.com/biopython/biopython-wheels/projects but I was able to build all our Mac and Linux wheels using TravisCI, and almost all the Windows wheels.

Our current appveyor.yml file,

https://github.com/biopython/biopython-wheels/blob/77b948054f4207cec563e922781f93054f82745d/appveyor.yml

This single target is failing:

    - PYTHON: "C:\\Miniconda34-x64"
      PYTHON_VERSION: "3.4"
      PYTHON_ARCH: "64"
      BUILD_DEPENDS: "numpy==1.9.0"
      TEST_DEPENDS: "numpy==1.9.0"

https://ci.appveyor.com/project/biopython/biopython-wheels/build/1.0.54/job/86t1cjvxigjqodv4

python setup.py bdist_wheel
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.4
creating build\lib.win-amd64-3.4\Bio
copying Bio\bgzf.py -> build\lib.win-amd64-3.4\Bio
copying Bio\DocSQL.py -> build\lib.win-amd64-3.4\Bio
...
running build_ext
building 'Bio.cpairwise2' extension
Traceback (most recent call last):
  File "setup.py", line 478, in <module>
    install_requires=REQUIRES,
  File "C:\Miniconda34-x64\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Miniconda34-x64\lib\distutils\dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "C:\Miniconda34-x64\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Miniconda34-x64\lib\site-packages\wheel\bdist_wheel.py", line 179, in run
    self.run_command('build')
  File "C:\Miniconda34-x64\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Miniconda34-x64\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Miniconda34-x64\lib\distutils\command\build.py", line 126, in run
    self.run_command(cmd_name)
  File "C:\Miniconda34-x64\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Miniconda34-x64\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 206, in run
    build_ext.run(self)
  File "C:\Miniconda34-x64\lib\site-packages\setuptools\command\build_ext.py", line 50, in run
    _build_ext.run(self)
  File "C:\Miniconda34-x64\lib\distutils\command\build_ext.py", line 339, in run
    self.build_extensions()
  File "C:\Miniconda34-x64\lib\distutils\command\build_ext.py", line 448, in build_extensions
    self.build_extension(ext)
  File "C:\Miniconda34-x64\lib\site-packages\setuptools\command\build_ext.py", line 183, in build_extension
    _build_ext.build_extension(self, ext)
  File "C:\Miniconda34-x64\lib\distutils\command\build_ext.py", line 503, in build_extension
    depends=ext.depends)
  File "C:\Miniconda34-x64\lib\distutils\msvc9compiler.py", line 460, in compile
    self.initialize()
  File "C:\Miniconda34-x64\lib\distutils\msvc9compiler.py", line 371, in initialize
    vc_env = query_vcvarsall(VERSION, plat_spec)
  File "C:\Miniconda34-x64\lib\site-packages\setuptools\msvc9_support.py", line 52, in query_vcvarsall
    return unpatched['query_vcvarsall'](version, *args, **kwargs)
  File "C:\Miniconda34-x64\lib\distutils\msvc9compiler.py", line 287, in query_vcvarsall
    raise ValueError(str(list(result.keys())))

Any thoughts on what might be breaking on this specific platform (64-bit Python 3.4)?

I consider this low priority as most of our Windows users are likely to want a more recent Python.

Build using NumPy 2.0

Quoting https://github.com/numpy/numpy/blob/main/doc/source/dev/depending_on_numpy.rst under build-time dependencies:

Before NumPy 1.25, the NumPy C-API was not backwards compatible. This means that when compiling with a NumPy version earlier than 1.25 you have to compile with the oldest version you wish to support. This can be done by using oldest-supported-numpy. Please see the NumPy 1.24 documentation.

We have been building against the older possible NumPy v1 as per the opening paragraph quoted.

However, according to the numpy mailing lists etc, I understand that once NumPy v2 is out, we should build or wheels using that in order to be compatible with both NumPy 1.x and 2.x. The documentation at the link above will likely be updated soon...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.