Git Product home page Git Product logo

conda-forge-ci-setup-feedstock's Introduction

About conda-forge-ci-setup-feedstock

Feedstock license: BSD-3-Clause

Home: https://github.com/conda-forge/conda-forge-ci-setup-feedstock

Package license: BSD-3-Clause

Summary: A package installed by conda-forge each time a build is run on CI. This package has side-effects to your conda config.

Current build status

Azure
VariantStatus
linux_64_c_compiler_version11c_stdlib_version2.17cuda_compilernvcccuda_compiler_version11.8python3.10.____cpython variant
linux_64_c_compiler_version11c_stdlib_version2.17cuda_compilernvcccuda_compiler_version11.8python3.11.____cpython variant
linux_64_c_compiler_version11c_stdlib_version2.17cuda_compilernvcccuda_compiler_version11.8python3.12.____cpython variant
linux_64_c_compiler_version11c_stdlib_version2.17cuda_compilernvcccuda_compiler_version11.8python3.8.____cpython variant
linux_64_c_compiler_version11c_stdlib_version2.17cuda_compilernvcccuda_compiler_version11.8python3.9.____cpython variant
linux_64_c_compiler_version12c_stdlib_version2.12cuda_compilerNonecuda_compiler_versionNonepython3.10.____cpython variant
linux_64_c_compiler_version12c_stdlib_version2.12cuda_compilerNonecuda_compiler_versionNonepython3.11.____cpython variant
linux_64_c_compiler_version12c_stdlib_version2.12cuda_compilerNonecuda_compiler_versionNonepython3.12.____cpython variant
linux_64_c_compiler_version12c_stdlib_version2.12cuda_compilerNonecuda_compiler_versionNonepython3.8.____cpython variant
linux_64_c_compiler_version12c_stdlib_version2.12cuda_compilerNonecuda_compiler_versionNonepython3.9.____cpython variant
linux_aarch64_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.10.____cpython variant
linux_aarch64_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.11.____cpython variant
linux_aarch64_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.12.____cpython variant
linux_aarch64_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.8.____cpython variant
linux_aarch64_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.9.____cpython variant
linux_aarch64_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.10.____cpython variant
linux_aarch64_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.11.____cpython variant
linux_aarch64_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.12.____cpython variant
linux_aarch64_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.8.____cpython variant
linux_aarch64_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.9.____cpython variant
linux_ppc64le_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.10.____cpython variant
linux_ppc64le_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.11.____cpython variant
linux_ppc64le_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.12.____cpython variant
linux_ppc64le_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.8.____cpython variant
linux_ppc64le_c_compiler_version11cuda_compilernvcccuda_compiler_version11.8python3.9.____cpython variant
linux_ppc64le_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.10.____cpython variant
linux_ppc64le_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.11.____cpython variant
linux_ppc64le_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.12.____cpython variant
linux_ppc64le_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.8.____cpython variant
linux_ppc64le_c_compiler_version12cuda_compilerNonecuda_compiler_versionNonepython3.9.____cpython variant
osx_64_python3.10.____cpython variant
osx_64_python3.11.____cpython variant
osx_64_python3.12.____cpython variant
osx_64_python3.8.____cpython variant
osx_64_python3.9.____cpython variant
osx_arm64_python3.10.____cpython variant
osx_arm64_python3.11.____cpython variant
osx_arm64_python3.12.____cpython variant
osx_arm64_python3.8.____cpython variant
osx_arm64_python3.9.____cpython variant
win_64_cuda_compilerNonecuda_compiler_versionNonepython3.10.____cpython variant
win_64_cuda_compilerNonecuda_compiler_versionNonepython3.11.____cpython variant
win_64_cuda_compilerNonecuda_compiler_versionNonepython3.12.____cpython variant
win_64_cuda_compilerNonecuda_compiler_versionNonepython3.8.____cpython variant
win_64_cuda_compilerNonecuda_compiler_versionNonepython3.9.____cpython variant
win_64_cuda_compilernvcccuda_compiler_version11.8python3.10.____cpython variant
win_64_cuda_compilernvcccuda_compiler_version11.8python3.11.____cpython variant
win_64_cuda_compilernvcccuda_compiler_version11.8python3.12.____cpython variant
win_64_cuda_compilernvcccuda_compiler_version11.8python3.8.____cpython variant
win_64_cuda_compilernvcccuda_compiler_version11.8python3.9.____cpython variant

Current release info

Name Downloads Version Platforms
Conda Recipe Conda Downloads Conda Version Conda Platforms

Installing conda-forge-ci-setup

Installing conda-forge-ci-setup from the conda-forge channel can be achieved by adding conda-forge to your channels with:

conda config --add channels conda-forge
conda config --set channel_priority strict

Once the conda-forge channel has been enabled, conda-forge-ci-setup can be installed with conda:

conda install conda-forge-ci-setup

or with mamba:

mamba install conda-forge-ci-setup

It is possible to list all of the versions of conda-forge-ci-setup available on your platform with conda:

conda search conda-forge-ci-setup --channel conda-forge

or with mamba:

mamba search conda-forge-ci-setup --channel conda-forge

Alternatively, mamba repoquery may provide more information:

# Search all versions available on your platform:
mamba repoquery search conda-forge-ci-setup --channel conda-forge

# List packages depending on `conda-forge-ci-setup`:
mamba repoquery whoneeds conda-forge-ci-setup --channel conda-forge

# List dependencies of `conda-forge-ci-setup`:
mamba repoquery depends conda-forge-ci-setup --channel conda-forge

About conda-forge

Powered by NumFOCUS

conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.

A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by Azure, GitHub, CircleCI, AppVeyor, Drone, and TravisCI it is possible to build and upload installable packages to the conda-forge anaconda.org channel for Linux, Windows and OSX respectively.

To manage the continuous integration and simplify feedstock maintenance conda-smithy has been developed. Using the conda-forge.yml within this repository, it is possible to re-render all of this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender.

For more information please check the conda-forge documentation.

Terminology

feedstock - the conda recipe (raw material), supporting scripts and CI configuration.

conda-smithy - the tool which helps orchestrate the feedstock. Its primary use is in the construction of the CI .yml files and simplify the management of many feedstocks.

conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)

Updating conda-forge-ci-setup-feedstock

If you would like to improve the conda-forge-ci-setup recipe or build a new package version, please fork this repository and submit a PR. Upon submission, your changes will be run on the appropriate platforms to give the reviewer an opportunity to confirm that the changes result in a successful build. Once merged, the recipe will be re-built and uploaded automatically to the conda-forge channel, whereupon the built conda packages will be available for everybody to install and use from the conda-forge channel. Note that all branches in the conda-forge/conda-forge-ci-setup-feedstock are immediately built and any created packages are uploaded, so PRs should be based on branches in forks and branches in the main repository should only be used to build distinct package versions.

In order to produce a uniquely identifiable distribution:

  • If the version of a package is not being increased, please add or increase the build/number.
  • If the version of a package is being increased, please remember to return the build/number back to 0.

Feedstock Maintainers

conda-forge-ci-setup-feedstock's People

Contributors

0xbe7a avatar 183amir avatar bdice avatar beckermr avatar bollwyvl avatar carterbox avatar chrisburr avatar cj-wright avatar conda-forge-admin avatar conda-forge-curator[bot] avatar github-actions[bot] avatar h-vetinari avatar hmaarrfk avatar isuruf avatar jaimergp avatar jakirkham avatar leofang avatar mariusvniekerk avatar mbargull avatar minrk avatar nichmor avatar ocefpaf avatar psavery avatar regro-cf-autotick-bot avatar ryanvolz avatar scopatz avatar sodre avatar timsnyder avatar wolfv avatar xhochy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

conda-forge-ci-setup-feedstock's Issues

Sporadic CI error: `conda-forge-ci-setup 3** does not exist` ?

Saw the following error on CI (also the attached log):

+ mamba install --update-specs --yes --quiet --channel conda-forge conda-build pip boa conda-forge-ci-setup=3
Could not solve for environment specs
The following package could not be installed
└─ conda-forge-ci-setup 3**  does not exist (perhaps a typo or a missing channel).

Note: Other jobs in the same build did not see this issue

filter outputs for upload/validation by name

It appears that when conda build runs downstream tests it leaves the downstream packages in the build artifacts directory. This causes our code to attempt to validate and upload them, which leads to jobs that fail when they should otherwise pass. We can compute the names of the outputs from the meta.yaml (but not their versions) and filter things we find by globing.

Cc @isuruf this is why your binutils pr appears to be failing.

Issue:


Environment (conda list):
$ conda list


Details about conda and system ( conda info ):
$ conda info

Cross-compilation tests do not work if libGL is required

Issue:

I have a python package that depends on libGL . I tried to switch it to do cross-compilation when building for aarch64 to work around conda-forge/status#122 (see conda-forge/idyntree-feedstock#8 (comment)), but doing so the test phase fails with:

2021-11-04T08:19:53.3899797Z Preparing transaction: ...working... done
2021-11-04T08:19:55.5224087Z Verifying transaction: ...working... done
2021-11-04T08:20:16.0447763Z Executing transaction: ...working... done
2021-11-04T08:20:16.1038551Z export PREFIX=/home/conda/feedstock_root/build_artifacts/idyntree_1636013219376/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho
2021-11-04T08:20:16.1039979Z export SRC_DIR=/home/conda/feedstock_root/build_artifacts/idyntree_1636013219376/test_tmp
2021-11-04T08:20:16.8049598Z import: 'idyntree'
2021-11-04T08:20:16.8500252Z Traceback (most recent call last):
2021-11-04T08:20:16.8507039Z   File "/home/conda/feedstock_root/build_artifacts/idyntree_1636013219376/test_tmp/run_test.py", line 2, in <module>
2021-11-04T08:20:16.8536213Z     import idyntree
2021-11-04T08:20:16.8537877Z   File "/home/conda/feedstock_root/build_artifacts/idyntree_1636013219376/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.7/site-packages/idyntree/__init__.py", line 1, in <module>
2021-11-04T08:20:16.8546323Z     from . import swig
2021-11-04T08:20:16.8547947Z   File "/home/conda/feedstock_root/build_artifacts/idyntree_1636013219376/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.7/site-packages/idyntree/swig.py", line 13, in <module>
2021-11-04T08:20:16.8554296Z     from . import _iDynTree
2021-11-04T08:20:16.8570597Z ImportError: libGL.so.1: cannot open shared object file: No such file or directory
2021-11-04T08:20:18.1950688Z Tests failed for idyntree-4.2.0-py37hdf036bd_1.tar.bz2 - moving package to /home/conda/feedstock_root/build_artifacts/broken
2021-11-04T08:20:18.1952119Z WARNING:conda_build.build:Tests failed for idyntree-4.2.0-py37hdf036bd_1.tar.bz2 - moving package to /home/conda/feedstock_root/build_artifacts/broken
2021-11-04T08:20:18.1953266Z WARNING conda_build.build:tests_failed(2955): Tests failed for idyntree-4.2.0-py37hdf036bd_1.tar.bz2 - moving package to /home/conda/feedstock_root/build_artifacts/broken
2021-11-04T08:20:18.2433557Z TESTS FAILED: idyntree-4.2.0-py37hdf036bd_1.tar.bz2
2021-11-04T08:20:22.4861613Z ##[error]Bash exited with code '1'.
2021-11-04T08:20:22.5070325Z ##[section]Finishing: Run docker build

See https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=402967&view=logs&jobId=d1321064-f6c3-56b7-0172-6d994d01c836&j=9a864fd9-6c8f-52ca-79ce-2aa6dca1a1de&t=10fc5aa2-324e-5982-4c88-6b31fcab16b3 . Note that the yum_requirements.txt file have all the necessary packages (see https://github.com/diegoferigo/idyntree-feedstock/blob/6738c2ccf293b2f3232f19f54eea7fa54d26a3de/recipe/yum_requirements.txt), but I do not know if that file is considered in this case.

I do not know if this is intended and if this is even the right repo to report this, but I tought reporting this somewhere was better than nothing.


Environment (conda list):

I am not sure which environment you are interested in this case?

$ conda list


Details about conda and system ( conda info ):
I am not sure which conda info you are interested in this case? ``` $ conda info
</details>

Disable uploading to conda-forge/label/main if sources include other labels

This section

upload_to_conda_forge = any(owner == "conda-forge" for owner, _ in channels)
if upload_to_conda_forge and "channel_sources" in specific_config:
unknown_channel = False
allowed_channels = ["conda-forge", "conda-forge/label/", "defaults", "c4aarch64", "c4armv7l"]
for source_channel in source_channels.split(","):
for c in allowed_channels:
if source_channel.startswith(c):
break
else:
print("Uploading to conda-forge with source channel '{}' is not allowed".format(source_channel))
return
needs to be improved.

reduce build matrix since the addition of CUDA testing

The inclusion of CUDA in the build matrix to make sure things are tested has caused some weird solver features where packages at the same build number and version are swapped back and forth.

This is annoying and there are a couple of solutions.

  1. Prioritize the cuda None builds via build number
  2. Test each CUDA version in a different python version (suggestion by @hmaarrfk)

We should do one of these likely.

osx SDKs confusion

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

We need to update the MacOS SDKs download procedure. Copying @isuruf and @matthiasdiener.

ref conda-forge/pytorch-cpu-feedstock#159

https://github.com/phracker/MacOSX-SDKs is out of date and default setting for OSX dir path is not sufficient: OSX_SDK_DIR="$(xcode-select -p)/Platforms/MacOSX.platform/Developer/SDKs" goes to latest Xcode which may not have all the SDKs preinstalled by the default osx images, e.g., see https://github.com/actions/runner-images/blob/main/images/macos/macos-12-Readme.md

Installed packages

n/a

Environment info

n/a

Attempting an upload of the same package again errors out

It appears that uploading packages again (if there version and build number are the same) errors. This marks the CI build as a failure, which is erroneous. Would be better if it simply skipped the upload in these cases and passed.

+ upload_or_check_non_existence /home/conda/recipe_root conda-forge --channel=main -m /home/conda/feedstock_root/.ci_support/linux_python3.5.yaml
Adding in variants from internal_defaults
Adding in variants from /home/conda/feedstock_root/.ci_support/linux_python3.5.yaml
Attempting to finalize metadata for numba
Solving environment: ...working... done
Solving environment: ...working... done
Processing numba
Traceback (most recent call last):
  File "/opt/conda/bin/upload_or_check_non_existence", line 129, in <module>
    main()
  File "/opt/conda/bin/upload_or_check_non_existence", line 119, in main
    upload(cli, fname, owner, channel)
  File "/opt/conda/bin/upload_or_check_non_existence", line 66, in upload
    env=os.environ)
  File "/opt/conda/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['anaconda', '--quiet', '-t', '/tmp/tmpws6dl6tj/binstar.token', 'upload', '/home/conda/feedstock_root/build_artifacts/linux-64/numba-0.36.1-py35hdc42b47_0.tar.bz2', '--user=conda-forge', '--channel=main']' returned non-zero exit status 1.

ref: https://circleci.com/api/v1.1/project/github/conda-forge/numba-feedstock/24/output/104/0?file=true

conda.config is not a thing

We have been using conda.config.subdir for some time to detect the platform and thus determine where things should be uploaded. This no longer exists in conda 4.4+. How would we like to address this?

cc @conda-forge/core

using git to download source fails on windows with conda-forge-ci-setup > 3.0.3

We run into this issue with the freecad-feedstock. Git is used to download the sources to retrive some version information. The build fails on windows with:

error: remote-curl: usage: git remote-curl <remote> [<url>]
Warning: failed to download source.  If building, will try again after downloading recipe dependencies.

Any ideas how this can be related to the conda-forge-ci-setup version?

This issue was resolved by pinning conda-forge-ci-setup to 3.0.3 in the past. But now there seems to be a dependency conflict and this workaround doesn't work any more.

the traceback:

Traceback (most recent call last):
  File "C:\Miniconda\Scripts\conda-mambabuild-script.py", line 9, in <module>
    sys.exit(main())
  File "C:\Miniconda\lib\site-packages\boa\cli\mambabuild.py", line 164, in main
    call_conda_build(action, config)
  File "C:\Miniconda\lib\site-packages\boa\cli\mambabuild.py", line 137, in call_conda_build
    result = api.build(
  File "C:\Miniconda\lib\site-packages\conda_build\api.py", line 186, in build
    return build_tree(
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 3068, in build_tree
    packages_from_this = build(metadata, stats,
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 2118, in build
    try_download(m, no_download_source=False, raise_error=True)
  File "C:\Miniconda\lib\site-packages\conda_build\render.py", line 663, in try_download
    raise RuntimeError("Failed to download or patch source. Please see build log for info.")

Might be related to:
conda/conda-build#4266

Issue:


Environment (conda list):
$ conda list


Details about conda and system ( conda info ):
$ conda info

Create it's own repo

Since this constitutes a real library now (with a setup.py and everything) we should most likely put the code in this repo into its own repo. This will allow us to write tests and have them run against CI without tying ourselves into knots.

Disable conflict reports?

Issue:

When conda cannot solve an environment, it will try to build a conflict report detailing the possible causes. This takes a while and is often not very helpful. Users are mostly recommended to try build the same package with mambabuild to get a better report.

Suggestion:

Turns out there's a config key (check bottom of page) to disable these reports (or hints as they call it): unsatisfiable_hints. If set to false, conda will not compute them. I guess this is honored by conda-build too.

Instead of printing the conflicts report, we can print instructions on how to enable mambabuild for debugging.

Do you think this would be a sane default behaviour for CF? It can save a lot of CI resources down the line.

Sporadic cleanup issues on AppVeyor after upload

As noted in this comment and again in this comment, it seems that sometimes we lack permissions to do a little bit of cleanup on AppVeyor (possibly due to resource contention), which causes the build to fail after uploading packages. Since this appears to be occurring after upload it isn't actually messing the uploads, but it does give one pause. Restarting seems to make it go away, but it does show up too often for comfort and it does make users nervous. Maybe sleeping for a bit before cleaning up would avoid this issue. There may be better fixes though.

Traceback (most recent call last):
  File "C:\Miniconda36-x64\Scripts\upload_or_check_non_existence.py", line 131, in <module>
    main()
  File "C:\Miniconda36-x64\Scripts\upload_or_check_non_existence.py", line 121, in main
    upload(cli, fname, owner, channel)
  File "C:\Miniconda36-x64\Scripts\upload_or_check_non_existence.py", line 68, in upload
    env=os.environ)
  File "C:\Miniconda36-x64\lib\contextlib.py", line 88, in __exit__
    next(self.gen)
  File "C:\Miniconda36-x64\Scripts\upload_or_check_non_existence.py", line 27, in get_temp_token
    shutil.rmtree(dn)
  File "C:\Miniconda36-x64\lib\shutil.py", line 494, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Miniconda36-x64\lib\shutil.py", line 389, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Miniconda36-x64\lib\shutil.py", line 387, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\tmphmihw0df\\binstar.token'

ref: https://ci.appveyor.com/project/conda-forge/conda-feedstock/build/1.0.122
ref: https://ci.appveyor.com/project/conda-forge/r-base-feedstock/build/1.0.90

Variants with multiple channel_sources

Issue

As part of the python312 migration uploaded the python3.12 RC to a seperate channel conda-forge/label/python_rc.
The migration yaml then adds a new python version value to a recipes feedstock and zips it with channel_sources. Thereby we can add a new channel_source that just gets used by the python312 build.

conda-build will render seperate variants for each (arch, python version) tuple for most feedstocks like https://github.com/conda-forge/markupsafe-feedstock/tree/main/.ci_support, however for some like https://github.com/conda-forge/brotli-feedstock/tree/main/.ci_support only a single variant for each arch is created.

setup_conda_rc is not yet able to handle this and will always just add channels from the first channel_sources value to the condarc. This results in the conda-forge/label/python_rc channel being missing in the condarc and mamba failing to solve.

As @isuruf pointed out just adding all channels from all values to the condarc will not work due to strict repo priority

See #279

Release of `urllib3` v2 breaks `binstar_client` from `anaconda-client`

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

Hi,

There is an issue with dependency of this package leading to this error:

conda create -n conda-forge-ci-setup-env conda-forge-ci-setup
conda activate conda-forge-ci-setup-env
python -c "from conda_forge_ci_setup import build_utils"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/peter/miniconda3/envs/conda-forge-ci-setup-env/lib/python3.11/site-packages/conda_forge_ci_setup/build_utils.py", line 15, in <module>
    from conda_forge_ci_setup.upload_or_check_non_existence import retry_upload_or_check
  File "/Users/peter/miniconda3/envs/conda-forge-ci-setup-env/lib/python3.11/site-packages/conda_forge_ci_setup/upload_or_check_non_existence.py", line 12, in <module>
    from binstar_client.utils import get_server_api
  File "/Users/peter/miniconda3/envs/conda-forge-ci-setup-env/lib/python3.11/site-packages/binstar_client/__init__.py", line 26, in <module>
    from .requests_ext import NullAuth
  File "/Users/peter/miniconda3/envs/conda-forge-ci-setup-env/lib/python3.11/site-packages/binstar_client/requests_ext.py", line 11, in <module>
    from urllib3.filepost import choose_boundary, iter_fields
ImportError: cannot import name 'iter_fields' from 'urllib3.filepost' (/Users/peter/miniconda3/envs/conda-forge-ci-setup-env/lib/python3.11/site-packages/urllib3/filepost.py)

This is more of an FYI as this is not really a direct issue with this package, but you might consider pinning urllib3 <2. I think the most upstream issue I can find is anaconda/anaconda-client#654.

Hope this helps, even if just an early warning.

Installed packages

conda-forge-ci-setup

Environment info

active environment : conda-forge-ci-setup-env
    active env location : /Users/peter/miniconda3/envs/conda-forge-ci-setup-env
            shell level : 2
       user config file : /Users/peter/.condarc
 populated config files : /Users/peter/.condarc
          conda version : 23.1.0
    conda-build version : not installed
         python version : 3.10.10.final.0
       virtual packages : __archspec=1=arm64
                          __osx=13.2.1=0
                          __unix=0=0
       base environment : /Users/peter/miniconda3  (writable)
      conda av data dir : /Users/peter/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/peter/miniconda3/pkgs
                          /Users/peter/.conda/pkgs
       envs directories : /Users/peter/miniconda3/envs
                          /Users/peter/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.1.0 requests/2.28.1 CPython/3.10.10 Darwin/22.3.0 OSX/13.2.1 solver/libmamba conda-libmamba-solver/22.8.1 libmambapy/1.4.0
                UID:GID : 501:20
             netrc file : None
           offline mode : False

Building locally fails, unbound variable `CI`

When running the build locally like so...

python build-locally.py <some_config_here>

One now sees an exception like this...

+ source run_conda_forge_build_setup
/opt/conda/bin/run_conda_forge_build_setup: line 4: CI: unbound variable
Traceback (most recent call last):
  File "build-locally.py", line 58, in <module>
    main()
  File "build-locally.py", line 54, in main
    run_docker_build(ns)
  File "build-locally.py", line 19, in run_docker_build
    subprocess.check_call(script)
  File "/Users/jkirkham/miniconda/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '.azure-pipelines/run_docker_build.sh' returned non-zero exit status 1.

Download macOS SDK 10.9 if missing generally

Would be good to update the check here to handle downloading the macOS 10.9 SDK generally if it is missing (as opposed to checking CircleCI specifically). Should make this work more generally on different providers with different offerings.

Increase page size for azure windows agents?

After often encountering

OSError: [WinError 1455] The paging file is too small for this operation to complete

in conda-forge/numpy-feedstock#237 & many other pull requests (for numpy), I decided to have a quick look at what's behind that.

Stumbling over this discussion (regarding GH Actions), someone noted:

I’ve investigated this issue and the problem is that Windows decides to create a single page file on the D:\ drive, which has only 14 GB, and by default it allocates only 1/8th of the volume size, or about 1.75 GB. This is a very small page file by modern standards and many applications will fail.

How about adding an option (or step) to the ci-setup that allows setting the page file size? It seems to be possible for GHAs - someone published an action for it (it's MIT licensed so we could even reuse some of that code).

Option for verbose upload

Currently we silence uploads

def upload(token_fn, path, owner, channels, private_upload=False):
cmd = ['anaconda', '--quiet', '-t', token_fn,
'upload', path, '--user={}'.format(owner),
'--channel={}'.format(channels)]

However if we run into an issue where the uploading itself is broken and we would like to debug it ( like issue conda-forge/kubo-feedstock#5 ), currently we lack the ability to configure a verbose upload. Would be good to have some way to configure this

[warning] failed package validation and/or copy for commit 22b4448b3d9feda2f78f94149b68a015366c3a9e

Hi @conda-forge/conda-forge-ci-setup! This is the friendly automated conda-forge-webservice!

It appears that one or more of your feedstock's outputs did not copy from the
staging channel (cf-staging) to the production channel (conda-forge). :(

This failure can happen for a lot of reasons, including an outdated feedstock
token. Below we have put some information about the failure to help you debug it.

Rerendering the feedstock will usually fix these problems.

If you have any issues or questions, you can find us on Element in the
community channel or you can bump us right here.

output validation (is this output allowed for your feedstock?):

  • linux-64/conda-forge-ci-setup-3.33.0-py310hb39dd43_0.conda: True

copied (did this output get copied to the production channel?):

  • linux-64/conda-forge-ci-setup-3.33.0-py310hb39dd43_0.conda: False

CMAKE_CROSSCOMPILING_EMULATOR not propagated automatically to cmake

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

The ./recipe/cross-compile-support.sh script configures CMAKE_CROSSCOMPILING_EMULATOR as an environment variable that gets propagated to conda-build:

echo "CMAKE_CROSSCOMPILING_EMULATOR: " >> ${CI_SUPPORT}/${CONFIG}.yaml
echo "- /usr/bin/qemu-$(echo $HOST_PLATFORM | cut -b 7-)-static" >> ${CI_SUPPORT}/${CONFIG}.yaml

However, AFAICT from https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html, that variable isn't one that cmake will automatically pick up from the environment, so one currently needs to include something like -DCMAKE_CROSSCOMPILING_EMULATOR:STRING="${CMAKE_CROSSCOMPILING_EMULATOR}" in the build script to pass it along manually.

See conda-forge/ldas-tools-framecpp-feedstock#26 for a feedstock build that failed because of this, but is resolved by adding the above cmake argument.

I'm not sure where this would be applied, but is it reasonable to add CMAKE_CROSSCOMPILING_EMULATOR to the CMAKE_ARGS variable currently configured by the C compiler activate scripts? Or maybe there's a better way.

Installed packages

-

Environment info

-

Appveyor build failing with [SSL: CERTIFICATE_VERIFY_FAILED]

Issue:

Trying to update climlab-feedstock (conda-forge/climlab-feedstock#19) and this build on Appveyor is failing with

urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>

Traceback shows this is triggered by a urllib2 call in ff_ci_pr_build.py.

The Appveyor build is working for Python 3.7 but not for Python 3.6

If there's a better place to open this, please let me know, thanks!

Full error message below.

Checking to see if this PR build is outdated.
9Traceback (most recent call last):
10  File "C:\projects\climlab-feedstock\ff_ci_pr_build.py", line 201, in <module>
11    sys.exit(main())
12  File "C:\projects\climlab-feedstock\ff_ci_pr_build.py", line 192, in main
13    exit_code = int(appveyor_check_latest_pr_build(repo, pr, bld) is False)
14  File "C:\projects\climlab-feedstock\ff_ci_pr_build.py", line 117, in appveyor_check_latest_pr_build
15    data = request_json(url.format(repo=repo, total_builds=total_builds), headers=headers)
16  File "C:\projects\climlab-feedstock\ff_ci_pr_build.py", line 46, in request_json
17    with contextlib.closing(urlopen(request)) as response:
18  File "C:\Python27\lib\urllib2.py", line 154, in urlopen
19    return opener.open(url, data, timeout)
20  File "C:\Python27\lib\urllib2.py", line 429, in open
21    response = self._open(req, data)
22  File "C:\Python27\lib\urllib2.py", line 447, in _open
23    '_open', req)
24  File "C:\Python27\lib\urllib2.py", line 407, in _call_chain
25    result = func(*args)
26  File "C:\Python27\lib\urllib2.py", line 1241, in https_open
27    context=self._context)
28  File "C:\Python27\lib\urllib2.py", line 1198, in do_open
29    raise URLError(err)
30urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>
31Command exited with code 1

Set CPU_COUNT correctly for Azure

Currently CPU_COUNT is set for the previously existing CIs (i.e Travis, Circle, and AppVeyor). However to the best of my knowledge CPU_COUNT is likely not set correctly for Azure. Would be good to update the values set in these scripts to configure Azure correctly.

Bootstrapping

This package does not bootstrap well as it used to. This needs to be fixed. Not urgent though

Conda should use newer (the newest?) OSX SDK for building

Please see the discussion here:
conda-forge/pyside2-feedstock#46 (comment)

Conda-forge currently builds against the OSX SDK 10.9. This causes issues for example with Qt + Pyside2, which requires both Qt and python to be built with OSX SDK 10.14 or 10.15.

I understand that Conda-forge builds against SDK 10.9 to be backwards compatible to that version of OSX. However, this is also possible with building against the newest SDK (10.15), and the setting the deployment target to a lower OSX version, like 10.9.

Here again the details:

Where did you read this? That's not how how SDKs work. You build with 10.12 and it will run on 10.14, not the other way around.

See on developer.apple.com:

It works both ways on OSX.

In the example of the link I posted, they build with base SDK 10.6 and run on 10.4 and newer. If the code doesn't use any of the newer API's (not existing in 10.6) this works out of the box. If the code wants to use new 10.5 or 10.6 API's on supported operating systems, the code must first check the OS version before using the API's. Which I guess Qt does, when they say on their official docs that compiling against the 10.14 SDK and then running on 10.12 works fine...

So Apple recommends always using the newest SDK, and then setting the Deployment target to the lowest OSX version you want to support...

CI output is prefixed by some weird unicode

Snippets from an actual run (don't have a link to Azure because it's washed out, sorry):

    Could not solve for environment specs
    The following packages are incompatible
    \u251c\u2500 cuda-cudart-dev is installable and it requires
    \u2502  \u2514\u2500 cuda-cudart 12.0.107 h63175ca_6, which requires
    \u2502     \u2514\u2500 cuda-cudart_win-64 12.0.107 h63175ca_6, which requires
    \u2502        \u2514\u2500 cuda-version >=12.0,<12.1.0a0 , which requires
    \u2502           \u2514\u2500 cudatoolkit 12.0|12.0.* , which can be installed;
    \u2514\u2500 cudnn >=8.0.*,<9  is not installable because there are no viable options
       \u251c\u2500 cudnn 8.0.5.39 would require
       \u2502  \u2514\u2500 cudatoolkit 10.1|10.1.* , which conflicts with any installable versions previously reported;
       \u251c\u2500 cudnn [8.0.5.39|8.1.0.77|8.2.0.53|8.2.1.32] would require
       \u2502  \u2514\u2500 cudatoolkit [10.2.* |10.2|10.2.* ], which conflicts with any installable versions previously reported;
       \u251c\u2500 cudnn 8.0.5.39 would require
       \u2502  \u2514\u2500 cudatoolkit 11.0|11.0.* , which conflicts with any installable versions previously reported;
       \u251c\u2500 cudnn [8.1.0.77|8.2.0.53|...|8.4.1.50] would require
       \u2502  \u2514\u2500 cudatoolkit 11.* , which conflicts with any installable versions previously reported;
       \u251c\u2500 cudnn [8.3.2.44|8.4.0.27] would require
       \u2502  \u2514\u2500 cudatoolkit >=11.2,<12 , which conflicts with any installable versions previously reported;
       \u2514\u2500 cudnn 8.8.0.121 would require
          \u2514\u2500 cuda-version >=11.0,<12.0a0 , which conflicts with any installable versions previously reported.

Something seems to be off when rendering the output.

The 2.0 branch is having issues uploading sometimes

I am seeing the following issue on the make feedstock, with some uploads: https://travis-ci.org/conda-forge/make-feedstock/jobs/418844902#L984

Not really sure what is going on here :( CC @mariusvniekerk

$ upload_package ./ ./recipe ./.ci_support/${CONFIG}.yaml
Adding in variants from internal_defaults
Adding in variants from ./.ci_support/osx_c_compilertoolchain_c.yaml
Attempting to finalize metadata for make
Solving environment: ...working... done
Solving environment: ...working... done
[ERROR] File "/Users/travis/miniconda3/conda-bld/osx-64/make-4.2.1-h470a237_1002.tar.bz2" does not exist
Traceback (most recent call last):
  File "/Users/travis/miniconda3/bin/upload_package", line 11, in <module>
    sys.exit(upload_package())
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/conda_forge_ci_setup/build_utils.py", line 81, in upload_package
    upload_or_check(recipe_root, owner, channel, [config_file])
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/conda_forge_ci_setup/upload_or_check_non_existence.py", line 123, in upload_or_check
    upload(token_fn, path, owner, channel)
  File "/Users/travis/miniconda3/lib/python3.6/site-packages/conda_forge_ci_setup/upload_or_check_non_existence.py", line 65, in upload
    env=os.environ)
  File "/Users/travis/miniconda3/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['anaconda', '--quiet', '-t', '/var/folders/vy/rcv48w3j4w79llzf_x6qnvw40000gn/T/tmpx90qiwh6/binstar.token', 'upload', '/Users/travis/miniconda3/conda-bld/osx-64/make-4.2.1-h470a237_1002.tar.bz2', '--user=conda-forge', '--channel=main']' returned non-zero exit status 1.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.