Git Product home page Git Product logo

conda-docker's People

Contributors

chrisburr avatar costrouc avatar saraedum avatar scopatz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

conda-docker's Issues

Does this support pip packages too?

Awesome and exciting project :D <3

Does this support pip packages as well? Or do they all need to be in conda? Not sure how the layering would work..

Improve error-checking in `chroot_install`

Currently, it has this comment:

    # FIXME: this should reall be check_output(), but chroot or fakechroot is
    # giving some weird segfault after the install command completes ¯\_(ツ)_/¯
    subprocess.call(
        [
            "fakechroot",
            "chroot",
            new_root,
            "/_conda.exe",
            "install",
            "--offline",
            "--file",
            "/opt/conda/pkgs/env.txt",
            "-y",
            "--prefix",
            "/opt/conda",
        ],
        env=env,
        cwd=host_conda_opt,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )

Suggestions:

  • Fix the typo: "reall" -> "really"
  • Check output as the comment suggests
  • See if fakechroot propagates the exit code if an internal command fails (otherwise, it'll succeed even with check_output)
  • Figure out the cause of the segfault if it repros.

Use requests instead of urllib for pulling images wierd issues on S3 with headers not set properly

Was able to reproduce this issue:

  • digest = get_blob(image, "sha256:2de88731a44c549d7de6b752d0ae11907850b8363e6d1e2ca862d1fe6a14a4e8", token) returned 400
  • requests.get(f"https://registry-1.docker.io/v2/{image}/blobs/sha256:2de88731a44c549d7de6b752d0ae11907850b8363e6d1e2ca862d1fe6a14a4e8", headers = {"Authorization": f"Bearer {token}"}) returned 200

So the easiest workaround is to have requests as a dependency. I'm sure there is something that is not being set properly with urllib.

Status

Hi,

this sounds really useful for many users who are using conda successfully but having trouble packaging their conda envs easily into docker or singularity containers.

My question is about the status and future of this project. Is this a working prototype ? Or mature already, or just a test? How does it compare to the experimental conda-pack ?

Thanks for your efforts
Colin

conda-forge install failing

I've tried a number of different ways and I'm having trouble installing the package via conda-forge due to package conflicts.

conda create -n conda_docker -c conda-forge --no-default-packages python=3.8 conda-docker

That returns the following:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: / 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                      

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package python conflicts for:
conda-docker -> conda -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0|3.4.*']
python=3.8
conda-docker -> python

Thanks!

Example fails if /bin not on PATH

On Arch Linux the example:

conda docker build -b frolvlad/alpine-glibc:latest -i example-image:123456 -o demo.tar numpy numba flask

fails with the error:

Executing transaction: done
Rolling back transaction: done

Exception('No compatible shell found!')
()

/.../sbin/chroot.fakechroot: line 147: 602419 Segmentation fault      (core dumped) env -u FAKECHROOT_BASE_ORIG FAKECHROOT_CMD_ORIG= LD_LIBRARY_PATH="$fakechroot_chroot_paths" FAKECHROOT_BASE="$fakechroot_chroot_base" "$fakechroot_chroot_chroot" "${@:1:$(($fakechroot_chroot_n - 1))}" "$fakechroot_chroot_final_newroot" "${@:$(($fakechroot_chroot_n + 1))}"
INFO:conda_docker.conda:building conda environment took 20.671 [s]
INFO:conda_docker.conda:adding conda environment in package layers
Traceback (most recent call last):
  File "/.../bin/conda-docker", line 11, in <module>
    load_entry_point('conda-docker', 'console_scripts', 'conda-docker')()
  File "/.../conda_docker/cli.py", line 121, in main
    cli(args)
  File "/.../conda_docker/cli.py", line 26, in cli
    args.func(args)
  File "/.../conda_docker/cli.py", line 114, in handle_conda_build
    layering_strategy=args.layering_strategy,
  File "/.../conda_docker/conda.py", line 635, in build_docker_environment
    layering_strategy=layering_strategy,
  File "/.../conda_docker/conda.py", line 582, in add_conda_layers
    image, hostpath, arcpath=arcpath, filter=filter, records=records
  File "/.../conda_docker/conda.py", line 548, in add_conda_package_layers
    with open(meta_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpi3vqxxt8/opt/conda/conda-meta/_libgcc_mutex-0.1-conda_forge.json'

This is because PATH is being propagated from the host and doesn't include /bin (in this alpine image the shell is /bin/bash). Appending /bin to PATH gets me slightly further (before hitting an unrelated segfault that I'm still debugging).

failing to get latest packages from conda-forge??

the minimal example fail on my computer:

> conda docker build -b frolvlad/alpine-glibc:latest \                                                                                              (dockertest) 
                                                                           -i example-image:123456 \
                                                                           -o demo.tar \
                                                                           numpy numba flask
INFO:conda_docker.conda:solving conda environment
Dry run. Exiting.
INFO:conda_docker.conda:solving conda environment took 20.188 [s]
INFO:conda_docker.conda:loading repodata
INFO:conda_docker.conda:loading repodata took 14.458 [s]
Traceback (most recent call last):
  File "/home/resolvent/miniforge3/envs/dockertest/bin/conda-docker", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/resolvent/miniforge3/envs/dockertest/lib/python3.11/site-packages/conda_docker/cli.py", line 125, in main
    cli(args)
  File "/home/resolvent/miniforge3/envs/dockertest/lib/python3.11/site-packages/conda_docker/cli.py", line 26, in cli
    args.func(args)
  File "/home/resolvent/miniforge3/envs/dockertest/lib/python3.11/site-packages/conda_docker/cli.py", line 96, in handle_conda_build
    precs = find_precs(
            ^^^^^^^^^^^
  File "/home/resolvent/miniforge3/envs/dockertest/lib/python3.11/site-packages/conda_docker/conda.py", line 255, in find_precs
    precs = precs_from_package_specs(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/resolvent/miniforge3/envs/dockertest/lib/python3.11/site-packages/conda_docker/conda.py", line 222, in precs_from_package_specs
    raise e    
    ^^^^^^^
  File "/home/resolvent/miniforge3/envs/dockertest/lib/python3.11/site-packages/conda_docker/conda.py", line 217, in precs_from_package_specs
    pkg_repodata = repodatas[channel]["packages"][fn]
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
KeyError: 'ca-certificates-2022.12.7-ha878542_0.tar.bz2'

looking into repodatas[channel]["packages"] I see that it has:

...
ca-certificates-2022.6.15-ha878542_0.tar.bz2
ca-certificates-2022.6.15.1-ha878542_0.tar.bz2
ca-certificates-2022.6.15.2-ha878542_0.tar.bz2
ca-certificates-2022.9.14-ha878542_0.tar.bz2
ca-certificates-2022.9.24-ha878542_0.tar.bz2
cabal-2.4.1.0-h2ccb070_0.tar.bz2
...

but https://anaconda.org/conda-forge/ca-certificates/files has 2022.12.7

How can this be??

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.