quantco / pixi-pack Goto Github PK

• Introduction
• Installation
• Usage

Starting with a pixi lockfile pixi.lock, you can create a packed environment that can be shared with others. This environment can be unpacked on any system using pixi-pack to recreate the original environment.

In contrast to conda-pack, pixi-pack does not require the original conda environment to be present on the system for packing. Instead, it uses the lockfile to download the required packages and puts them into a .tar archive. This archive can then be shared with others and installed using pixi-pack unpack to recreate the original environment.

The original motivation behind pixi-pack was to create a conda-pack alternative that does not have the same reproducibility issues as conda-pack. It also aims to allow cross-platform building packs, so you can create a pack for win-64 on a linux-64 system.

You can install pixi-pack using pixi:

pixi global install pixi-pack

Or using cargo:

cargo install --locked --git https://github.com/quantco/pixi-pack.git

Or by downloading our pre-built binaries from the releases page.

Instead of installing pixi-pack globally, you can also use pixi exec to run pixi-pack in a temporary environment:

pixi exec pixi-pack pack
pixi exec pixi-pack unpack environment.tar

With pixi-pack pack, you can pack a conda environment into a environment.tar file:

pixi-pack pack --manifest-file pixi.toml --environment prod --platform linux-64

This will create a environment.tar file that contains all conda packages required to create the environment.

# environment.tar
| pixi-pack.json
| environment.yml
| channel
|    ├── noarch
|    |    ├── tzdata-2024a-h0c530f3_0.conda
|    |    ├── ...
|    |    └── repodata.json
|    └── linux-64
|         ├── ca-certificates-2024.2.2-hbcca054_0.conda
|         ├── ...
|         └── repodata.json

With pixi-pack unpack environment.tar, you can unpack the environment on your target system. This will create a new conda environment in ./env that contains all packages specified in your pixi.toml. It also creates an activate.sh (or activate.bat on Windows) file that lets you activate the environment without needing to have conda or micromamba installed.

$ pixi-pack unpack environment.tar
$ ls
env/
activate.sh
environment.tar
$ cat activate.sh
export PATH="/home/user/project/env/bin:..."
export CONDA_PREFIX="/home/user/project/env"
. "/home/user/project/env/etc/conda/activate.d/activate_custom_package.sh"

Since pixi-pack just downloads the .conda and .tar.bz2 files from the conda repositories, you can trivially create packs for different platforms.

pixi-pack pack --platform win-64

You can inject additional packages into the environment that are not specified in pixi.lock by using the --inject flag:

pixi-pack pack --inject local-package-1.0.0-hbefa133_0.conda --manifest-pack pixi.toml

This can be particularly useful if you build the project itself and want to include the built package in the environment but still want to use pixi.lock from the project.

If you don't have pixi-pack available on your target system, you can still install the environment if you have conda or micromamba available. Just unarchive the environment.tar, then you have a local channel on your system where all necessary packages are available. Next to this local channel, you will find an environment.yml file that contains the environment specification. You can then install the environment using conda or micromamba:

tar -xvf environment.tar
micromamba create -p ./env --file environment.yml
# or
conda env create -p ./env --file environment.yml