Git Product home page Git Product logo

setup-pixi's Introduction

License CI Latest release Project Chat

setup-pixi ๐Ÿ“ฆ

GitHub Action to set up the pixi package manager.

Usage

- uses: prefix-dev/[email protected]
  with:
    pixi-version: v0.20.0
    cache: true
    auth-host: prefix.dev
    auth-token: ${{ secrets.PREFIX_DEV_TOKEN }}
- run: pixi run test

Warning

Since pixi is not yet stable, the API of this action may change between minor versions. Please pin the versions of this action to a specific version (i.e., prefix-dev/[email protected]) to avoid breaking changes. You can automatically update the version of this action by using Dependabot.

Put the following in your .github/dependabot.yml file to enable Dependabot for your GitHub Actions:

version: 2
updates:
  - package-ecosystem: github-actions
    directory: /
    schedule:
      interval: monthly # or daily, weekly
    groups:
      dependencies:
        patterns:
          - "*"

Features

To see all available input arguments, see the action.yml file.

Caching

The action supports caching of the pixi environment. By default, caching is enabled if a pixi.lock file is present. It will then use the pixi.lock file to generate a hash of the environment and cache it. If the cache is hit, the action will skip the installation and use the cached environment. You can specify the behavior by setting the cache input argument.

If you need to customize your cache-key, you can use the cache-key input argument. This will be the prefix of the cache key. The full cache key will be <cache-key><conda-arch>-<hash>.

Only save caches on main

In order to not exceed the 10 GB cache size limit as fast, you might want to restrict when the cache is saved. This can be done by setting the cache-write argument.

- uses: prefix-dev/[email protected]
  with:
    cache: true
    cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}

Multiple environments

With pixi, you can create multiple environments for different requirements. You can also specify which environment(s) you want to install by setting the environments input argument. This will install all environments that are specified and cache them.

[project]
name = "my-package"
channels = ["conda-forge"]
platforms = ["linux-64"]

[dependencies]
python = ">=3.11"
pip = "*"
polars = ">=0.14.24,<0.21"

[feature.py311.dependencies]
python = "3.11.*"
[feature.py312.dependencies]
python = "3.12.*"

[environments]
py311 = ["py311"]
py312 = ["py312"]

Multiple environments using a matrix

The following example will install the py311 and py312 environments in different jobs.

test:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      environment: [py311, py312]
  steps:
  - uses: actions/checkout@v4
  - uses: prefix-dev/[email protected]
    with:
      environments: ${{ matrix.environment }}

Install multiple environments in one job

The following example will install both the py311 and the py312 environment on the runner.

- uses: prefix-dev/[email protected]
  with:
    # separated by spaces
    environments: >-
      py311
      py312
- run: |
    pixi run -e py311 test
    pixi run -e py312 test

Warning

If you don't specify any environment, the default environment will be installed and cached, even if you use other environments.

Authentication

There are currently three ways to authenticate with pixi:

  • using a token
  • using a username and password
  • using a conda-token

For more information, see the pixi documentation.

Warning

Please only store sensitive information using GitHub secrets. Do not store them in your repository. When your sensitive information is stored in a GitHub secret, you can access it using the ${{ secrets.SECRET_NAME }} syntax. These secrets will always be masked in the logs.

Token

Specify the token using the auth-token input argument. This form of authentication (bearer token in the request headers) is mainly used at prefix.dev.

- uses: prefix-dev/[email protected]
  with:
    auth-host: prefix.dev
    auth-token: ${{ secrets.PREFIX_DEV_TOKEN }}

Username and password

Specify the username and password using the auth-username and auth-password input arguments. This form of authentication (HTTP Basic Auth) is used in some enterprise environments with artifactory for example.

- uses: prefix-dev/[email protected]
  with:
    auth-host: custom-artifactory.com
    auth-username: ${{ secrets.PIXI_USERNAME }}
    auth-password: ${{ secrets.PIXI_PASSWORD }}

Conda-token

Specify the conda-token using the conda-token input argument. This form of authentication (token is encoded in URL: https://my-quetz-instance.com/t/<token>/get/custom-channel) is used at anaconda.org or with quetz instances.

- uses: prefix-dev/[email protected]
  with:
    auth-host: anaconda.org # or my-quetz-instance.com
    conda-token: ${{ secrets.CONDA_TOKEN }}

Custom shell wrapper

setup-pixi allows you to run command inside of the pixi environment by specifying a custom shell wrapper with shell: pixi run bash -e {0}. This can be useful if you want to run commands inside of the pixi environment, but don't want to use the pixi run command for each command.

- run: | # everything here will be run inside of the pixi environment
    python --version
    pip install --no-deps -e .
  shell: pixi run bash -e {0}

You can even run Python scripts like this:

- run: | # everything here will be run inside of the pixi environment
    import my_package
    print("Hello world!")
  shell: pixi run python {0}

If you want to use PowerShell, you need to specify -Command as well.

- run: | # everything here will be run inside of the pixi environment
    python --version | Select-String "3.11"
  shell: pixi run pwsh -Command {0} # pwsh works on all platforms

Note

Under the hood, the shell: xyz {0} option is implemented by creating a temporary script file and calling xyz with that script file as an argument. This file does not have the executable bit set, so you cannot use shell: pixi run {0} directly but instead have to use shell: pixi run bash {0}. There are some custom shells provided by GitHub that have slightly different behavior, see jobs.<job_id>.steps[*].shell in the documentation. See the official documentation and ADR 0277 for more information about how the shell: input works in GitHub Actions.

--frozen and --locked

You can specify whether setup-pixi should run pixi install --frozen or pixi install --locked depending on the frozen or the locked input argument. See the official documentation for more information about the --frozen and --locked flags.

- uses: prefix-dev/[email protected]
  with:
    locked: true
    # or
    frozen: true

If you don't specify anything, the default behavior is to run pixi install --locked if a pixi.lock file is present and pixi install otherwise.

Debugging

There are two types of debug logging that you can enable.

Debug logging of the action

The first one is the debug logging of the action itself. This can be enabled by running the action with the RUNNER_DEBUG environment variable set to true.

- uses: prefix-dev/[email protected]
  env:
    RUNNER_DEBUG: true

Alternatively, you can enable debug logging for the action by re-running the action in debug mode:

Re-run in debug mode Re-run in debug mode

For more information about debug logging in GitHub Actions, see the official documentation.

Debug logging of pixi

The second type is the debug logging of the pixi executable. This can be specified by setting the log-level input.

- uses: prefix-dev/[email protected]
  with:
    # one of `q`, `default`, `v`, `vv`, or `vvv`.
    log-level: vvv

If nothing is specified, log-level will default to default or vv depending on if debug logging is enabled for the action.

Self-hosted runners

On self-hosted runners, it may happen that some files are persisted between jobs. This can lead to problems or secrets getting leaked between job runs. To avoid this, you can use the post-cleanup input to specify the post cleanup behavior of the action (i.e., what happens after all your commands have been executed).

If you set post-cleanup to true, the action will delete the following files:

  • .pixi environment
  • the pixi binary
  • the rattler cache
  • other rattler files in ~/.rattler

If nothing is specified, post-cleanup will default to true.

On self-hosted runners, you also might want to alter the default pixi install location to a temporary location. You can use pixi-bin-path: ${{ runner.temp }}/bin/pixi to do this.

- uses: prefix-dev/[email protected]
  with:
    post-cleanup: true
    # ${{ runner.temp }}\Scripts\pixi.exe on Windows
    pixi-bin-path: ${{ runner.temp }}/bin/pixi

You can also use a preinstalled local version of pixi on the runner by not setting any of the pixi-version, pixi-url or pixi-bin-path inputs. This action will then try to find a local version of pixi in the runner's PATH.

Using the pyproject.toml as a manifest file for pixi.

setup-pixi will automatically pick up the pyproject.toml if it contains a [tool.pixi.project] section and no pixi.toml. This can be overwritten by setting the manifest-path input argument.

- uses: prefix-dev/[email protected]
  with:
    manifest-path: pyproject.toml

More examples

If you want to see more examples, you can take a look at the GitHub Workflows of this repository.

Local Development

  1. Clone this repository.
  2. Run pnpm install inside the repository (if you don't have pnpm installed, you can install it with pixi global install pnpm).
  3. Run pnpm dev for live transpilation of the TypeScript source code.
  4. To test the action, you can run act (inside docker) or use โœจ CI driven development โœจ.

setup-pixi's People

Contributors

dependabot[bot] avatar github-actions[bot] avatar nichmor avatar pavelzw avatar ruben-arts avatar wpbonelli avatar ytausch avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

setup-pixi's Issues

Non working state on self-hosted workers

Setup-pixi often falls into a non-working state on my self-hosted workers. I happens if it somehow doesn't managed to clean up after itself nicely (post-cleanup: true).

Then on next run the pipeline fails with:

image

Maybe just force delete it, or check if the version of pixi is OK, and then just continue.

Right now I have a workaround steps in the pipeline to recover from this. Any other good ideas?

Cache restore fails on self-hosted windows runners

I stumbled on this problem with out self-hosted windows runners.

It doesn't cause the action to fail, so I only saw by chance.

image

It looks like the cache is downloading but tar fails when trying to extract it, reporting that the file is missing. I tried to look in the temp folder while the pipeline is running, and there is no cache.tgz file in the folder.

Support installer environment variables?

The pixi docs describe several environment variables which control the behavior of the installer scripts. I know this action installs pixi via downloadTool(), not with the install scripts, but perhaps still worth supporting? At least the ones shared on all platforms:

  • PIXI_VERSION (this is just VERSION on linux/mac, maybe worth standardizing?)
  • PIXI_HOME

Just convenience, but it would be nice to be able to set PIXI_VERSION at the top of a workflow instead of setting version separately wherever this action is used.

I guess the way to do this would be to check process.env here

Matrix testing

I am experimenting using pixi and its GH Actions to replace my micromamba workflow.

Currently my CI looks like:

      - name: Setup mamba
        uses: mamba-org/setup-micromamba@v1
        with:
          environment-file: env.yml
          environment-name: my_env
          cache-environment: true
          cache-downloads: true
          create-args: >-
            python=${{ matrix.python-version }}
            rdkit=${{ matrix.rdkit-version }}

I am wondering whether something similar is possible with prefix-dev/setup-pixi?

Pixi location change breaks self-hosted runner

Today I updated our main branch to use pixi v0.12.0 and setup-pixi to v0.4.1. After this, I continued work on another branch, which was behind main, and my CI jobs fail as follows:

Prepare all required actions
Getting action download info
Download action repository 'prefix-dev/[email protected]' (SHA:ccc5c07ed948b849df4b[3](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7560629057/job/20587085983#step:4:3)578d785f3afd7[4](https://github.com/DeepStructure-io/DeepStructure/actions/runs/7560629057/job/20587085983#step:4:4)65c67)
Run ./.github/actions/pixi-setup
Run prefix-dev/[email protected]
Downloading Pixi
Restoring pixi cache
/home/github/work/_temp/bin/pixi install --locked --manifest-path pixi.toml
  ร— the project location seems to be change from `/home/runner/work/
  โ”‚ DeepStructure/DeepStructure/.pixi` to `/home/github/work/DeepStructure/
  โ”‚ DeepStructure/.pixi`, this is not allowed.
  โ”‚ Please remove the `.pixi` folder and run again

Error: The process '/home/github/work/_temp/bin/pixi' failed with exit code 1

We are using a self-hosted runner for our CI/CD jobs. I had a similar issue in the past with cache on self-hosted runners when upgrading (#25), but this seems slightly different.

Auth is not working on `ubuntu-latest`

Depends on prefix-dev/pixi#334

[project]
name = "test-auth"
version = "0.1.0"
description = "Add a short description here"
authors = ["Pavel"]
channels = ["conda-forge", "https://repo.prefix.dev/setup-pixi-test"]
platforms = ["osx-arm64", "osx-64", "linux-64", "linux-aarch64", "win-64"]

[tasks]
test = "python -c 'print(\"Hello world!\")'"

[dependencies]
python = "3.9.*"
private-package = "*"
  auth-token-install:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - name: Move pixi.toml
        run: mv test/auth/* .
      - uses: prefix-dev/[email protected]
        with:
          auth-host: https://repo.prefix.dev
          auth-token: ${{ secrets.PREFIX_DEV_TOKEN }}
      - name: Ensure private-package is installed
        run: |
          test -f .pixi/env/conda-meta/private-package-0.0.1-0.json

This currently doesn't work on ubuntu-latest because on default linux runners, there is no keychain available. pixi then default to the insecure auth method to write the credentials in ~/.rattler but apparently they are not read.

See also #13


A workaround to fix this is adding the following before uses: setup-pixi

      - name: Unlock Keychain
        if: matrix.os == 'ubuntu-latest'
        run: |
          sudo apt-get update
          sudo apt-get install -y dbus-x11 gnome-keyring
          dbus-launch --sh-syntax
          echo '' | /usr/bin/gnome-keyring-daemon --unlock

Issues using self-hosted runners

I'm having a bunch of issues running this action with a self-hosted runner. Everything worked well until I made a change today to update some versions in pixi.toml.

My action basically looks like this:

env:
  PIXI_VERSION: 0.5.0

jobs:
  Lint:
    runs-on: [self-hosted, Linux, X64, ubuntu-22.04]
    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Setup pixi
      uses: prefix-dev/[email protected]
      with:
        pixi-version: v${{ env.PIXI_VERSION }}

    ...
  
  Test:
    runs-on: [self-hosted, Linux, X64, ubuntu-22.04]
    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3

    - name: Setup pixi
      uses: prefix-dev/[email protected]
      with:
        pixi-version: v${{ env.PIXI_VERSION }}

    ...

When I updated the pixi.toml and pixi.lock, I get errors:

TestENOENT: no such file or directory, lstat '.pixi'
--
TestDestination file path /home/github/.pixi/bin/pixi already exists

or

Run prefix-dev/[email protected]
Downloading Pixi
Error: Destination file path /home/github/.pixi/bin/pixi already exists

I've tried every combo of post-cleanup:true/post-cleanup:false based on the docs, and tried with and without cache:true.

I'm not sure how to fix this. Any tips?

`pixi-version` errors when given an actual version

When trying a sightly simplified example from the README

      - uses: prefix-dev/[email protected]
        with:
          pixi-version: "0.2.0"
          cache: true

I get the following error:

/home/runner/work/_actions/prefix-dev/setup-pixi/v0.2.0/dist/index.js:57957
    throw result.error;
    ^

ZodError: [
  {
    "validation": "regex",
    "code": "invalid_string",
    "message": "Invalid",
    "path": []
  }
]
    at get error [as error] (/home/runner/work/_actions/prefix-dev/setup-pixi/v0.2.0/dist/index.js:57858:24)
    at ZodUnion.parse (/home/runner/work/_actions/prefix-dev/setup-pixi/v0.2.0/dist/index.js:57957:18)
    at parseOrUndefined (/home/runner/work/_actions/prefix-dev/setup-pixi/v0.2.0/dist/index.js:60973:17)
    at getOptions (/home/runner/work/_actions/prefix-dev/setup-pixi/v0.2.0/dist/index.js:61050:18)
    at Object.<anonymous> (/home/runner/work/_actions/prefix-dev/setup-pixi/v0.2.0/dist/index.js:61073:15)
    at Module._compile (node:internal/modules/cjs/loader:1233:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
  issues: [
    {
      validation: 'regex',
      code: 'invalid_string',
      message: 'Invalid',
      path: []
    }
  ],
  addIssue: [Function (anonymous)],
  addIssues: [Function (anonymous)],
  errors: [
    {
      validation: 'regex',
      code: 'invalid_string',
      message: 'Invalid',
      path: []
    }
  ]
}

Setting it to "latest" works fine so I guess something is broken with the regex:

      - uses: prefix-dev/[email protected]
        with:
          pixi-version: latest
          cache: true

Restoring cache fails with pyproject.toml

Say my repo is named foo, and inside the repo I have another directory named foo.
At the root of the repo I have a pyproject.toml file that contains:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "foo"
requires-python = ">=3.10"
dynamic = ["version"]

[tool.pixi.project]
name = "foo"
channels = ["conda-forge"]

[tool.pixi.pypi-dependencies]
foo = { path = "./", editable = true }

When I use the setup-pixi action version 0.5.1 it fails:

Restoring pixi cache
Error: Failed to generate cache key: Error: ENOENT: no such file or directory, open '/home/runner/work/foo/foo/pyproject.lock'

GitHub job:

jobs:
  pytest:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
      - uses: prefix-dev/[email protected]
        with:
          manifest-path: pyproject.toml
          pixi-version: v0.18.0
          cache: true
          cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}
      - name: pytest
        run: pixi run pytest --capture=no --verbose

Updating pixi version breaks pixi install

I have the following workflow:

env:
    PIXI_VERSION: 0.5.0
    ....

jobs:
    Test:
        # Use our own self-hosted runner for this
        runs-on: [self-hosted, Linux, X64, ubuntu-22.04]
        steps:
            ...
            - name: Setup pixi
              uses: prefix-dev/[email protected]
              with:
                  pixi-version: v${{ env.PIXI_VERSION }}
                  # Don't use $HOME/.pixi to work around self-runner env issues
                  # https://github.com/prefix-dev/setup-pixi/issues/25
                  pixi-bin-path: ${{ runner.temp }}/bin/pixi
                  cache: true
                  post-cleanup: true

I updated it to use the new release of pixi (0.7.0) and it breaks as follows:

Restoring pixi cache
  Cache Size: ~108 MB (112936695 B)
  /usr/bin/tar -xf /home/github/work/_temp/1c6d344c-c994-48ee-aa1b-cc8e0f6be174/cache.tzst -P -C /home/github/work/Project/Project --use-compress-program unzstd
  Received 112936695 of 112936695 (100.0%), 107.7 MBs/sec
  Cache restored successfully
  Restored cache with key `pixi-linux-64-374cf0d4d1aa8b897059fc7d641107d070f1dafa01ec8a5d5ceeefe4f4f9adb1`
/home/github/work/_temp/bin/pixi install --locked --manifest-path pixi.toml
  ร— the project location seems to be change from `/home/runner/work/
  โ”‚ Project/Project/.pixi` to `/mnt/github/work/Project/
  โ”‚ Project/.pixi`, this is not allowed.
  โ”‚ Please remove the `.pixi` folder and run again

Use proxy with pixi

Is it possible to setup a proxy for pixi to use before downloading the packages?

Something similar to http_proxy in conda.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.