linuxdeploy / linuxdeploy-plugin-conda Goto Github PK

View Code? Open in Web Editor NEW

29.0 29.0 12.0 44 KB

Python plugin for linuxdeploy. Sets up miniconda in an AppDir and installs user specified packages.

License: MIT License

Shell 100.00%

linuxdeploy-plugin-conda's Introduction

linuxdeploy

AppDir creation and maintenance tool.

About

AppImages are a well known and quite popular format for distributing applications from developers to end users.

appimagetool, the tool creating AppImages, expects directories in a specific format that will then be converted into the final AppImage. This format is called AppDir. It is not very difficult to understand, but creating AppDirs for arbitrary applications tends to be a very repetitive task. Also, bundling all the dependencies properly can be a quite difficult task. It seems like there is a need for tools which simplify these tasks.

linuxdeploy is designed to be an AppDir maintenance tool. It provides extensive functionalities to create and bundle AppDirs for applications. It features a plugin system that allows for easy bundling of frameworks and creating output bundles such as AppImages with little effort.

linuxdeploy was greatly influenced by linuxdeployqt, and while it employs stricter rules on AppDirs, it's more flexible in use. If you use linuxdeployqt at the moment, consider switching to linuxdeploy today!

User guides and examples

Please see the linuxdeploy user guide and the native binaries packaging guide in the AppImage documentation. There's also an examples section.

Projects using linuxdeploy

This is an incomplete list of projects using linuxdeploy. You might want to read their build scripts to see how they use linuxdeploy.

Plugins

linuxdeploy features a plugin system. Plugins are separate executables which implement a CLI-based plugin interface (specification).

There are two types of plugins: bundling and output plugins. Bundling plugins can be used to add resources to the AppDir. Output plugins turn the AppDir in actual bundles, e.g., AppImages.

linuxdeploy looks for plugins in the following places:

the directory containing the linuxdeploy binary
when using the AppImage: the directory containing the AppImage
the directories in the user's $PATH

You can use ./linuxdeploy*.AppImage --list-plugins to get a list of all the plugins linuxdeploy has detected on your system.

linuxdeploy currently ships with some plugins. These are likely out of date. In case of issues, please download the latest version, which will take precendence over the bundled plugin.

If you want to use a plugin to bundle additional resources, please add ./linuxdeploy*.AppImage --plugin <name> to your linuxdeploy command. Output plugins can be activated using ./linuxdeploy*.AppImage --output <name>.

A list of official and community plugins can be found in the awesome-linuxdeploy project.

Note: If you want to suggest a plugin for a specific framework, language etc., please feel free to create a new issue. Current plugin requests can be found here.

Troubleshooting

I bundled additional resources, but when I try to run them, either the system binary is called or the file is not found.

linuxdeploy does not change any environment variables such as $PATH. Your application must search for additional resources such as icon files or executables relative to the main binary.

Contact

The easiest way to get in touch with the developers is to join the IRC chatroom #AppImage on https://libera.chat. This is the preferred way for general feedback or questions how to use this application.

To report problems, please create an issue on GitHub.

Contributions welcome! Please feel free to fork this repository and send us a pull request. Even small changes, e.g., in this README, are highly appreciated!

linuxdeploy-plugin-conda's People

Contributors

Stargazers

Watchers

Forkers

probonopd l3dlp-sandbox saisasidhar thelastproject knarfs obitel-forks guallo aghamir sarnold nshiell szymonk211 hmaarrfk

linuxdeploy-plugin-conda's Issues

$HOME ends up hardcoded in shebangs

Using this script, and then deleting the original AppDir, I get when I try to run the AppImage:

me@host:~$ ./squashfs-root/AppRun # Without existing WINEPREFIX overlay
bash: ./squashfs-root/AppRun: /home/me/AppDir/usr/conda/bin/python: bad interpreter: No such file or directory

/home/me/AppDir/ was the location of the AppDir at creation time. It should not be hardcoded, especially not in shebangs.

pip requirements error

Dear @TheAssassin ,
Pip requirements.txt failed to be installed.

[conda/stderr] WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))': /simple/web3/
[conda/stderr] WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))': /simple/web3/
[conda/stderr] WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))': /simple/web3/
[conda/stderr] WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))': /simple/web3/
[conda/stderr] WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))': /simple/web3/
[conda/stderr] ERROR: Could not find a version that satisfies the requirement web3
[conda/stderr] ERROR: No matching distribution found for web3

Investigate pex

https://github.com/pantsbuild/pex
May or may not be useful.

Reference:
lineageos4microg/update_verifier#2

Good candidates for being packaged w/ linuxdeploy-plugin-conda

Good candidates for being packaged w/ linuxdeploy-plugin-conda:

Possible strategy for "thinning" AppImages

Here is one possible strategy for "thinning" AppImages so that they contain only the subset of Python and other files that is actually needed. Using this, I could reduce the size of TruFont from approximately 120 MB to just one third, under 40 MB.

mkdir squashfs-root
sudo mount -t tmpfs tmpfs squashfs-root/ -o strictatime,nodiratime 
./TruFont-*-x86_64.AppImage --appimage-extract
touch test
./squashfs-root/AppRun
# Kill
find squashfs-root/ -mindepth 3 -not -anewer test -not -path '*/share/icons/*' -not -path '*/share/applications/*' -not -path '*/share/metainfo/*' -delete || true
find squashfs-root/ -type d -empty -delete
./squashfs-root/AppRun 
appimagetool ./squashfs-root

The downside is that you need to manually "exercise" all parts of the application, so that the mechanism detects all files that are actually needed.

This application failed to start because it could not find or load the Qt platform plugin "xcb" in "".

We have something that begins to launch now, but:

$ Downloads/Lector-ff03947-x86_64.AppImage 
(...)
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Reinstalling the application may fix this problem.
Aborted
me@host:~$ LD_DEBUG=libs Downloads/Lector-ff03947-x86_64.AppImage 2<&1 | grep init: /usr
grep: /usr: Is a directory
me@host:~$ LD_DEBUG=libs Downloads/Lector-ff03947-x86_64.AppImage 2<&1 | grep "init: /usr"
     10760:	calling init: /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0
     10760:	calling init: /usr/lib/x86_64-linux-gnu/libX11.so.6
     10760:	calling init: /usr/lib/x86_64-linux-gnu/libGLX.so.0
     10760:	calling init: /usr/lib/x86_64-linux-gnu/libXext.so.6
     10760:	calling init: /usr/lib/x86_64-linux-gnu/libGL.so.1
me@host:~$ LD_DEBUG=libs Downloads/Lector-ff03947-x86_64.AppImage 2<&1 | grep "init: /lib"
     10774:	calling init: /lib/x86_64-linux-gnu/libpthread.so.0
     10774:	calling init: /lib/x86_64-linux-gnu/libc.so.6
     10774:	calling init: /lib/x86_64-linux-gnu/libz.so.1
     10774:	calling init: /lib/x86_64-linux-gnu/libdl.so.2
     10774:	calling init: /lib/x86_64-linux-gnu/libfuse.so.2
     10774:	calling init: /lib/x86_64-linux-gnu/libc.so.6
     10774:	calling init: /lib/x86_64-linux-gnu/libdl.so.2
     10774:	calling init: /lib/x86_64-linux-gnu/libtinfo.so.5
     10783:	calling init: /lib/x86_64-linux-gnu/libc.so.6
     10781:	calling init: /lib/x86_64-linux-gnu/libc.so.6
     10774:	calling init: /lib/x86_64-linux-gnu/libpthread.so.0
     10774:	calling init: /lib/x86_64-linux-gnu/libc.so.6
     10774:	calling init: /lib/x86_64-linux-gnu/libm.so.6
     10774:	calling init: /lib/x86_64-linux-gnu/librt.so.1
     10774:	calling init: /lib/x86_64-linux-gnu/libutil.so.1
     10774:	calling init: /lib/x86_64-linux-gnu/libdl.so.2

Not sure what is going on here... ping @TheAssassin any idea?

Use miniforge3 over miniconda because of commercial usage constraints

CC @TheAssassin

https://www.anaconda.com/blog/is-conda-free

As written above, Anaconda, Miniconda, and the main/anaconda channels are not free for commercial organizations with over 200 or more people. However, almost all other channels including conda-forge do not have this constraint.

However, conda-forge maintains its own installation of Miniconda, named Miniforge3. This contains mamba and conda-libmamba-solver as default, and is functionally equal except for only containing the conda-forge channel.

Located:
https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-ppc64le.sh

Runs the system-installed Python

If we bundle a script that uses !/usr/bin/env python, then we end up running the system-installed Python rather the bundled one. We need some sort of a wrapper to set up $PATH at least.

wget -c "https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh"
wget -c "https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage"
chmod +x linuxdeploy-x86_64.AppImage linuxdeploy-plugin-conda.sh

wget -c "https://raw.githubusercontent.com/probonopd/ptouch-770/patch-1/ptouch-770.desktop"
cp /usr/share/icons/gnome/256x256/devices/printmgr.png .
rm -rf ./_temp_home AppDir/ || true
mkdir -p AppDir/usr/bin
wget -c "https://raw.githubusercontent.com/probonopd/ptouch-770/patch-1/ptouch-770-gui" -O AppDir/usr/bin/ptouch-770-gui
chmod +x AppDir/usr/bin/*

export CONDA_CHANNELS=ostrokach CONDA_PACKAGES=gtk;pil;pyusb

./linuxdeploy-x86_64.AppImage --appdir AppDir -d ptouch-770.desktop  --plugin conda --output appimage -i printmgr.png 

./P-touch_P700_Labels-x86_64.AppImage

Running twice on the same AppDir gives an error

It is quite normal to have to run the deployment tool many times iteratively during development. However, currently it can error out if it is supposed to operate on a pre-existing AppDir:

me@host:~$ ./linuxdeploy-x86_64.AppImage --appdir AppDir --plugin conda -i $APPNAME.png -d $(readlink -f "$APPNAME.desktop") --custom-apprun AppRun --output appimage
linuxdeploy version 1-alpha (git commit ID 5c3a51f), Travis build 365 built on 2019-05-01 21:37:23 UTC

-- Creating basic AppDir structure -- 
Creating directory AppDir/usr/bin/ 
Creating directory AppDir/usr/lib/ 
Creating directory AppDir/usr/share/applications/ 
Creating directory AppDir/usr/share/icons/hicolor/ 
Creating directory AppDir/usr/share/icons/hicolor/16x16/apps/ 
Creating directory AppDir/usr/share/icons/hicolor/32x32/apps/ 
Creating directory AppDir/usr/share/icons/hicolor/64x64/apps/ 
Creating directory AppDir/usr/share/icons/hicolor/128x128/apps/ 
Creating directory AppDir/usr/share/icons/hicolor/256x256/apps/ 
Creating directory AppDir/usr/share/icons/hicolor/scalable/apps/ 

-- Deploying dependencies for existing files in AppDir -- 

-- Copying files into AppDir -- 

-- Running input plugin: conda -- 
Running process: /home/me/linuxdeploy-plugin-conda.sh --appdir AppDir
[conda/stdout] Error: directory exists: AppDir/usr/conda
ERROR: Failed to run plugin: conda (exit code: 1)

Don't remove header files when removing bloat

Don't remove the header files / the include directory when removing bloat. This way a application can be compiled against these header files.
Maybe add a separate switch (CONDA_SKIP_INCLUDE_CLEANUP) to skip the cleanup of the header files?

conda install shell

deleted

Need to symlink AppRun manually, -e does not work

When I manually symlink AppRun, then it works:

...
( cd AppDir ; ln -s usr/bin/FreeCAD AppRun )
...

./linuxdeploy-x86_64.AppImage --appdir AppDir -i AppDir/usr/conda/data/Mod/Start/StartPage/freecad.png -d AppDir/usr/share/applications/freecad.desktop --output appimage

./FreeCAD-x86_64.AppImage # Works!

When I do not manually symlink AppRun but instead specify -b ... it fails:

./linuxdeploy-x86_64.AppImage --appdir AppDir -i AppDir/usr/conda/data/Mod/Start/StartPage/freecad.png -d AppDir/usr/share/applications/freecad.desktop -e AppDir/usr/bin/FreeCAD --output appimage

# FAILS WITH
# -- Deploying executables -- 
# Deploying executable AppDir/usr/bin/FreeCAD 
# Deploying copyright files for file AppDir/usr/bin/FreeCAD 
# Deploying dependencies for ELF file AppDir/usr/bin/FreeCAD 
# ERROR: Could not find dependency: libFreeCADGui.so 
# Failed to deploy executable: AppDir/usr/bin/FreeCAD

Automatically set and use $VERSION

Automatically set and use $VERSION.

If the main application comes from Conda, use that version
If the main application comes from PyPI (pip), use that version
If the main application comes from git, use git rev-parse --short HEAD

TruFont AppImage (Qt based) fails to run due to missing library

#! /bin/bash

sudo apt -y install git

export VERSION=$(wget -q "https://api.github.com/repos/trufont/trufont/commits?sha=master" -O - | grep sha | head -n 1 | cut -d '"' -f 4 | head -c 7
)

export APPNAME=trufont
export PIP_REQUIREMENTS="-e git+https://github.com/trufont/trufont#egg=$APPNAME fs"
# export CONDA_PACKAGES="$APPNAME" # Only use this if the app is in a Conda channel (e.g., conda-forge)

wget -c "https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage"
wget -c "https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh"
chmod +x linuxdeploy-x86_64.AppImage linuxdeploy-plugin-conda.sh

rm -r AppDir || true

cat > $APPNAME.desktop <<EOF
[Desktop Entry]
Version=1.0
Name=TruFont
Exec=trufont %U
Terminal=false
Type=Application
Icon=$APPNAME
Categories=Graphics;
EOF

cat > AppRun <<\EOF
#!/bin/bash
HERE="$(dirname "$(readlink -f "${0}")")"
exec "$HERE/usr/conda/bin/python" "$HERE/usr/conda/bin/trufont" "$@"
EOF
chmod +x AppRun

wget -c "https://github.com/trufont/trufont/raw/master/Lib/trufont/resources/app.png"
convert app.png -resize 512x512 $APPNAME.png

./linuxdeploy-x86_64.AppImage --appdir AppDir --plugin conda -i $APPNAME.png -d $(readlink -f "$APPNAME.desktop") --custom-apprun AppRun --output appimage

results in

me@host:~$ ./TruFont-d1ef225-x86_64.AppImage 
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

Aborted

on Xubuntu 18.04.4 LTS.

Fails to run on CentOS 6 because of too old coreutils

Script fails on Centos 6 because of a too old ln command without the -r argument.

Output:

[conda/stderr] ln: invalid option -- 'r'
[conda/stderr] Try `ln --help' for more information.

Activating Conda Environment

This one is similar to #25 and #12 but slightly different, I use the plugin to pack local files into a AppImage with Conda runtime:

I download and install the packages I need
I copy my Python files into AppDir/
I create a start script
I run ./linuxdeploy-x86_64.AppImage

Here is the script for creating the AppImage and here the start script. It is kind of hacky.

It actually works, but only as long as I don't delete AppDir/ (line 44). It uses the "build" environment for execution, as soon as it is gone, the application doesn't work.

I guess the problem is, that Conda uses a lot of hard-coded, absolute paths. Is there a way around this? Meaning being able to activate the Conda environment when launching the AppImage?

Files do not end up in usr/conda/share/...

Trying to make an AppImage for GPodder with

#! /bin/bash

sudo apt -y install git pkg-config libcairo2-dev gobject-introspection libgirepository1.0-dev libdbus-1-dev

export VERSION=$(wget -q "https://api.github.com/repos/gpodder/gpodder/commits?sha=master" -O - | grep sha | head -n 1 | cut -d '"' -f 4 | head -c 7
)

export APPNAME=gpodder
export PIP_REQUIREMENTS="eyeD3 html5lib webencodings certifi six dbus-python pycairo pygobject mygpoclient podcastparser -e git+https://github.com/gpodder/gpodder#egg=$APPNAME"
# export CONDA_CHANNELS="conda-forge"
# export CONDA_PACKAGES=";;"  # Only use this if the package is in a Conda channel (e.g., conda-forge); can also be used for dependencies if the main application has no depends.txt

wget -c "https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage"
wget -c "https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh"
chmod +x linuxdeploy-x86_64.AppImage linuxdeploy-plugin-conda.sh

rm -r AppDir || true

wget -c "https://github.com/gpodder/gpodder/raw/master/share/applications/gpodder.desktop.in"
sed -i -e 's|^_||g' *.in # FIXME
sed -i -e 's|__PREFIX__/bin/||g' *.in # FIXME
mv *.in gpodder.desktop

cat > AppRun <<\EOF
#!/bin/bash
HERE="$(dirname "$(readlink -f "${0}")")"
exec "$HERE/usr/conda/bin/python" "$HERE/usr/conda/bin/gpodder" "$@"
EOF
chmod +x AppRun

wget -c "https://github.com/gpodder/gpodder/raw/master/share/icons/hicolor/64x64/apps/gpodder.png" -O $APPNAME.png
# convert app.png -resize 512x512 $APPNAME.png

./linuxdeploy-x86_64.AppImage --appdir AppDir --plugin conda -i $APPNAME.png -d $(readlink -f "$APPNAME.desktop") --custom-apprun AppRun --output appimage

Getting

1547894800.128399 [gpodder.gtkui.main] ERROR: Cannot find gtk/menus.ui in ['/home/me/squashfs-root/usr/conda/share/gpodder/ui/gtk', '/home/me/squashfs-root/usr/conda/share/gpodder/ui'], exiting

And indeed usr/conda/share/gpodder is missing entirely.

Output AppImage prefers ~/.local for pip packages

This one confused me for a long time: Why was my Pext 0.20 AppImage stating it was Pext-0.19+18.g11a6dea?

Well, I found it when I looked at the logs:

file:///home/sylvia/.local/lib/python3.6/site-packages/pext/qml/AboutDialog.qml:87:17: QML Text: Binding loop detected for property "width"

After doing a "pip3 uninstall pext", it started working fine.

The output AppImages probably need some $PATH changes to prefer bundled libraries over the system ones to prevent really weird issues.

ResolvePackageNotFound: conda==23.5.2

Previously the script I wrote worked:
https://github.com/nshiell/database-dossier/blob/master/tools/app-image-create.sh

I ran it today and saw:

[conda/stderr] ==> WARNING: A newer version of conda exists. <==
[conda/stderr]   current version: 23.5.2
[conda/stderr]   latest version: 23.7.4
[conda/stderr]
[conda/stderr] Please update conda by running
[conda/stderr]
[conda/stderr]     $ conda update -n base -c defaults conda
[conda/stderr]
[conda/stderr] Or to minimize the number of packages updated during conda update use
[conda/stderr]
[conda/stderr]      conda install conda=23.7.4

...

[conda/stderr] ResolvePackageNotFound:
[conda/stderr]   - conda==23.5.2

Has something changed somewhere that could have caused my issue?

Give an example for an application from the Python Package Index

The README.md currently shows an example of how to use linuxdeploy-plugin-conda for an application that is "in" Conda, using export CONDA_CHANNELS=freecad CONDA_PACKAGES=freecad.

However, not all applications are "in" Conda. I think they can still be installed using pip if they are "in" the Python Package Index (PyPI). Python should be taken from (Mini)conda and the application from PyPI in this case.

Can you please give an example of how to use linuxdeploy-plugin-conda to produce an AppImage of, say, https://pypi.org/project/galacteek/?

Reference:
pinnaculum/galacteek#1 (comment)

ModuleNotFoundError when using linuxdeploy-plugin-conda

Not sure if this is a linuxdeploy-plugin-conda bug or a bug in the TruFont Python packaging.

Here is my script:

#! /bin/bash

export APPNAME=trufont
export PIP_REQUIREMENTS="$APPNAME"
# export CONDA_PACKAGES="$APPNAME" # Only use this if the app is in a Conda channel (e.g., conda-forge)

wget -c "https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage"
wget -c "https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh"
chmod +x linuxdeploy-x86_64.AppImage linuxdeploy-plugin-conda.sh

rm -r AppDir || true

cat > $APPNAME.desktop <<EOF
[Desktop Entry]
Version=1.0
Name=TruFont
Exec=trufont %U
Terminal=false
Type=Application
Icon=$APPNAME
Categories=Graphics;
EOF

touch "$APPNAME.svg"
./linuxdeploy-x86_64.AppImage --appdir AppDir --plugin conda -i $(readlink -f "$APPNAME.svg") -d $(readlink -f "$APPNAME.desktop") --output appimage

An AppImage gets generated, but it does not run:

me@host:~$ ./TruFont-x86_64.AppImage 
Traceback (most recent call last):
  File "/tmp/.mount_TruFonZMkCwr/AppRun", line 7, in <module>
    from trufont.__main__ import main
  File "/home/me/AppDir/usr/conda/lib/python3.7/site-packages/trufont/__main__.py", line 1, in <module>
    from defconQt import representationFactories as baseRepresentationFactories
  File "/home/me/AppDir/usr/conda/lib/python3.7/site-packages/defconQt/representationFactories/__init__.py", line 3, in <module>
    from defcon import Glyph, Image, registerRepresentationFactory
  File "/home/me/AppDir/usr/conda/lib/python3.7/site-packages/defcon/__init__.py", line 10, in <module>
    from defcon.objects.font import Font
  File "/home/me/AppDir/usr/conda/lib/python3.7/site-packages/defcon/objects/font.py", line 7, in <module>
    from fontTools.ufoLib import UFOReader, UFOWriter, UFOLibError, UFOFileStructure
  File "/home/me/AppDir/usr/conda/lib/python3.7/site-packages/fontTools/ufoLib/__init__.py", line 9, in <module>
    import fs
ModuleNotFoundError: No module named 'fs'

Repeated runs should not download Conda each time

Repeated runs should not download Conda each time. Especially during development of a "recipe"...

There should be a download stage and a separate stage where the downloaded stuff goes into the AppDir. On a re-run, only the AppDir shall be deleted but not the downloaded stuff. Similar to pkg2appImage...

Fail to run in non-interactive shell

Hello,

Since changes (905ae7c and cf67896) our CI build fails with the following error:

-- Running input plugin: conda -- 
Running process: /home/user/jenkins/workspace/Project/build/linuxdeploy-plugin-conda.sh --appdir /home/user/jenkins/workspace/Project/build/ProjectAppDir
[conda/stderr] tput: No value for $TERM and no -T specified
ERROR: Failed to run plugin: conda (exit code: 2)

I rolled back to reviewion cbab82c and our builds are succeeding again.

After some futher investigating it seems all of the changes in those commits refer to commands that are fine in non-interactive shells (I tried to reproduce with flock and wget etc). But the one difference is the call to log() which does use tput - see here

I found that it appears to be due to running with a non-interactive shell. This bug reports a similar issue
https://bugzilla.redhat.com/show_bug.cgi?id=804604

Perhaps the suggested fix will work in in the log function too?

Numpy not installing

I'm trying to install numpy, but it's not working for me
The appimage part of my creator script:

export CONDA_CHANNELS='conda-forge'
export PIP_REQUIREMENTS='numpy'

# Deploy
./linuxdeploy-x86_64.AppImage \
   --appdir AppDir \
    -i ./xxx.png \
    -d ./xxx.desktop \
    --plugin conda \
    --custom-apprun ./buildset/src/AppRun.sh \
    --output appimage`

My AppRun.sh looks like:

APPDIR=`dirname $0`
export PATH="$PATH":"${APPDIR}"/usr/bin
${APPDIR}/usr/bin/python3 ${APPDIR}/opt/xxx.py $@

And xxx.py looks like:

import numpy

When I run it I see:

  File "/tmp/.mount_Blue_LxKM3XP/usr/conda/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/tmp/.mount_Blue_LxKM3XP/usr/conda/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/tmp/.mount_Blue_LxKM3XP/usr/conda/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libgfortran-2e0d59d6.so.5.0.0: ELF load command address/offset not properly aligned

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/.mount_Blue_LxKM3XP/opt/xxx/xxx.py", line 5, in <module>
    import numpy
  File "/tmp/.mount_Blue_LxKM3XP/usr/conda/lib/python3.8/site-packages/numpy/__init__.py", line 145, in <module>
    from . import core
  File "/tmp/.mount_Blue_LxKM3XP/usr/conda/lib/python3.8/site-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/tmp/.mount_Blue_LxKM3XP/usr/bin/python3"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libgfortran-2e0d59d6.so.5.0.0: ELF load command address/offset not properly aligned

I have been able to install other pip projects, but not numpy

Make plugin idempotent

Repeated runs fail with [conda/stderr] mkdir: cannot create directory ‘_temp_home’: File exists . How do you develop this interactively (trial and error during development of an AppImage)?

Similar,

Running process: /home/me/linuxdeploy-plugin-conda.sh --appdir AppDir
[conda/stdout] Error: directory exists: AppDir/usr/conda
ERROR: Failed to run plugin: conda (exit code: 1)

cleanup breaks .so libraries

in plugin script we have line:

find -type f -iname '*.so*' -print -exec strip '{}' \;

and this line causes error in opencv-python-headless library from pip:

ImportError: /home/greensort/apples-greensort/appimage/AppDirCp/usr/conda/lib/python3.6/site-packages/cv2/cv2.cpython-36m-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned

I can obviously work around this by commenting out this line from script.

Env var for using specific Python version

It'd be great to have something like CONDA_PYTHON_VERSION or so to be able to explicitly request for example 3.6 instead of 3.7. In my case, one of my dependencies doesn't build on 3.7 yet, and conda upgraded from 3.6 to 3.7, so now my build is broken with seemingly nothing I can do.

Support Python 2

See #3 (comment) for more information.

system packages hide appimages one

Let say that we generate a conda image with numpy packaged inside, e.g. as:

export CONDA_CHANNEL=anaconda
export CONDA_PACKAGES=numpy
./linuxdeploy-x86_64.AppImage --appdir AppDir --plugin conda -i numpy.png -d numpy.desktop --output appimage

Then if there is a system install of numpy, in /usr/local/lib/python3.7/site-packages/ it will be loaded instead of the AppImage one. You can check this by running:

sudo pip3.7 install numpy
./numpy-x86_64.AppImage -c "import numpy; print(numpy)"
sudo pip3.7 uninstall numpy
./numpy-x86_64.AppImage -c "import numpy; print(numpy)"

The reason is that at its init Python thinks it is running from a system install and as so it adds this fixed location to is search path before the AppImage site packages. You can see this by running:

./numpy-x86_64.AppImage -c "import sys; print(sys.path)"

The last item is the AppImage site packages. It is preceded by the fixed /usr/local/lib/python3.7/site-packages/ search path.

Possible solution: remove this /usr/local/lib/python3.7/site-packages/ at Python's init by adding a sitecustomize.py file to the AppImage site packages. For example as here. Note that it might happen that someone explicitly requests this location by adding it to PYTHONPATH. In this case it should not be removed from the search path.

Give an example for an application that uses setup.py

Please give an example for an application that uses setup.py such as this one:

https://github.com/BasioMeusPuga/Lector

command 'install' has no such option 'single_version_externally_managed'

Getting this when trying to run on Travis CI. Seemingly not getting this on a local Xubuntu 18.04.
Could it be that something from the host's Python is still being used?

[conda/stderr] distutils.errors.DistutilsOptionError: error in setup.cfg: command 'install' has no such option 'single_version_externally_managed'
ERROR: Failed to run plugin: conda (exit code: 2)

Build log: https://travis-ci.com/probonopd/Lector/builds/96218255#L2218

"$APPDIR"/usr/conda//lib/python3.7/site-packages does NOT support .pth files

#! /bin/bash

sudo apt -y install git imagemagick

export VERSION=$(wget -q "https://api.github.com/repos/trufont/trufont/commits?sha=master" -O - | grep sha | head -n 1 | cut -d '"' -f 4 | head -c 7
)

export APPNAME=trufont
export PIP_REQUIREMENTS="-e git+https://github.com/trufont/trufont@wx#egg=$APPNAME fs"
export CONDA_PACKAGES="wxpython " # Only use this if the app is in a Conda channel (e.g., conda-forge)

wget -c "https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage"
wget -c "https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh"
chmod +x linuxdeploy-x86_64.AppImage linuxdeploy-plugin-conda.sh

rm -r AppDir || true

cat > $APPNAME.desktop <<EOF
[Desktop Entry]
Version=1.0
Name=TruFont
Exec=trufont %U
Terminal=false
Type=Application
Icon=$APPNAME
Categories=Graphics;
EOF

cat > AppRun <<\EOF
#!/bin/bash
HERE="$(dirname "$(readlink -f "${0}")")"
exec "$HERE/usr/conda/bin/python" "$HERE/usr/conda/bin/trufont" "$@"
EOF
chmod +x AppRun

wget -c "https://github.com/trufont/trufont/raw/master/Lib/trufont/resources/app.png"
convert app.png -resize 512x512 $APPNAME.png

./linuxdeploy-x86_64.AppImage --appdir AppDir --plugin conda -i $APPNAME.png -d $(readlink -f "$APPNAME.desktop") --custom-apprun AppRun --output appimage