Pacman support about appimage-builder HOT 20 CLOSED

As requested on IRC, I have some recommendations on how you can achieve your goals. bash implementation follows.

pacman --dbpath tempdb/ is the new pacman as far as you're concerned -- you want to ignore all info about the currently installed host system (and pretend nothing is installed yet). Assume whenever you run pacman, you're running it with this option.

• init the db: pacman --dbpath tempdb/ -Sy
• get the entire dependency tree of the package you want to download:

  $ pacman --dbpath tempdb/ -Sp --print-format '%n' bash
  linux-api-headers
  tzdata
  iana-etc
  filesystem
  glibc
  gcc-libs
  ncurses
  readline
  bash
  

• create a filter to remove packages you don't want, reading from stdout of one program and emitting to stdin of another program, e.g. a bash function:

  appimage-builder-filter-pkgnames() {
      grep -v -f EXCLUDE_ME
  }
  

• download packages (from stdin) without dependencies: pacman --dbpath tempdb/ -Swdd -
• OR, print download urls for packages (from stdin) without dependencies, that you wish to download: pacman --dbpath tempdb/ -Spdd -

Connecting it together:

$ pacman --dbpath tempdb/ -Sy
$ printf '%s\n' "filesystem" "tzdata" > EXCLUDE_ME
$ pacman --dbpath tempdb/ -Sp --print-format '%n' bash | appimage-builder-filter-pkgnames | pacman --dbpath tempdb/ -Spdd -
file:///var/cache/pacman/pkg/linux-api-headers-5.8-1-any.pkg.tar.zst
file:///var/cache/pacman/pkg/iana-etc-20201012-1-any.pkg.tar.zst
file:///var/cache/pacman/pkg/glibc-2.32-5-x86_64.pkg.tar.zst
file:///var/cache/pacman/pkg/gcc-libs-10.2.0-3-x86_64.pkg.tar.zst
file:///var/cache/pacman/pkg/ncurses-6.2-1-x86_64.pkg.tar.zst
file:///var/cache/pacman/pkg/readline-8.0.004-1-x86_64.pkg.tar.zst
file:///var/cache/pacman/pkg/bash-5.0.018-2-x86_64.pkg.tar.zst

from appimage-builder.

Anyway, point is these are standard tar archives so you can use whatever you wish to extract them. Just prune the reserved root "dotfile" metadata as it's useless to you.

If you like you can use pacman to do the extraction: pacman --dbpath tempdb/ -r AppDir/ -Udd <package-files-minus-glibc> and again with AppDir/opt/libc for just the glibc package.

Using pacman would ensure install scripts get run, which may or may not matter (they will chroot into AppDir/ before executing). --noscriplet can suppress this.

from appimage-builder.

I would say

# download the packages, do not install ofc
sudo pacman -Sw bash

# get the path to the downloaded packages 
sudo pacman -Sp bash

# create your appdir
mkdir -p /tmp/appimagetest/AppDir
cd /tmp/appimagetest  # any directory

# copy the archive into the AppDir
# the file returned by -Sp has a file:// protocol
# we would like to remove that using sed
# and then copy all the files returned by sed into the AppDir
sudo pacman -Sp bash | sed 's,file://,,g' | xargs -I % cp % AppDir/.

# change directory into the AppDir
cd AppDIr

# extract all pacman packages using zstd decompression
tar -I zstd -xvf *.pkg.tar.zst

# remove the downloaded packages 
rm -rf *.pkg.tar.zst

Here, you would get all the required directories, like usr, etc, Optionally filter the unwanted deps as @eli-schwartz suggested

from appimage-builder.

Interesting, so the extraction is just fine as it is. We just need to make the links, testing ...

from appimage-builder.

It works! 🥳 🎉

Were the symlinks what we were missing, thanks a lot @srevinsaju @eli-schwartz !!

Next step setting the pacman options, which includes:

• arch
• servers
• complete the list of default excluded packages (required for X11 apps)

from appimage-builder.

At this point, we have downloaded only the required packages with out much fuss, and that's great.
The next step will be extracting those files into AppDir and AppDir/opt/libc, any suggestions ?

from appimage-builder.

Extract the archive using /usr/bin/bsdtar or libarchive.so or your choice of tar format handler (should support at least zstd and xz compression) directly into your desired filesystem root or emulated root. Exclude any files in the top-level directory that begin with a dot, e.g. .PKGINFO or .CHANGELOG, as those are reserved for pacman metadata.

If you need to choose some files to extract to different directories, you'll need to figure out which ones yourself, pacman doesn't distinguish between types of packaged files... not really sure what your goal is there.

from appimage-builder.

Thanks for the input,

If you need to choose some files to extract to different directories

All the files from a package will be deployed using the same root, so this is not an issue.

not really sure what your goal is there.

Once I have all the packages that will end in the AppDir, I need to extract the glib and libstdcc++ into an specific location (AppDir/opt/libc). The goal is to be able to easily include or exclude then at runtime, this allows using the final bundle systems with a newer or an older glibc.

The rest of the packages will be extracted to

from appimage-builder.

So extract most of them to AppDir/ but extract the "glibc" package to AppDir/opt/libc/ I guess?

from appimage-builder.

That's correct.

from appimage-builder.

Awesome, I was making my way with bsdtar but I guess that using pacman would be better as it will use the right decompression tool and will opt-out the control files as required.

Regarding the scripts, as we are not creating a full chroot the scripts will fail to run, also I would like to add support for cross-architecture building (as it's already supported using apt).

from appimage-builder.

I'm trying to extract packages one at the time using the command:
pacman --dbpath /tmp/pacman-venv/db --cachedir /tmp/pacman-venv/pkg --root /tmp/AppDir/opt/glibc --upgrade --noconfirm --noscriptlet --nodeps /tmp/pacman-venv/pkg/glibc-2.32-5-x86_64.pkg.tar.zst

But it's giving me the following error:

error: failed to initialize alpm library
(could not find or read directory: /tmp/pacman-venv/db)

Notice that /tmp/pacman-venv/db do exists and is reachable.

Fixed, the target dir didn't exists.

from appimage-builder.

So far I've followed the @srevinsaju snippet, but when the glibc package is extracted using tar (or bsdtar) some files are not created, to be specific, /lib64/ld-linux-x86-64.so.2 is not created which is critical for the bundle to work.

Using pacman -Udd --root to extract a package fails with the error above ^.

Why the Archlinux glibc package install ld-linux-x86-64.so.2 in two different places ? This makes a bit hard to choose the right one.

What need to be solved, extract packages including all it's files including (hard links and soft links)

from appimage-builder.

@azubieta I guess it fails with bsdtar, because, /lib64/ld*.so is a symlink to the original /use/lib/ld*.so

I guess adding -c to the tar command should create the symlink, rather than not creating it.

from appimage-builder.

tar -I zstd -cxvf ​*​.pkg.tar.zst

I have not tried this yet, afk

from appimage-builder.

@srevinsaju the -c command didn't play well along with the -x. At least that's what the output says:

tar: You may not specify more than one '-Acdtrux', '--delete' or  '--test-label' option
Try 'tar --help' or 'tar --usage' for more information.

If I remove it the package is extracted but still the files in /lib64 are not being created. I assume that those files are provided by the glibc package because of the output of pacman -Qo /lib64/ld-linux-x86-64.so.2.

from appimage-builder.

/lib64 is a symlink to /usr/lib if its a helpful pointer

-x and -c cannot be used together, probably thats the reason

from appimage-builder.

% ls -al /
total 73
drwxr-xr-x  19 root root  4096 Nov 27 15:40 .
drwxr-xr-x  19 root root  4096 Nov 27 15:40 ..
lrwxrwxrwx   1 root root     7 Sep  3 01:30 bin -> usr/bin
drwxr-xr-x   4 root root  4096 Nov 27 15:40 boot
drwxr-xr-x  23 root root  3820 Nov 27 18:55 dev
drwxr-xr-x   3 root root   512 Jan  1  1970 efi
drwxr-xr-x 118 root root 12288 Nov 27 20:19 etc
drwxr-xr-x   5 root root  4096 Nov 27 17:26 home
lrwxrwxrwx   1 root root     7 Sep  3 01:30 lib -> usr/lib
lrwxrwxrwx   1 root root     7 Sep  3 01:30 lib64 -> usr/lib
drwx------   2 root root 16384 Sep 18 23:47 lost+found
drwxr-xr-x   4 root root  4096 Sep 26 20:25 media
drwxr-xr-x   5 root root  4096 Nov 27 15:54 mnt
drwxr-xr-x  10 root root  4096 Nov 27 15:41 opt
dr-xr-xr-x 376 root root     0 Nov 27 18:54 proc
drwxr-x---  20 root root  4096 Nov 27 18:53 root
drwxr-xr-x  34 root root   880 Nov 27 18:56 run
lrwxrwxrwx   1 root root     7 Sep  3 01:30 sbin -> usr/bin
drwxr-xr-x   4 root root  4096 Sep 19 00:01 srv
dr-xr-xr-x  13 root root     0 Nov 27 18:54 sys
drwxrwxrwt  19 root root   540 Nov 27 20:28 tmp
drwxr-xr-x  11 root root  4096 Nov 27 20:19 usr
drwxr-xr-x  14 root root  4096 Nov 27 15:51 var

from appimage-builder.

Yay!

Next step setting the pacman options, which includes:

• arch
• servers
• complete the list of default excluded packages (required for X11 apps)

For
mirrors, I would suggest https://mirrors.kernel.org/archlinux/$repo/os/$arch

Source: https://wiki.archlinux.org/index.php/mirrors

from appimage-builder.

Changes merged! Thanks everyone for helping! 👍

from appimage-builder.