Use patchelf to set RPATH about legacy-linuxbrew HOT 22 CLOSED

Ok forgot to mention that I also had to adjust ENV.rb to look for gcc and g++ instead of gcc-4.2 and g++-4.2. Seems to work nicely.

from legacy-linuxbrew.

@aktau The compiler name issue (gcc/g++ vs gcc-4.2/g++-4.2) is now fixed.

from legacy-linuxbrew.

@aktau Hi Nicolas,

Homebrew is usually installed in /usr/local, and on OS X /usr/local/lib is in the default library search path. When I install Homebrew in my home directory, I set LD_LIBRARY_PATH to $HOME/local/lib.

On Linux, /usr/local/lib is not in the default library search path. One simple solution is to add the Homebrew library path to the ld.so config file, /etc/ld.so.conf.

Cheers,
Shaun

from legacy-linuxbrew.

Actually, we could also just set it as a flag to gcc if and when that compiler is used, to obviate the need for patchelf:

LDFLAGS += -Wl,-rpath,/path/to/homebrew/lib

Actually I'm not entirely sure if the -Wl, is necessary when it's put in LDFLAGS...

from legacy-linuxbrew.

This issue affects the master branch of Homebrew just as well as Linuxbrew. Would you open an issue at http://github.com/mxcl/homebrew? It would be good to see it solved upstream.

from legacy-linuxbrew.

The problem discussed here is IMHO actually a bit more complex, and not yet resolved (despite the referenced issue #9 being marked as closed). Quite many libraries are keg-only, and hence dependent binaries must include ${HOMEBREW_PREFIX}/opt/<dep_pkg>/lib in their RPATH. Take for example the fontconfig package, which depends on the freetype package, which in turn depends on libpng. All of them are keg-only. Linking the programs in fontconfig (without any modifications) fails, because libpng16.so.16, required by libfreetype.so.6, cannot be found.

The hackish solution is to update every single formula to add -Wl,-rpath,... flags to LDFLAGS for every single dependency, which is frankly quite unmaintainable. However, I don't see how this could be handled automagically right now.

from legacy-linuxbrew.

The hackish solution is to update every single formula to add -Wl,-rpath,... flags to LDFLAGS for every single dependency, which is frankly quite unmaintainable.

Does linuxbrew use superenv? If so, it wouldn't be difficult to add to flags automatically based on the existing code there.

from legacy-linuxbrew.

Well, of course {home,linux}brew might just do it based on the listed dependencies. Would that be safe?

from legacy-linuxbrew.

@mistydemeo Linuxbrew does not currently use superenv. It would require a few patches to Library/ENV to get it going, I think.

@themiwi How does this work on Mac OS?

from legacy-linuxbrew.

@sjackman It's been a while I used Mac OS, but if I remember correctly, each dylib contains an embedded path (the so-called install_name, something like the SONAME on Linux, but usually it is absolute), and this is then used when linking. Hence, (almost) every binary references its libraries via an absolute path, and thus the issue does not exist. I think you can dump this info using otool -L <somefile> and can be modified using the install_name_tool.

from legacy-linuxbrew.

Just in case anyone was still wondering why changing the LD_LIBRARY_PATH is not necessarily the best solution on linux (this is one of the less subtle issues):

from legacy-linuxbrew.

@aktau Absolutely, I stumbled across this issue myself just recently (both with ViM and also with GTK+ programs no longer working properly because those got linked).

For linuxbrew it probably is almost always a bad idea to link /lib/ and to modify LD_LIBRARY_PATH. But this would require to extend the rpath accordingly.

from legacy-linuxbrew.

@aktau @themiwi Could you please test the branch bottle. It uses patchelf to set the RPATH. If the package glibc is installed, it also set the dynamic loader. Be sure to brew install patchelf. With this work, setting LD_LIBRARY_PATH should no longer be necessary.

I'm using this patch to implement portable bottles on Linux. I've tested building a bottle on Ubuntu 14.04 and running it on CentOS 5.10, and it works! I'm pretty happy with that result.

Cheers,
Shaun

from legacy-linuxbrew.

I'd appreciate your help if you could also test brew install glibc.

from legacy-linuxbrew.

I'd appreciate your help if you could also test brew install glibc.

Don't have any systems I can safely mess up at the moment.

I'm using this patch to implement portable bottles on Linux. I've tested building a bottle on Ubuntu 14.04 and running it on CentOS 5.10, and it works! I'm pretty happy with that result.

Pretty great, but it's still risky unless all libraries have been source from homebrew's own I guess. I like the rpath fix much more for not having to mess with my LD_LIBRARY_PATH. I have no problem compiling my own stuff (little need for bottles).

Could you details a few steps for how I would go about testing this? I'm not a homebrew/ruby expert by any stretch of the imagination.

from legacy-linuxbrew.

Hi, Nicolas. Checkout the bottle branch of Linuxbrew cd $HOME/.linuxbrew; git checkout bottle, install patchelf brew install patchelf, and from then on any executables installed with Homebrew will have their RPATH set to $HOMEBREW_PREFIX/lib. Here's the relevant commit:
34b4c36#diff-9f183506315342e7eb6e1ab973ea1996R66

from legacy-linuxbrew.

Apparently it didn't work:

➜  .linuxbrew git:(34b4c36) cd
➜  ~  brew install ag
==> Installing dependencies for the_silver_searcher: autoconf, automake, pkg-config, pcre, xz
==> Installing the_silver_searcher dependency: autoconf
==> Downloading http://ftpmirror.gnu.org/autoconf/autoconf-2.69.tar.gz
######################################################################## 100.0%
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/autoconf/2.69
==> make install
==> Caveats
This formula is keg-only, so it was not symlinked into /home/player/.linuxbrew.

Xcode provides this software prior to version 4.3.
==> Summary
/home/player/.linuxbrew/Cellar/autoconf/2.69: 66 files, 3.2M, built in 11 seconds
==> Installing the_silver_searcher dependency: automake
==> Downloading http://ftpmirror.gnu.org/automake/automake-1.14.1.tar.gz
######################################################################## 100.0%
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/automake/1.14.1
==> make install
==> Caveats
This formula is keg-only, so it was not symlinked into /home/player/.linuxbrew.

Xcode provides this software prior to version 4.3.
==> Summary
/home/player/.linuxbrew/Cellar/automake/1.14.1: 131 files, 3.3M, built in 11 seconds
==> Installing the_silver_searcher dependency: pkg-config
==> Downloading http://pkgconfig.freedesktop.org/releases/pkg-config-0.28.tar.gz
######################################################################## 100.0%
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/pkg-config/0.28 --disable-host-tool --with-internal-glib --with-pc-path=/home/player/.linuxbrew/lib/pkgconfig:/home/player/.linuxbrew/sha
==> make
==> make check
==> make install
/home/player/.linuxbrew/Cellar/pkg-config/0.28: 10 files, 644K, built in 96 seconds
==> Installing the_silver_searcher dependency: pcre
==> Downloading https://downloads.sourceforge.net/project/pcre/pcre/8.35/pcre-8.35.tar.bz2
######################################################################## 100.0%
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/pcre/8.35 --enable-utf8 --enable-pcre8 --enable-pcre16 --enable-pcre32 --enable-unicode-properties --enable-pcregrep-libz --enable-pcregr
==> make
==> make test
==> make install
/home/player/.linuxbrew/Cellar/pcre/8.35: 146 files, 5.5M, built in 2.2 minutes
==> Installing the_silver_searcher dependency: xz
==> Downloading http://fossies.org/linux/misc/xz-5.0.5.tar.gz
######################################################################## 100.0%
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/xz/5.0.5
==> make install
/home/player/.linuxbrew/Cellar/xz/5.0.5: 63 files, 1.7M, built in 30 seconds
==> Installing the_silver_searcher
==> Downloading https://github.com/ggreer/the_silver_searcher/archive/0.23.0.tar.gz
######################################################################## 100.0%
==> Downloading https://github.com/thomasf/the_silver_searcher/commit/867dff8631bc80d760268f653265e4d3caf44f16.diff
######################################################################## 100.0%
==> Patching
patching file ag.bashcomp.sh
==> aclocal -I /home/player/.linuxbrew/share/aclocal
==> autoconf
==> autoheader
==> automake --add-missing
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/the_silver_searcher/0.23.0
==> make
==> make install
==> Caveats
Bash completion has been installed to:
  /home/player/.linuxbrew/etc/bash_completion.d
==> Summary
/home/player/.linuxbrew/Cellar/the_silver_searcher/0.23.0: 8 files, 148K, built in 12 seconds
You have new mail.
➜  ~  which ag
/home/player/.linuxbrew/bin/ag
➜  ~  ag -v
ag: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory

This was on a fresh linuxbrew install. Not sure what I did wrong.

from legacy-linuxbrew.

Reported too soon. I had forgotten to install patchelf. New results: works perfectly!

➜  ~  brew install patchelf
==> Downloading https://github.com/NixOS/patchelf/archive/0.8.tar.gz
######################################################################## 100.0%
==> ./bootstrap.sh
==> ./configure --disable-silent-rules --prefix=/home/player/.linuxbrew/Cellar/patchelf/0.8
==> make install
/home/player/.linuxbrew/Cellar/patchelf/0.8: 6 files, 172K, built in 11 seconds
➜  ~  brew remove ag
Uninstalling /home/player/.linuxbrew/Cellar/the_silver_searcher/0.23.0...
➜  ~  brew install ag
==> Downloading https://github.com/ggreer/the_silver_searcher/archive/0.23.0.tar.gz
Already downloaded: /home/player/.cache/Homebrew/the_silver_searcher-0.23.0.tar.gz
==> Downloading https://github.com/thomasf/the_silver_searcher/commit/867dff8631bc80d760268f653265e4d3caf44f16.diff
Already downloaded: /home/player/.cache/Homebrew/the_silver_searcher--patch-09502c60a11658d9a08a6825e78defad96318bd9.diff
==> Patching
patching file ag.bashcomp.sh
==> aclocal -I /home/player/.linuxbrew/share/aclocal
==> autoconf
==> autoheader
==> automake --add-missing
==> ./configure --prefix=/home/player/.linuxbrew/Cellar/the_silver_searcher/0.23.0
==> make
==> make install
==> Caveats
Bash completion has been installed to:
  /home/player/.linuxbrew/etc/bash_completion.d
==> Summary
/home/player/.linuxbrew/Cellar/the_silver_searcher/0.23.0: 8 files, 152K, built in 8 seconds
➜  ~  ag -v
ERR: What do you want to search for?
➜  ~  which ag
/home/player/.linuxbrew/bin/ag

from legacy-linuxbrew.

@sjackman need any more input? It seems to work fine here.

About the only suggestion I'd make is to make it more obvious that patchelf needs to be installed for correct operation without altering LD_LIBRARY_PATH.

In fact, if it were up to me, I'd make patchelf mandatory. Linux is not OSX, and I firmly believe patchelf is a superior solution for Linux systems.

from legacy-linuxbrew.

I've create a new wiki page with instructions for a Standalone Installation of Linuxbrew. Would you be able to test it out?

from legacy-linuxbrew.

@aktau With the recent commits to bottle branch, patchelf is no longer necessary. Setting the RPATH and dynamic linker is handled using LDFLAGS -rpath and --dynamic-linker. The patchelf code remains as a backup plan for builds that ignore or reset LDFLAGS.

from legacy-linuxbrew.

Very nice :). It's good that there's a fallback to patchelf. Especially when/if there are toolchains that don't have the concept of LDFLAGS but do generate elf binaries that need to dynamically link with something.

from legacy-linuxbrew.