"Couldn't register scheme handler" errors for tool linking (likely incorrectly?) to htslib about htslib HOT 12 CLOSED

Well, that's CMake for ya 😄

No worries, I should have figured it out sooner but it's been a while since anyone was immersed in this code.

Your issue has prompted me to flesh out what a revised plugin mechanism that was immune to these issues would look like, and in fact it can be done while maintaining good compatibility with existing plugins and htslibs. So I'll turn that into a draft PR if the maintainers are interested.

from htslib.

Minutes after writing the issue, I found that the issue was resolved when dynamically linking to libhts.so rather than statically linking to libhts.a. Of course that cannot be a hard requirement, since as I noted above htsfile goes the static route, so I'm uncertain why it was necessary in this case -- perhaps some conflicting in my unshared build settings.

from htslib.

Does setting HTS_PATH help? Mentioned here.

from htslib.

I did have the HTS_PATH set; when it was unset I received different error messages stating that the plugins could not be loaded.

Based on my current workaround of using dynamic linking, a better title for this issue would be the requirements for statically linking to libhts and using its plugins.

from htslib.

The plugins are separately compiled, so you should be able to point HTS_PATH at the plugins in a standalone HTSlib directory or installation. Then you can just configure and compile your tool's htslib with --enable-plugins and not compile the actual plugins, simplifying your build. Of course, ideally your tool would not bundle and build HTSlib itself but would be able to build against a previously installed HTSlib.

The plugin machinery does not use LD_LIBRARY_PATH, so you shouldn't set that in an attempt to have it find the plugins. Setting LD_LIBRARY_PATH can confuse matters (e.g. which libhts.so.* is actually used at runtime), so ideally you would compile your tool so as to avoid needing to do that.

The “Couldn't register scheme handler” message appears fairly generic, but in fact it appears in one specific place in the code where sadly there is no errno or similar to report. So you'll need to bring out the big guns to debug this.

First off, what do ldd /path/to/yourtool and ldd /GLnexus/external/src/htslib/hfile_libcurl.so report?

Next, what does the following report (with hts_verbose still cranked up)? The output will be voluminous, so it would probably be best to attach or upload it rather than paste it into a comment.

LD_DEBUG=libs,files,symbols /path/to/yourtool s3://whatever/whatever.bam 2> loader.log

from htslib.

If there is no update on this by the end of the week I am going to close it.

from htslib.

Apologies for the long delay --

# ldd glnexus_cli 
        linux-vdso.so.1 (0x00007ffe8c371000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb6d7118000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb6d6ef9000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb6d6b5b000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb6d6943000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb6d6552000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb6d7ec4000)

libhts.so.3 not found for the plugin w/o LD_LIBRARY_PATH (since the library is bundled and not getting installed):

# ldd ./external/src/htslib/hfile_libcurl.so
        linux-vdso.so.1 (0x00007ffe27b9c000)
        libhts.so.3 => not found
        libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4 (0x00007f27f2bad000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f27f298e000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f27f259d000)
        libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f27f2378000)
        libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f27f215b000)
        librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f27f1f3f000)
        libpsl.so.5 => /usr/lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f27f1d31000)
        libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f27f1aa4000)
        libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f27f15d8000)
        libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f27f138d000)
        libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f27f113b000)
        liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f27f0f2d000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f27f0d10000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f27f3038000)
        libunistring.so.2 => /usr/lib/x86_64-linux-gnu/libunistring.so.2 (0x00007f27f0992000)
        libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f27f062c000)
        libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f27f03f6000)
        libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f27f01c0000)
        libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f27eff3f000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f27efd3b000)
        libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f27efa65000)
        libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f27ef833000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f27ef62f000)
        libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f27ef424000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f27ef20a000)
        libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f27eefef000)
        libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f27eedae000)
        libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f27eea7f000)
        libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f27ee86c000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f27ee668000)
        libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f27ee45f000)
        libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f27ee1d2000)
        libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f27edf30000)
        libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f27edcfa000)
        libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f27edae4000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f27ed8dc000)
        libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f27ed6b3000)
        libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f27ed4a4000)
        libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f27ed25a000)
        libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f27ecf51000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f27ecd19000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f27ec97b000)

Setting LD_LIBRARY_PATH, libhts.so is no longer "not found" of course.

I generated a loader.log file as instructed, attached here: loader.log

Thanks for your help, and apologies again for the delay.

from htslib.

What we learn from your loader.log is that when each plugin uses a function like hfile_add_scheme_handler(), it needs to look through all the loaded shared libraries to find it (see lines 147041–49) in the libhts.so.3 pulled in by the plugin. The plugin system expects it to be found in the same place as the load_hfile_plugins() and hopen() functions, namely ./glnexus_cli.

As noted in the issue title, the suspicion here is that the tool is likely being incorrectly linked — but you haven't shown us the link command that is producing the glnexus_cli executable. 😄 I suspect the problem is that you are not using -rdynamic.

At present, if enabling plugins and linking your executable against libhts.a statically, you need to use -rdynamic when linking your executable. This probably needs to be documented better.

It would be possible to revise the plugin mechanism so that the plugin initialisation function was explicitly provided with a table of pointers to plugin-setup functions etc as an argument rather than calling them by name and depending on the loader symbol resolution and so requiring only one copy of hfile.o (and the schemes table in particular) to exist or at least to be in use. (Then it wouldn't matter how many copies of hfile.o there were at load time, RTLD_GLOBAL defaults wouldn't matter, and you wouldn't need to use -rdynamic to ensure looking up symbols found the main executable's copy.) Now that the main plugins have bedded in very well, it's probably time to revise it in that way and sidestep this linking and loading complexity.

from htslib.

Indeed, adding rdynamic to the compilation of the executable resolves the issue -- I no longer need to set LD_LIBRARY_PATH and the plugins load successfully -- thanks!

And thanks for the insight despite me not providing a the link command. I didn't mean to be cryptic, this is a fork of an open source tool after all (https://github.com/jwimberl/GLnexus, forked from https://github.com/dnanexus-rnd/GLnexus), but the building and linking is handled by a lengthy CMake config that I hadn't fully digested, let alone stripped down a MRE. I thought including it might add more noise than actual helpful info. As it turns out, there was a change in CMake from version 3.3 to 3.4 in which it stopped adding rdynamic linking flags by default (https://cmake.org/cmake/help/latest/policy/CMP0065.html). The canonical way to resolve this in CMake is to add a line like

set_property(TARGET executable_name PROPERTY ENABLE_EXPORTS 1)

which, while it works, has the somewhat unfortunate downside of obfuscating the setting of the "rdynamic" flag by renaming the concept to ENABLE_EXPORTS.

from htslib.

I wouldn't have any objections to a more robust plugin mechanism, but @daviesrob is the arbiter.

from htslib.

I'd certainly be interested in an improved plug-in PR.

from htslib.

I guess we can close this now the original problem has been dealt with and #1611 merged. We'll remember the bit about the plugin API separately.

from htslib.