juliahep / root.jl Goto Github PK

View Code? Open in Web Editor NEW

39.0 10.0 9.0 176 KB

CERN ROOT compatiblility for Julia

License: Other

Julia 100.00%

julia file-io root-cern

root.jl's People

Contributors

Stargazers

Watchers

Forkers

cshimmin tkelman oschulz gipert simonschnake shamrocklee standardgalactic meslubi2021

root.jl's Issues

stop recompiling the julia binary

On the long term, we need to get rid of recompiling the julia binary with ROOT enabled. Some discussion of this is here: #4

Chars are not invariant

Creating a dataframe with Chars is buggy

df = DataFrame(x=Char['a', 'b'])
writetree("test.root", df)
tdf TreeDataFrame("test.root")

since ROOT converts 'a'::Char -> ['a', '0']::Vector{Int8}.

ROOT6 support

Currently, ROOT6 is not supported. Compilation succeeds but simple calls result in a segfault. Probably a result of mixing different LLVM versions between julia and ROOT6.

sanitize the API

Would be good to

agree on standard solutions for wrapping ROOT classes
write julian interfaces on top of the ROOT classes.

Planning here: http://github.com/jpata/API
in particular https://github.com/jpata/API/blob/master/ROOT.md

Currently we compile the root-enable julia binary in the julia source tree. That makes it impossible to install on systems where you have a central binary julia installation, as well as do travis testing.

Investigate how we could best compile the rjulia exe in the ROOT.jl directory. The issues there are:

propagation of JULIA_HOME so that the julia binary is found for things like @spawn etc
making sure rjulia always finds sysimg.so

This would also be necessary for travis testing #6.

travis testing

Currently, travis testing for this package doesn't work, because it requires a recompiled julia binary (see #4).

calling ROOT via Cxx.jl

I was able to get ROOT.jl (on https://github.com/jpata/ROOT.jl/tree/cxx) with ROOT 6 to work directly through Cxx.jl under Mac OS X without re-compiling the julia binary to load-and-hide ROOT before the julia LLVM is called.

This seems to be possible due to LLVM >=3.6 using versioned symbols as described in https://bugs.freedesktop.org/show_bug.cgi?id=93103. However, ROOT still uses an old LLVM with un-versioned symbols.

julia> versioninfo()
Julia Version 0.5.0-dev+5503
Commit 93ee8dd* (2016-07-19 06:48 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin15.4.0)
  CPU: Intel(R) Core(TM) i5-4278U CPU @ 2.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

$ root-config --version
6.06/02

Enabling RTTI in Cxx.jl: jpata/Cxx.jl#2, JuliaInterop/Cxx.jl#104 (on top of JuliaInterop/Cxx.jl@f6bc213)

ROOT prints the usual error but runs successfully:

$ ~/Documents/julia/julia ~/Documents/.julia/v0.5/ROOT/test/rootcxx.jl
Error in <UnknownClass::InitInterpreter()>: LLVM SYMBOLS ARE EXPOSED TO CLING! This will cause problems; please hide them or dlopen() them after the call to TROOT::InitInterpreter()!

Trying the same under linux (via https://github.com/jpata/docker-julia-root-cc7/blob/master/Dockerfile) fails with a segfault

(gdb) bt
#0  0x00007ffff6d145f7 in raise () from /lib64/libc.so.6
#1  0x00007ffff6d15ce8 in abort () from /lib64/libc.so.6
#2  0x00007ffff6d54327 in __libc_message () from /lib64/libc.so.6
#3  0x00007ffff6d5a194 in malloc_printerr () from /lib64/libc.so.6
#4  0x00007ffddd0081e3 in clang::LangOptions::~LangOptions() () from /root/.julia/v0.5/Cxx/src/../deps/usr/lib/libcxxffi-debug.so
#5  0x00007ffdde068152 in clang::PrintingPolicy::~PrintingPolicy (
    this=0x7ffddaa809a0 <TCling::InspectMembers(TMemberInspector&, void const*, TClass const*, bool)::printPol>, __in_chrg=<optimized out>)
    at /opt/julia/usr/bin/../../usr/include/clang/AST/PrettyPrinter.h:35
#6  0x00007ffff6d17e69 in __run_exit_handlers () from /lib64/libc.so.6
#7  0x00007ffff6d17eb5 in exit () from /lib64/libc.so.6
#8  0x00007ffff6d00b1c in __libc_start_main () from /lib64/libc.so.6
#9  0x00000000004012e9 in _start ()

Here's the docker image to reproduce that

jpata/julia               v0.5_root_cxx       f7e4de6be62a        About an hour ago   5.989 GB

ROOT looks for the following LLVM symbol to detect a clash: which is named a bit differently under OSX

$ nm -a /Users/joosep/Documents/julia/usr/lib/libLLVM-3.7.dylib | grep Pretty
0000000000cb8c10 T _LLVMEnablePrettyStackTrace

and linux

$ nm -a /opt/julia/usr/lib/libLLVM-3.7.so | grep Pretty
00000000004a6b40 T LLVMEnablePrettyStackTrace

The long-term fix would renaming the LLVM symbols when building it for julia: JuliaLang/julia#12644

@oschulz @jpivarski you may be interested in this

Use BinaryBuilder & BinaryProvider

After #15 is fixed, would it make sense to build ROOT with BinaryBuilder.jl and install the dependency here with BinaryProvider.jl? I'll probably be able to have a look at this if you're interested

Info about upcoming removal of packages in the General registry

As described in https://discourse.julialang.org/t/ann-plans-for-removing-packages-that-do-not-yet-support-1-0-from-the-general-registry/ we are planning on removing packages that do not support 1.0 from the General registry. This package has been detected to not support 1.0 and is thus slated to be removed. The removal of packages from the registry will happen approximately a month after this issue is open.

To transition to the new Pkg system using Project.toml, see https://github.com/JuliaRegistries/Registrator.jl#transitioning-from-require-to-projecttoml.
To then tag a new version of the package, see https://github.com/JuliaRegistries/Registrator.jl#via-the-github-app.

If you believe this package has erroneously been detected as not supporting 1.0 or have any other questions, don't hesitate to discuss it here or in the thread linked at the top of this post.

Segfault with Root and Julia

Hi - I'm trying to use Root and Julia with your Root.jl package. I followed the instructions for installing and am using the special build of Julia. If I run your example in the README.md, things work until I do @cxx canvas->SaveAs(pointer("myhist.png")) and then Julia segfaults. The important part of the stack trace seems to be...

[vagrant@localhost ~]$ rjulia run_root.jl

signal (11): Segmentation fault
while loading /home/vagrant/run_root.jl, in expression starting on line 11
_ZN4llvm19SmallPtrSetImplBase9erase_impEPKv at /Users/lyon/Development/julia/julia-centos6/julia/usr/lib/libLLVM-3.9.so (unknown line)
_ZN5clang4Sema30UpdateMarkingForLValueToRValueEPNS_4ExprE at /Users/lyon/Development/julia/julia-centos6/pkgs/v0.6/Cxx/src/../deps/usr/lib/libcxxffi.so (unknown line)
_ZN5clang4Sema23DefaultLvalueConversionEPNS_4ExprE at /Users/lyon/Development/julia/julia-centos6/pkgs/v0.6/Cxx/src/../deps/usr/lib/libcxxffi.so (unknown line)
_ZN5clang4Sema36DefaultFunctionArrayLvalueConversionEPNS_4ExprEb at /Users/lyon/Development/julia/julia-centos6/pkgs/v0.6/Cxx/src/../deps/usr/lib/libcxxffi.so (unknown line)
_ZN5clang4Sema31PerformMemberExprBaseConversionEPNS_4ExprEb at /Users/lyon/Development/julia/julia-centos6/pkgs/v0.6/Cxx/src/../deps/usr/lib/libcxxffi.so (unknown line)
unknown function (ip: 0x7feb60fe7c4e)
_ZN5clang4Sema24BuildMemberReferenceExprEPNS_4ExprENS_8QualTypeENS_14SourceLocationEbRNS_12CXXScopeSpecES4_PNS_9NamedDeclERKNS_19DeclarationNameInfoEPKNS_24TemplateArgumentListInfoEPNS0_26ActOnMemberAccessExtraArgsE at /cvmfs/gm2.opensciencegrid.org/prod7/external/root/v6_06_04b/Linux64bit+2.6-2.12-e10-prof/lib/libCling.so (unknown line)
_ZN5clang4Sema21ActOnMemberAccessExprEPNS_5ScopeEPNS_4ExprENS_14SourceLocationENS_3tok9TokenKindERNS_12CXXScopeSpecES5_RNS_13UnqualifiedIdEPNS_4DeclEb at /cvmfs/gm2.opensciencegrid.org/prod7/external/root/v6_06_04b/Linux64bit+2.6-2.12-e10-prof/lib/libCling.so (unknown line)
# ... a lot more  ...

You'll notice that up until _ZN5clang4Sema31PerformMemberExprBaseConversionEPNS_4ExprEb, functions are being pulled from Root's libCling.so. At that function, it switches to libcxxffi.so from Julia. That function is defined in both libCling.so and libcxxffi.so, so my guess is that the wrong one is being chosen and that eventually leads to the segfault. I imagine that the version of Clang/LLVM in libcxxffi.so is quite a bit newer than what libCling.so can handle.

Any ideas here? I realize this is not an easy problem. My setup is also somewhat complicated...

I'm running centos 6.8 (in a VirtualBox virtual machine on my Mac).
The version of Root is 6.06/04 built with gcc 4.9.3 . This Root is in a special place and LD_LIBRARY_PATH is used to get the shared objects correct. I build Julia, Cxx.jl and Root.jl with this same compiler. My julia versioninfo (from the special executable) is,

julia> versioninfo()
Julia Version 0.6.1
Commit 0d7248e (2017-10-24 22:15 UTC)
Platform Info:
  OS: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

Thanks for any advice!

Make ROOT.jl run on julia 1.1

Hi,

thank you for the great repo and module!

I've been trying to run ROOT.jl on Julia-1.1.0 for a little bit right now and I'm trying to understand why it fails in the end.
All the build process works fine, but when I run the rjulia executable I get a nasty error. This is what valgrind tells me:

Conditional jump or move depends on uninitialised value(s)
==10023==    at 0x507B604: TInetAddress::TInetAddress() (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x51A0450: TUnixSystem::GetHostByName(char const*) (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x50CD1A1: TUUID::GetNodeIdentifier() (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x50CD438: TUUID::TUUID() (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x5092024: TProcessID::AddProcessID() (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x4FCFA38: TROOT::TROOT(char const*, char const*, void (**)()) (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x4FD07B3: ROOT::Internal::GetROOT1() (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x4FB809F: _GLOBAL__sub_I_TROOT.cxx (in /home/filo/root/lib/libCore.so.6.16.00)
==10023==    by 0x4010732: call_init (dl-init.c:72)
==10023==    by 0x4010732: _dl_init (dl-init.c:119)
==10023==    by 0x40010C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==10023==  Uninitialised value was created by a stack allocation
==10023==    at 0x50CD0F2: TUUID::GetNodeIdentifier() (in /home/filo/root/lib/libCore.so.6.16.00)

This is the Julia version I am using:

Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: AMD Ryzen 5 2600 Six-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, znver1)
Environment:
  JULIA_CXX_RTTI = 1

this happen as soon as I call the julia_init() function.
ROOT is being initialized before the new Julia executable, but this seems to be another issue.
I have no idea where to look at, I tried to change some things but nothing worked. I guess here might be the right place to ask for some help with this.

Thanks

Make ROOT.jl Julia v1 compatible

Also, inspired by https://github.com/scikit-hep/uproot/ , do you think it's possible to have Julia native ROOT io functionality?

edit: we have it now! https://github.com/tamasgal/UnROOT.jl

Add support for ROOT libraries compiled with -shared

You assume that, under OSX, ROOT libraries are compiled with -dynamiclib, and so have the .dylib extension, however I installed ROOT via Homebrew and they seems to be compiled with -shared:

julia> import ROOT
ERROR: LoadError: could not find ROOT library Core at "/usr/local/Cellar/root/6.10.06/lib/root/libCore.dylib"

I managed to make it work simply redefining SHEXT=".so" here. Maybe it would be nice to add a little bit of logic to look also for .so libraries :)