Run rootpy tests on travis-ci about rootpy HOT 25 CLOSED

@cdeil Travis CI looks amazing... Let me know how this goes. I'm looking at examples of how to manually install dependencies:

https://github.com/bergie/midgardmvc_core/blob/master/tests/travis_midgard2.sh

Hopefully it is possible to get a ROOT installation working there.

from rootpy.

I am testing how to install root / numpy / rootpy here:
https://github.com/cdeil/travis_test

@ndawe If you want to try something yourself, you should be able to push (or edit the files directly on github).

I'll try to get a ROOT installed with --enable-python either via the binaries or from source.

from rootpy.

Unfortunately the ROOT build times out on travis-ci after 15 minutes, even if I use --minimal --enable-python:
https://travis-ci.org/#!/cdeil/travis_test/builds/2810086

The build would take about 18 minutes. :-)

So now I'll try to install a binary ROOT version. I have a local virtual machine with 32 bit Ubuntu like travis-ci uses, if the CERN Scientific Linux binaries work, fine, otherwise I think I should be able to build ROOT myself in the VM, then create a tarball and download that from travis-ci?

from rootpy.

Ubuntu is not compatible with the ROOT Scientific Libraries:

deil@ubuntu:~/software/root/v5.34.02.Linux-slc5-gcc4.3$ root
Couldn't find font "-adobe-helvetica-medium-r-*-*-10-*-*-*-*-*-iso8859-1",
trying "fixed". Please fix your system so helvetica can be found, 
this font typically is in the rpm (or pkg equivalent) package 
XFree86-[75,100]dpi-fonts or fonts-xorg-[75,100]dpi.
  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version   5.34/02 21 September 2012   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************

ROOT 5.34/02 (tags/v5-34-02@46115, Sep 21 2012, 16:29:13 on linux)

CINT/ROOT C/C++ Interpreter version 5.18.00, July 2, 2010
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.
root [0] new TBrowser
dlopen error: libjpeg.so.62: cannot open shared object file: No such file or directory
Load Error: Failed to load Dynamic link library /home/deil/software/root/v5.34.02.Linux-slc5-gcc4.3/lib/libASImage.so
Error in <TCint::AutoLoad>: failure loading library libASImage.so for class TASImage
Error in <TPluginHandler::SetupCallEnv>: class TASImage not found in plugin ASImage
Error in <TGVSplitter::TGVSplitter>: splitterv.xpm not found
Error in <TGHSplitter::TGHSplitter>: splitterh.xpm not found
Error in <TGComboBox::TGComboBox>: arrow_down.xpm not found
Error in <TGVScrollBar::TGVScrollBar>: arrow_*.xpm not found

 *** Break *** segmentation violation
Root > 
root [1] .q
deil@ubuntu:~/software/root/v5.34.02.Linux-slc5-gcc4.3$ python -c 'import ROOT'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/deil/software/root/v5.34.02.Linux-slc5-gcc4.3/lib/ROOT.py", line 85, in <module>
    import libPyROOT as _root
ImportError: libcrypto.so.6: cannot open shared object file: No such file or directory

If you know how to distribute a ROOT I build on my machine to another machine (DESTDIR?), let me know, otherwise I'll just try some things.

from rootpy.

I really think building against a matrix of ROOT and Python on the public Travis CI is not possible. Since the build environments don't persist between builds, all ROOT versions we test against need to be downloaded and compiled every time. I just ran a minimal build in an Ubuntu 32bit VM and it took 5m to compile when giving the VM 8 cores and 4G of RAM. The Travis VMs are much more limited.

Maybe we should consider setting up a private travis instance without these build limitations, and one that doesn't wipe out our ROOT matrix between each build?

from rootpy.

I can get a minimal build in my host Ubuntu (not in the VM) in 3m56 but that's with a hyper-threaded quad-core i7-2600K CPU @ 3.40GHz, 16GB RAM, and an SSD...

from rootpy.

Even if we can manage to compile a binary for each ROOT version we want to test against that loads fine on Travis, each binary will be rather large. I see that find root -name "*.so" | xargs du -hc gives 78M. We would need to host these somewhere with enough bandwidth, and downloading these for each build on travis seems like a big waste. Too bad we can't cache these on Travis (or can we?). I suppose we would be expected to pay for that.

I have several spare machines lying around and could get a minimal always-on build machine running. That's an option.

from rootpy.

Sorry, clicked the wrong button...

from rootpy.

Travis has the philosophy to start with a clean VM on each build, so there's no way to pre-install something and have it available for future builds. But they do have a 10 MB / sec download speed, so fetching a 100 MB root binary for each build is not a problem and as far as I can tell allowed (I agree it is a waste).

Here's my first attempt at making such a 47 MB binary:
https://github.com/downloads/cdeil/travis_test/root_v5.34.02.Linux-Ubuntu32-gcc4.6.tar.gz

I used these commands in my Ubuntu 32-bit VM:

cd /home/deil/software/root/v5.34.02.source
./configure --minimal --enable-python --prefix=/home/deil/software/root/install/
make
make install
cd ..
tar zcvf root_v5.34.02.Linux-Ubuntu32-gcc4.6.tar.gz install

I'll now see if it works on travis-ci.

In the long run as you say we probably want a build matrix like Linux / Mac, different compilers, ROOT versions, python version, numpy versions and this is too much for travis-ci. I think the best solution would be a build server running Jenkins and several virtual machines, but that will take some learning / time to set up.

from rootpy.

https://github.com/downloads/cdeil/travis_test/root_v5.34.02.Linux-Ubuntu32-gcc4.6.tar.gz
fails like this (https://travis-ci.org/#!/cdeil/travis_test/L4291):

$ source $ROOTSYS/bin/thisroot.sh
root-config: command not found

The reason is that --prefix and make install results in hard-coded PATHs for the install DIR on the build machine:

grep deil thisroot.sh 
   PATH=/home/deil/software/root/install//bin; export PATH
   PATH=/home/deil/software/root/install//bin:$PATH; export PATH
   LD_LIBRARY_PATH=/home/deil/software/root/install//lib/root; export LD_LIBRARY_PATH       # Linux, ELF HP-UX
   LD_LIBRARY_PATH=/home/deil/software/root/install//lib/root:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH
...

I could of course install into the same absolute PATH on travis-ci as in my build VM, but apparently the correct method is to use make install DESTDIR=... ( http://autotoolset.sourceforge.net/tutorial.html#SEC9 ), so I'll try that now.

from rootpy.

Using the ROOT binary works:
https://travis-ci.org/#!/cdeil/rootpy/builds/2823808

If you like it you can turn it on now already for rootpy/rootpy (you should do it as the admin).
Or we work on these things and figure them out first:

• How do we want to run the tests (the tests above should not have failed because numpy is optional for rootpy)
• Do we want python3 support? (in that case I think I'd have to generate an extra binary for python3)
• Do we multiple ROOT versions? (again, requires extra binaries, but I can easily produce them)

We maybe want to avoid using python virtualenvs on travis, I have to try this out:
https://github.com/nipy/nipy/blob/master/.travis.yml

from rootpy.

Great news! Sure, sounds like avoiding virtualenvs on travis is the way to go to get numpy and matplotlib. Let's stick to python 2.6 and .7 for now and add 3 after running some local tests.

from rootpy.

Since ROOT is now in the quantal repositories, you can also install it through apt-get, which works well on travis. I set it up for someone here. Unfortunately there are some missing includes in the official builds, hence the extra includes download. That mostly involves things like geometry though, so you might not need it.

Travis are also planning to update to quantal at some point, then it won't even be necessary to run apt-get update.

from rootpy.

@pwaller Thanks for the tip on the Ubuntu quantal repos.

I'd like to have a small build matrix for travis-ci though, which requires that we build different ROOT binaries for ourselves:

• Python 2.7, 3.2
• ROOT 5.32, 5.34 and trunk

Older ROOT versions unfortunately don't work on the Ubuntu used in travis-ci:
https://savannah.cern.ch/bugs/?func=detailitem&item_id=98354

Building and hosting the ROOT binaries in some online repo instead of in a VM on my Macbook would be nice. I haven't tried this yet (e.g. at https://build.opensuse.org), if you know how to set this up please let me know.

from rootpy.

With the build configuration you can do something like:

env:
    - ROOT_VERSION=5.32
    - ROOT_VERSION=5.34
    - PYTHON_VERSION=2.7
    - PYTHON_VERSION=3.2

Then at the installation stage you could install pre-built packages with something like:

X=root-${ROOT_VERSION}-python-${PYTHON_VERSION}.deb
wget http://url/to/your/cern/area/${X}
dpkg -i ${X}

Of course, I'm assuming you're able to produce debian packages for root in the appropriate form. That would run the four possible tests.

from rootpy.

@ndawe I've added a preliminary https://github.com/rootpy/rootpy/blob/master/.travis.yml

It's only one ROOT / Python version at the moment, but it should already help a bit.
Can you as the admin of the rootpy repo enable the travis-ci build?

from rootpy.

Great! I just flicked the switch. Let's see what happens...

from rootpy.

Can we add a build status badge at the top of our main README?

from rootpy.

I made a commit to trigger a build:
07931a9

from rootpy.

Nice.. watching the build as it goes....

from rootpy.

Wow, everything looks fine!

from rootpy.

Just missing pytables

from rootpy.

I'll fix pytables and add the badge.

I'll also make a pull request to see if we need to do something else to have PRs tested.

If you want to execute more tests scripts just add them to .travis.yml for now.

from rootpy.

The travis-ci status image is not visible in the README:
ab49834

PRs against master are also automatically tested:
#55

Clicking on the details link doesn't works in Chrome, but not in Safari, which I presume is a Safari bug.

I'm now generating the four ROOT binaries for ROOT 5.32 / 5.34 and Python 2.7 / 3.2 and I'm working on Python 3 compatibility.

We need unit tests and a test runner that checks which optional dependencies (like numpy, pytables, ...) so that the tests don't fail for those users!
I don't know how to do this, but I presume we can copy the test infrastructure from other projects.

from rootpy.

Closing this issue, since we now have travis tests.

The issue of making them more thorough can move to new, more specific issues as necessary, such as #61.

from rootpy.