Shippable as alternative to jenkins and travis? about galaxytools HOT 13 OPEN

Ooh, always love a new CI competitor. (Btw, we can use docker in Jenkins, I just haven't gone to the trouble of configuring it.) I'll read up on shippable a little :)

Since we have the information from tool_dependencies.xml we could even install packages from the TS before testing tools.

Oh, how do you envision this happening? Shed2tap or similar?

from galaxytools.

Eric;

no concrete plan, but I could imagine something like this:

• we already have a TS Docker image
• now we import all IUC, devteam repositories (build this image every 2 days)
• import the tool/package into our TS Docker image
• spin up Galaxy and install repository
• use planemo to test the tool

from galaxytools.

(Sorry for rambling, I just woke up...)

Not a bad plan at all! Having not dug into shippable very much I'm still thinking in terms of Jenkins....

• IUC+devteam sounds good, we can call it "ToolShed Core" or something, and build that image (locally?) as part of the PR testing phase for both repos (I plan to add jenkins testing for tools-devteam)

I've thought about it a bit, and I don't think this'll be a separate job, I don't think we can take advantage of Jenkins job dependencies here. I think this has to be part of the build/test phase for each repository.
Consider the following:

• PR comes in with package + tool
• need to rebuild TS image
• only then tool testing will pass.

That makes me a bit sad, as I really love having job dependencies in Jenkins. Oh well. We need to develop a Makefile of some sort for handling the build/testing process. I'll probably do that in https://github.com/galaxy-iuc/jenkins-testing as I'd like it to be easier for others to set this up.

from galaxytools.

Yeah, on the other side I would like to use such services integrated in github. Is there a free Jenkins hosting service, like travis?

I don't think we need to rebuild the image for every PR. We don't change our packages so frequently and most of them should be included. So this can be pre-calculated every few days from docker-hub.

from galaxytools.

Sure, using services "properly" integrated into github would be nice...but I'm not aware of a hosted Jenkins solution, or really what benefit we'd get out of it over the VM I've dedicated for IUC use.

If we can get the TS/docker/planemo installing tools bits working, then I'll turn off "unstable" builds and builds will produce a hard failure if even one test fails. I'd like to see new tool PRs build and test successfully rather than have a delay of 2 days causing build failures. I don't mind the CPU overhead; that VM averages <5% utilisation. It's completely wasted! I want to test ALL the things just to get my money's worth out of the compute time allocated to it. :P

from galaxytools.

I have just a bad conscience to use your money for this. It's not just IUC, it's devteam, galaxytools and probably galaxyp-tools. Do you really think there will be a delay for 2 days, I don't think so. But importing all IUC/devteam repos on demand will also work, if the delay turns out to be a problem.

from galaxytools.

Nah, don't worry. I've been happy with Digital ocean so far, and I use that VM for a couple of personal purposes as well. If someone ever gets around to setting this up elsewhere (i.e. research funded), I'm happy to move it then.

Well, here's an example:

• Imagine that the PR is perfect the first time around and requires follow up commits.
• the first set of tests will fail as the associated package isn't installed.
• In <2 days I'll have to add another commit to trigger a rebuild, at which point the packages will be installed (from a PR branch? Via magic?), and the tests will pass.

This is in contrast to:

• Perfect PR is opened
• Build TS image from tools-iuc/packages_* + tools-devteam@master.
• Test again a galaxy instance connected to that local TS image.
• Tests pass the first time around.

I literally have more CPU time than I know what to do with. If it takes another 15 minutes to build that image, or if it bumps my CPU usage up to 10, 20, 30%, I really, really don't care compared to the overhead of not being able to accept a PR after the very first set of passing tests.

Btw, I'm happy to set up jenkins jobs for all of the repos you mentioned, I just want to streamline the process first and get the TS/package_ installation working first. Again, I have more than enough CPU time to accommodate this.

from galaxytools.

An other possibility seems to be circleci: https://circleci.com/docs/docker
This could be used to test galaxy-stable, but with a bit of magic also uploading/installing and testing of tools (based on galaxy-stable).

from galaxytools.

Yeah, that is another alternative. Getting the "magic" figured out is the only hard part. There are plenty of free options for us once we get there :)

from galaxytools.

I'm thinking about activating the TS inside of Galaxy Docker ... with this in place it should be doable.

from galaxytools.

I was using smithcr's TS image to handle this, hadn't worked out the steps to:

• find all package dependencies that are not in the current repository
• mirror those dependencies from the MTS
• upload all packages (shed_serve doesn't seem to do quite that)
• the run shed tests.

from galaxytools.

First one seems to be the hard one, on the other hand all packages should be owned by IUC ;) ...

from galaxytools.

Oh what a nice world that would be :P

from galaxytools.