Useful imaging tools for Brain data about galaxytools HOT 14 OPEN

@bgruening The installation instructions for https://github.com/paolo-f/bcfind install the head of the git repo. For the tool_dependencies.xml recipe, should I use a "git clone..." followed by a "git checkout..." as the mechanism for providing a specific feature set for reproducibility? This would be similar to what is being done for the freebase package recipe and others. If so, should I just use the current tip for the recipe?

from galaxytools.

@gregvonkuster I use git reset --hard <commit_hash> after the git clone .

from galaxytools.

@nsoranzo Thanks, I'll go that route and use the current repo head as the commit hash.

from galaxytools.

@bgruening @nsoranzo This package is going to require a fairly complex installation recipe with many dependencies (e.g., PIL, tables, pandas, scikit-learn, progressbar-latest, numpy, scipy, mahotas, ujson) all installed into a version of Continuum Analytic Anaconda Python (http://continuum.io/downloads).

My understanding of how package dependencies like this work may be flawed, so I'm hoping you can clarify or confirm things for me.

Like other package recipes that install Python as a dependency, Galaxy job execution for tools that use this package will run with the Continuum Analytic Anaconda Python (due to the dependency setting the PYTHONPATH variable prior to job execution) instead of the Galaxy Python environment that consists of a "vanilla" Python along with the eggs/wheels components that provides Galaxy dependencies.

This means that the Continuum Analytic Anaconda Python will need to install all dependencies Galaxy needs for job execution (similar to how this is done with the other "package_python..." packages). This list of dependencies currently includes package_bzlib_1_0, package_sqlite_3_8_3 and package_zlib_1_2_8 (possibly) among others.

The weakness here is that recipes like this will need to be "fixed" whenever the Galaxy job framework adds an additional dependency.

Is my understanding of this correct? I appreciate you feedback on this as I have other recipes I'm planning to submit to open source that use this same approach and I want to make sure I'm going about this the right way.

from galaxytools.

@gregvonkuster @nsoranzo you don't need to use git to get a specific commit from github. Curl/wget works for every revision.

For example this one for the last revision: https://github.com/paolo-f/bcfind/archive/ed3ea2c29d22652996fe27ae509625c52b5b77c2.tar.gz.

from galaxytools.

@gregvonkuster regarding your python installation question, this is actually a topic I'm currently work on.
Please have a look at the deeptools package this should help you a lot without conda: galaxyproject/tools-iuc#277

Please let me know if this fixes your problem.

from galaxytools.

@bgruening I've got the installation recipes with all dependencies basically done for bcfind. I noticed that your current tool outline at https://github.com/bgruening/galaxytools/blob/master/tools/image_processing/bcfind/bcfind.xml has set the bcfind dependency version to 1.3.18 via your requirement tag.

<requirement type="package" version="1.3.18">bcfind</requirement>

Is this the version you want me to set in the bcfind tool_dependencies.xml recipes I have? Currently I have this since it is the repo head.

<package name="bcfind" version="ed3ea2c29d"&gtl;

My install recipe installs the package using this URL which is the repo head.

<action type="download_file" target_filename="bcfind.tar.gz">https://github.com/paolo-f/bcfind/archive/ed3ea2c29d22652996fe27ae509625c52b5b77c2.tar.gz&lt;/action>

Please let me know what you would like for the version string. Thanks!

from galaxytools.

This was a old version, we can/should update the version :)
Thanks!

from galaxytools.

@bgruening ok, just to confirm, should it be "ed3ea2c29d"? Thanks!

from galaxytools.

I think this is ok!

from galaxytools.

@bgruening Need some insight on the tool(s) you are looking for here. The bcfind manual at http://bcfind.dinfo.unifi.it/guide.html sort of implies a suite of tools as:

bcfind_make_substacks
bcfind_cell_finder
bcfind_measure_performance
bcfind_supervised_semantic_deconvolution
bcfind_merge_markers
bcfind_manifold_filter

Can you let me know if this baseline tool set is what you are looking for?

FYI:
The current scripts are written in such a way that wrapping them in Galaxy will be a bit more complex than usual. There are a lot of assumptions on input and output directory names, specific file extensions, etc. So the Galaxy wrappers will likely have to include a lot of temporary manipulations of these items.

Also, some of the scripts (make_substacks.py at https://github.com/paolo-f/bcfind/blob/master/bcfind/scripts/make_substacks.py) seem to imply the Galaxy wrapper should be written to work with collections which I've not yet done. Any pointers to tools that have been optimally written to deal with collections will be helpful.

Thanks!

from galaxytools.

I have forwarded you a mail from the author, with more informations.
Here you can find an example of producing dataset collections:
https://github.com/bgruening/galaxytools/blob/master/tools/text_processing/split_file_on_column/split_file_on_column.xml
Also inside of the planemo documentation is a really good tutorial:
https://planemo.readthedocs.org/en/latest/writing_advanced.html#collections

And last but not least the functional collection tests are awesome as training material:
https://github.com/galaxyproject/galaxy/tree/dev/test/functional/tools
This is where I'm looking for suggestions :)

from galaxytools.

A new version was released a few days back: https://github.com/paolo-f/bcfind/releases

from galaxytools.

Thanks Bjorn,

I’ll keep this on my list, but it will be a while. I’m currently buried with Galaxy consulting for 1 lab and I have an additional lab I’ll be supporting starting next week. I’ll take a look at this as soon as I get a chance though.

Thanks Bjorn!

Greg

On Nov 6, 2015, at 4:39 AM, Björn Grüning [email protected] wrote:

A new version was released a few days back: https://github.com/paolo-f/bcfind/releases https://github.com/paolo-f/bcfind/releases
—
Reply to this email directly or view it on GitHub #142 (comment).

from galaxytools.