bioconda / bioconda-paper Goto Github PK

View Code? Open in Web Editor NEW

4.0 20.0 3.0 13.96 MB

Data analysis related to the bioconda paper

License: MIT License

Python 67.56% TeX 32.44%

snakemake reproducibility bioconda analysis-workflows

bioconda-paper's Introduction

Data analysis for the Bioconda paper

This Snakemake workflow automatically generates all results and figures from the Bioconda paper.

Requirements

Any 64-bit Linux installation with GLIBC 2.5 or newer (i.e. any Linux distribution that is newer than CentOS 6). Note that the restriction of this workflow to Linux is purely a design decision (to save space and ensure reproducibility) and not related to Conda/Bioconda. Bioconda packages are available for both Linux and MacOS in general.

Usage

This workflow can be used to recreate all results found in the Bioconda paper.

Step 1: Setup system

Variant a: Installing Miniconda on your system

If you are on a Linux system with GLIBC 2.5 or newer (i.e. any Linux distribution that is newer than CentOS 6), you can simply install Miniconda3 with

curl -o /tmp/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh && bash /tmp/miniconda.sh

Make sure to answer yes to the question whether your PATH variable shall be modified. Afterwards, open a new shell/terminal.

Variant b: Use a Docker container

Otherwise, e.g., on MacOS or if you don't want to modify your system setup, install Docker, run

docker run -it continuumio/miniconda3 /bin/bash

and execute all the following steps within that container.

Variant c: Use an existing Miniconda installation

If you want to use an existing Miniconda installation, please be aware that this is only possible if it uses Python 3 by default. You can check this via

python --version

Further, ensure it is up to date with

conda update --all

Step 2: Setup Bioconda channel

Setup Bioconda with

conda config --add channels defaults
conda config --add channels conda-forge
conda config --add channels bioconda

Step 3: Install bioconda-utils and Snakmake

Install bioconda-utils and Snakemake >=4.6.0 with

conda install bioconda-utils snakemake

If you already have an older version of Snakemake, please make sure it is updated to >=4.6.0.

Step 4: Download the workflow

First, create a working directory:

mkdir bioconda-workflow
cd bioconda-workflow

Then, download the workflow archive from https://doi.org/10.5281/zenodo.1068297 and unpack it with

tar -xf bioconda-paper-workflow.tar.gz

Step 5: Run the workflow

Execute the analysis workflow with Snakemake

snakemake --use-conda

Please wait a few minutes for the analysis to finish. Results can be found in the folder figs/. If you have been running the workflow in the docker container (see above), you can obtain the results with

docker cp <container-id>:/bioconda-workflow/figs .

whith <container-id> being the ID of the container.

Known errors

If you see an error like
```
ImportError: No module named 'appdirs'
```
when starting Snakemake, you are likely suffering from a bug in an older conda version. Make sure to update your conda installation with
```
conda update --all
```
and then reinstall the appdirs and snakemake package with
```
conda install -f appdirs snakemake
```
If you see an error like
```
ImportError: Missing required dependencies ['numpy']
```
you are likely suffering from a bug in an older conda version. Make sure to update your conda installation with
```
conda update --all
```
and then reinstall the snakemake package with
```
conda install -f snakemake
```

bioconda-paper's People

Contributors

Stargazers

Watchers

Forkers

cokelaer luispedro

bioconda-paper's Issues

package dependencies (metric and plot)

@johanneskoester If you are interested:

mean number of package dependencies: 3.6
median number of package dependencies: 2

I can update the Snakefile to produce the histogram of package dependencies

add scripts

@johanneskoester would you please commit scripts/plot-comparison.py and script/plot-turnaround.py?

Show that bioconda recipes are constantly updated

For me, one of the main selling points (for lack of a better word) of bioconda is the availability of recent versions of software. When bioinformatics software is available in Linux package managers like apt, apart from the usual problem of not having root access to install it, it tends to be horribly outdated. Can we pour this into a graphic somehow? Some interesting parameters might be:

How many updates do existing recipes receive? (This could be evaluated by counting the different version numbers available or commits changing the version number. I am not sure what is representative for actual "updates" to a recipe.
If a bioconda package is also available as a ubuntu package, is the bioconda version older/ the samer/ newer than the ubuntu version? This could be done for any other distribution and/ or package manger.
It would be awesome to find out how many recipes are at the most recent version of the respective software, but I do not see an easy way to check this.

I will try to implement this if you think this would be helpful.

Testing the analysis workflow

@bioconda/core, everybody else reading this issue. One of the reviewers complained about errors when executing our analysis workflow. Those were caused by (a) bioconda-utils was not easy to install and (b) there was a (temporal) error in the numpy package from conda-forge and (c) my instructions were probably too sparse. To ensure that everything works this time, I would like to ask you to test the fixed version of our workflow with updated instructions: https://github.com/bioconda/bioconda-paper

It would be great if you could simply go through the steps in the README and report any occuring problems or unclear instructions here.

Thanks,
Johannes

link authorea paper with github repo?

Apparently Authorea can be made to link up with a github repo (syncing both ways):

https://intercom.help/authorea/user-settings/user-settings/syncing-articles-to-github

@johanneskoester Thoughts? It may be worthwhile to do even if it's late now, if only to preserve the paper as we preserve everything else. Just close this if you decide it's not worth the bother.

Update security considerations part to include a note about lookalike packages

In the past couple of years, pretty much all big package repositories were attacked by malware squatting on common typos of popular packages. I can look up the publication describing some proof-of-concept attacks on NPM in a bit.

We should at least have a note regarding this problem in there, instead of the blanket "users need to make sure they trust the source".