Conda installation error about grenepipe HOT 7 CLOSED

Hi All, I tried to install grenepipe on my AWS conda environment but it shows an error that I have pasted above. Could you please help?

from grenepipe.

Hi @Samarth-T-Setty,

that error seems to come from some mis-configured AWS conda setup file. The error is for example described here, so it's coming from a yaml file that is not correct - but not from the yaml file that you are using to specify the conda environment (that is, the envs/grenepipe.yaml file is not the broken one).

With some digging and google sleuth work, and looking at the exact line where that error occurs in your error report above:

     ... le" content="{"groups": [], "environmentKey" ... 
                                         ^ (line: 28)

I find that google gives me some obscure match to a file, see here (fittingly, that file is name Malicious.txt, but no worries, it's just a text file and doesn't do harm) that contains this bit of text. And that file is related to amazonaws (see the beginning of that file), so hence I believe that this error is not related to grenepipe at all, but is a problem with AWS.

I'd contact their support, and tell them that your command

mamba env create -f envs/grenepipe.yaml

fails, and ask them to fix it. That should be a pretty straight forward conda/mamba environment installation, and that already failing means that something is wrong on their end. I hope they get this figured out :-)

Hope that helps, so long
Lucas

from grenepipe.

Thanks @lczech, I managed to figure out that the ### envs/grenepipe.yaml### was not downloaded fully and I could get it working after creating a new one. I have used the DNA-seq cookie cutter template before and currently I see that installing the conda environments for the tools (from the wrappers) takes a really long time. From the past 2 days I have just been waiting on the following steps -

Building DAG of jobs...
Creating conda environment https:/github.com/snakemake/snakemake-wrappers/raw/0.51.3/bio/samtools/faidx/environment.yaml...
Downloading and installing remote packages.
Environment for ../../../../tmp/tmptv6ah28_.yaml created (location: .snakemake/conda/cd6d13c632d04332f5d4a51c66280271)
Removing incomplete Conda environment ../envs/picard.yaml...
Creating conda environment ../envs/picard.yaml...
Downloading and installing remote packages.

Environment for ../envs/picard.yaml created (location: .snakemake/conda/3eeaf74a4852753209edc9a65d7e91a3)
Creating conda environment ../envs/bwa.yaml...
Downloading and installing remote packages.
Environment for ../envs/bwa.yaml created (location: .snakemake/conda/fb41ee5b51ce5a2d5decf9573b3a8e96)
Removing incomplete Conda environment ../envs/qualimap.yaml...
Creating conda environment ../envs/qualimap.yaml...
Downloading and installing remote packages.

Any way to speeden up the conda env. installs?

from grenepipe.

Here are the versions -
Conda: 4.12.0
Python: 3.7.10
Snakemake: 6.0.5
Grenepipe: 0.9.0
Command: /home/samarth/miniconda3/envs/grenepipe/bin/snakemake
--use-conda -j 25 -kp
--directory run

Base directory:     /home/samarth/sample_run
Working directory:  /home/samarth/sample_run/run
Config file(s):     /home/samarth/sample_run/run/config.yaml
Samples:            32

from grenepipe.

Yes, when running, also specify --conda-frontend mamba, that will speed things up massively! You'll need to have mamba installed for this, which conveniently can be done via conda with conda install -c conda-forge mamba.

I have had similar experiences with conda being slow, also for the picard environment. It seems that conda is struggling with that... I'll keep it in mind and see if I can find a fix for that. But for now, just use mamba ;-)

from grenepipe.

Thanks, mamba worked well for me and the pipeline is running smoothly so far.

from grenepipe.

Yeeey, happy to hear! I'll close this issue then, but feel free to start a new one should anything else not work :-)

from grenepipe.