Hi Lucas! In an effort to troubleshoot as much as I can with Grenepi

Hey <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Hey <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

GRENEPIPE v12.1,about moiexpositoalonsolab/grenepipe

Comments (5)

lczech commented on July 30, 2024

Hey Mariano,

it seems that you are using a newer version of grenepipe with an older config.yaml - that won't work. It's a bit unfortunate, but while we are in grenedalf pre-v1.0.0, to keep the code simple, releases are not always backwards compatible. So, to solve this, you'll have to use the config.yaml of the version you are using, and adapt that to your dataset. Usually, you can re-use most of it - use the diff or sdiff command on your terminal to find the differences between the two config files, and make changes as to the new one as needed.

Cheers
Lucas

from grenepipe.

mrese001 commented on July 30, 2024

Hey Lucas!

Thanks for the advice. I changed the yaml file to the correct version and now onto what I think may be the source of my stress around this workflow: I found that by typing:

$ conda info

I get the following collapsible details and error. I think this may be because either/both:

I have my conda environment in a space with too little memory
I am asking snakemake to use miniconda3 from the cluster and not the one locally

Would this be possible? If so how can I ask snakemake to use the local version?

Thanks again and talk soon :)

Details

active environment : base
    active env location : /opt/linux/rhel/8.x/x86_64/pkgs/miniconda3/py39_4.12.0
            shell level : 1
       user config file : /rhome/rpisc002/.condarc
 populated config files :
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.9.15.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.28=0
                          __linux=4.18.0=0
                          __unix=0=0
       base environment : /opt/linux/rhel/8.x/x86_64/pkgs/miniconda3/py39_4.12.0  (read only)
      conda av data dir : /opt/linux/rhel/8.x/x86_64/pkgs/miniconda3/py39_4.12.0/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/linux/rhel/8.x/x86_64/pkgs/miniconda3/py39_4.12.0/pkgs
                          /rhome/rpisc002/.conda/pkgs
       envs directories : /rhome/rpisc002/.conda/envs
                          /opt/linux/rhel/8.x/x86_64/pkgs/miniconda3/py39_4.12.0/envs
               platform : linux-64
             user-agent : conda/23.3.1 requests/2.28.1 CPython/3.9.15 Linux/4.18.0-348.12.2.el8_5.x86_64 rocky/8.5 glibc/2.28
                UID:GID : 3907:1247
             netrc file : None
           offline mode : False

Here is my complete error:


Touching output file called/all.scaffold_576.g.done.
Full Traceback (most recent call last):
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2347, in run_wrapper
    edit_notebook,
  File "/bigdata/koeniglab/rpisc002/GRENEPIPE/grenepipeWD/rules/qc-bam.smk", line 311, in __rule_picard_collectmultiplemetrics
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/script.py", line 938, in script
    executor.evaluate()
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/script.py", line 313, in evaluate
    self.execute_script(fd.name, edit=edit)
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/script.py", line 506, in execute_script
    self._execute_cmd("{py_exec} {fname:q}", py_exec=py_exec, fname=fname)
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/script.py", line 354, in _execute_cmd
    **kwargs
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/shell.py", line 231, in __new__
    raise sp.CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'set -euo pipefail;  /rhome/rpisc002/.conda/envs/grenepipe/bin/python3.7 /bigdata/koeniglab/rpisc002/GRENEPIPE/grenep
ipeWD/parents_draft/.snakemake/scripts/tmpdupx9ezb.picard-collectmultiplemetrics.py' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 568, in _callback
    raise ex
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 554, in cached_or_run
    run_func(*args)
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2359, in run_wrapper
    ex, lineno, linemaps=linemaps, snakefile=file, show_traceback=True
snakemake.exceptions.RuleException: CalledProcessError in line 205 of /bigdata/koeniglab/rpisc002/GRENEPIPE/grenepipeWD/rules/qc-bam.smk:
Command 'set -euo pipefail;  /rhome/rpisc002/.conda/envs/grenepipe/bin/python3.7 /bigdata/koeniglab/rpisc002/GRENEPIPE/grenepipeWD/parents_draft/.snakemake/
scripts/tmpdupx9ezb.picard-collectmultiplemetrics.py' returned non-zero exit status 1.
  File "/rhome/rpisc002/.conda/envs/grenepipe/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2347, in run_wrapper
  File "/bigdata/koeniglab/rpisc002/GRENEPIPE/grenepipeWD/rules/qc-bam.smk", line 205, in __rule_picard_collectmultiplemetrics

from grenepipe.

lczech commented on July 30, 2024

Hey @mrese001,

yes, those errors look like they could come from mispatching conda/python/pandas versions. You seem to be operating in base environment, and not in a conda environment for grenepipe, as described here. That might be the issue. The beginning of the grenepipe output log lists all the base conda and python versions and environments - can you compare if those match your expectations, and/or post that first bit of the log file here as well? For example, above, it says that python is in version 3.9.15, while grenpipe uses python 3.7.10, so something is not quite right there.

I would highly recommend to follow the setup as describe here, as otherwise you can easily run into such incompatibilities. Please check that this is the case.

Cheers
Lucas

from grenepipe.

lczech commented on July 30, 2024

Hi @mrese001,

any updates on this?

Cheers
Lucas

from grenepipe.

mrese001 commented on July 30, 2024

Hey @lczech ,

You were indeed correct - I had the base environment loaded instead of the build for grenepipe. Sorry it took so long for me to respond to your replies!

Mariano

from grenepipe.

GRENEPIPE v12.1 about grenepipe HOT 5 CLOSED

Comments (5)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent