Git Product home page Git Product logo

nextflow_scri_config's Introduction

nextflow_scri_config

An SCRI nextflow profile for NF-Core for reference which is pulled to 'https://github.com/nf-core/configs/blob/master/conf/seattlechildrens.config'.

The provided config here does not include the singularity scope, but SCRI does have some recommended settings for Singularity in the nextflow config, but might not be correct for all pipelines.

Here is an example of the singularity scope settings used in most SCRI pipelines.

//Configs for singularity containers on cybertron
singularity {
    autoMounts = true
    cacheDir = "$HOME/singularity_test"
    runOptions = '--containall --no-home'
}

Test the config file

Please note that --project parameter can be specified on the command line in nextflow run or in the seattlechildrens.config.

ml singularity

# For running on an interactive session on cybertron with singularity module loaded
nextflow run -c 'conf/seattlechildrens.config' \
    nf-core/rnaseq \
    -r 3.14.0 \
    -profile test,local_singularity \
    --outdir "/home/$USER/temp/nf-core-rnaseq-test" \
    --project "207f23bf-acb6-4835-8bfe-142436acb58c" \
    --validationSchemaIgnoreParams 'queue,project'

# For executing the jobs on the HPC cluster with singularity containers
# doesn't work due to bioconductor-dupradar:1.28.0--r42hdfd78af_0
nextflow run -c 'conf/seattlechildrens.config' \
    nf-core/rnaseq \
    -r 3.14.0 \
    -profile test,PBS_singularity \
    --max_memory 32.GB \
    --outdir "/home/$USER/temp/nf-core-rnaseq-test" \
    --project "207f23bf-acb6-4835-8bfe-142436acb58c" \
    --validationSchemaIgnoreParams 'queue,project'
nextflow run -c 'conf/seattlechildrens.config' \
    nf-core/fetchngs \
    -r 1.12.0 \
    -profile test,PBS_singularity \
    --input ./ids.csv \
    --download_method 'sratools' \
    --project "207f23bf-acb6-4835-8bfe-142436acb58c" \
    --outdir "/home/$USER/fetchngs"
nf-core launch nf-core/methylseq -r 2.6.0

Change log

2023.03.18 Built during nf-core biohackathon which synced for Cybertron and PBS.

nextflow_scri_config's People

Contributors

chaojenwong avatar jennylsmith avatar nkatiyar avatar ybaeus avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

nextflow_scri_config's Issues

wget: unable to resolve host address

When testing with FETCHNGS pipeline, it throws error regarding pulling container via FTP.

Command error:
  WARNING: Skipping mount /depot/apps/singularity-ce/3.9.9/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  wget: unable to resolve host address 'ftp.sra.ebi.ac.uk'

Details:

ERROR ~ Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)'

Caused by:
  Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (SRX9626017_SRR13191702)` terminated with an error exit status (4)

Command executed:

  wget \
      -t 5 -nv -c -T 60 \
      -O SRX9626017_SRR13191702_1.fastq.gz \
      ftp.sra.ebi.ac.uk/vol1/fastq/SRR131/002/SRR13191702/SRR13191702_1.fastq.gz
  
  echo "89c5be920021a035084d8aeb74f32df7  SRX9626017_SRR13191702_1.fastq.gz" > SRX9626017_SRR13191702_1.fastq.gz.md5
  md5sum -c SRX9626017_SRR13191702_1.fastq.gz.md5
  
  wget \
      -t 5 -nv -c -T 60 \
      -O SRX9626017_SRR13191702_2.fastq.gz \
      ftp.sra.ebi.ac.uk/vol1/fastq/SRR131/002/SRR13191702/SRR13191702_2.fastq.gz
  
  echo "56271be38a80db78ef3bdfc5d9909b98  SRX9626017_SRR13191702_2.fastq.gz" > SRX9626017_SRR13191702_2.fastq.gz.md5
  md5sum -c SRX9626017_SRR13191702_2.fastq.gz.md5
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP":
      wget: $(echo $(wget --version | head -n 1 | sed 's/^GNU Wget //; s/ .*$//'))
  END_VERSIONS

Command exit status:
  4

Command output:
  (empty)

Command error:
  WARNING: Skipping mount /depot/apps/singularity-ce/3.9.9/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  wget: unable to resolve host address 'ftp.sra.ebi.ac.uk'

Work dir:
  /active/taylor_s/people/ybae/RSC/nextflow_scri_config/work/ec/61fea21fd14d5de2b97c27914b5a64

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details

RNA-seq Pipeline Cannot Pull Dupradar

On HPC, cannot pull dupradar from repository.

Error:
singularity pull
exit code 143

To Reproduce:

# For running on an interactive session on cybertron with singularity module loaded
nextflow run -c 'conf/seattlechildrens.config' \
    nf-core/rnaseq \
    -r 3.14.0 \
    -profile test,local_singularity \
    --outdir "/home/$USER/temp/nf-core-rnaseq-test" \
    --project "207f23bf-acb6-4835-8bfe-142436acb58c" 

# For executing the jobs on the HPC cluster with singularity containers
# doesn't work due to bioconductor-dupradar:1.28.0--r42hdfd78af_0
nextflow run -c 'conf/seattlechildrens.config' \
    nf-core/rnaseq \
    -r 3.14.0 \
    -profile test,PBS_singularity \
    --max_memory 32.GB \
    --outdir "/home/$USER/temp/nf-core-rnaseq-test" \
    --project "207f23bf-acb6-4835-8bfe-142436acb58c" 

About Singularity Scope

SCRI does have some recommended settings for Singularity in the nextflow config, but might not be correct for all pipelines. Here is an example of the singularity scope settings used in most SCRI pipelines.

//Configs for singularity containers on cybertron
singularity {
    autoMounts = true
    cacheDir = "$HOME/singularity_test"
    runOptions = '--containall --no-home'
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.