playing with docker and try making a variant calling pipeline inspired by Omics Pipe and NGSeasy
NGS_docker is a dockerized NGS pipeline for variant calling. This program includes GATK, which is only free for academic and non-profit use. For details see: https://www.broadinstitute.org/gatk/about/#licensing
TODO: build gatk from https://hub.docker.com/r/biodckrdev/gatk/~/dockerfile/ instead of using
ADD
- NGS_docker is designed for a readable and lightweighted variant calling pipeline. The
wgs_pipe.py
is the main entry of the pipeline. Tasks are executed and logged byrun_task
decorator, Additional parameters can be added inprog_cfg.py
. functions following the same formatdef foo(args, param_dict): ... return cmd, output_file
where cmd are the command to be executed by therun_task
,output_file
are required for check existance. - all programs like BWA, GATK and related reference data are dockerized.
- the structure of output dir is:
rootdir/
samplename/
/samplename.vcf #final result
/log/{samplename}.std.txt # put all program output here
/log/{samplename}.err.txt # put error msg here
/log/{samplename}.run.txt # put pipeline running status here
/tmp # all intermadiate files
/tmp/cache_dict.pkl
/report
/variantanno # variant annotation result by snpEff
/fastqc # quality control report by fastqc
cache_dict.pkl
is apickle
dumped dictionary with key:value be the command:outputfiles and is updated after successfully finishing each task. NGS_docker checks the return status of each tasks and delete theoutput_file
if task fails. So for each tasks if the outputfile exists and the command is the same with what stores incache_dict.pkl
, we will skip it.- all reference data is named by its version (eg. hg19) and is stored in
/ref
and mount by tasks by--volumes-from {name}
. rootdir
is a host directory, its subdirrootdir/samplename
will be used to store output. This directory is mounted as/out_dir
and set as working directory for all tasks.