Can't seem to get linearham to run -- probably an issue understanding inputs about linearham HOT 6 CLOSED

Thanks for the help. We can’t run docker on our HPC systems and instead have a container system called singularity-- https://sylabs.io/docs/. I guess it is consider more secure in some way. singularity will pull docker images from DockerHub and convert them to its own format, which is a “sif” file – you’ll see that in my command line. One aspect of it (or possibly our config) is that all container file systems are mounted read-only, so I have to explicitly override the /node directory to make that writable. My command to start the container is this: singularity shell -B node:/node ../linearham_latest.sif After I execute that command I’m “in” the container. Singularity linearham_latest.sif:~/v5x3838/linearham> df -h Filesystem Size Used Avail Use% Mounted on overlay 16M 12K 16M 1% / devtmpfs 126G 0 126G 0% /dev tmpfs 126G 8.0M 126G 1% /dev/shm /dev/mapper/rootvg-rootlv 30G 24G 7.0G 77% /etc/hosts gulf.d51.lilly.com:/nasexp10/v5x3838 579G 404G 175G 70% /home/v5x3838 /dev/mapper/rootvg-tmplv 50G 47M 50G 1% /tmp /dev/mapper/rootvg-varlv 20G 666M 20G 4% /var/tmp nfs-ssd-1fs:/ifs/lrl/users/v5x3838/linearham 3.8P 3.1P 562T 85% /lrlhps/users/v5x3838/linearham tmpfs 16M 12K 16M 1% /etc/group nfs-ssd-1fs:/ifs/lrl/users/v5x3838/linearham/node 3.8P 3.1P 562T 85% /node Singularity linearham_latest.sif:~/v5x3838/linearham> This is my file layout: Singularity linearham_latest.sif:/linearham> cd /home/v5x3838/v5x3838/linearham Singularity linearham_latest.sif:~/v5x3838/linearham> pwd /home/v5x3838/v5x3838/linearham Singularity linearham_latest.sif:~/v5x3838/linearham> ls -l total 117211 -rw-r--r-- 1 v5x3838 sashare 106173662 Nov 14 18:21 TSPN-m6-dedup2.part.yaml drwxr-xr-x 3 v5x3838 sashare 106 Nov 15 15:17 _output drwxr-xr-x 3 v5x3838 sashare 98 Nov 13 16:39 data -rw------- 1 v5x3838 sashare 0 Nov 13 16:46 hat3.seed -rwxr--r-- 1 v5x3838 sashare 1368 Nov 6 21:45 lham.recipe drwxr-xr-x 10 v5x3838 sashare 414 Nov 6 21:54 linearham drwxr-xr-x 3 v5x3838 sashare 25 Nov 13 16:50 node drwxr-xr-x 2 v5x3838 sashare 0 Nov 13 16:40 output drwxr-xr-x 3 v5x3838 sashare 28 Nov 14 21:36 plots -rw------- 1 v5x3838 sashare 0 Nov 13 16:46 pre Singularity linearham_latest.sif:~/v5x3838/linearham> ls -ld /node drwxr-xr-x 3 v5x3838 sashare 25 Nov 13 16:50 /node Singularity linearham_latest.sif:~/v5x3838/linearham> ls -l /linearham/ total 132 -rw-r--r-- 1 root root 1315 Nov 13 14:47 Dockerfile -rw-r--r-- 1 root root 106281 Nov 13 14:47 Doxyfile -rw-r--r-- 1 root root 6907 Nov 13 14:47 README.md -rw-r--r-- 1 root root 17625 Nov 13 14:47 SConstruct drwxr-xr-x 5 root root 60 Nov 12 16:42 _build drwxr-xr-x 4 root root 278 Nov 12 13:31 data drwxr-xr-x 2 root root 117 Nov 12 13:31 doc_images drwxr-xr-x 9 root root 189 Nov 12 13:31 lib drwxr-xr-x 2 root root 232 Nov 12 13:31 scripts drwxr-xr-x 2 root root 354 Nov 12 13:31 src drwxr-xr-x 2 root root 77 Nov 12 13:31 templates drwxr-xr-x 2 root root 48 Nov 12 13:31 test -rw-r--r-- 1 root root 2043 Nov 13 14:47 wercker.yml Then I try to run using the command shown at DockerHub and making an attempt to refer to the outputs of my partis run. Singularity linearham_latest.sif:/linearham> scons --run-linearham --cluster-ind=0 --partis-yaml-file=/home/v5x3838/v5x3838/linearham/TSPN-m6-dedup2.part.yaml --outdir=/home/v5x3838/v5x3838/linearham/_output/ --parameter-dir=/home/v5x3838/v5x3838/linearham/_output/._data_TSPN-Bulk-VH-M6_S1_L001_RM_001.Annot.seq.dedup2/ scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets. Singularity linearham_latest.sif:/linearham> Clearly this is scons thinking that none of its build targets need to be executed, but I can’t really remove any outputs and do not know how to force it. I tried to figure out the partis command being executed, but the SConstruct remaps the parameters and I couldn’t get them right. William F. Smith Bioinformatician BCforward Lilly Biotechnology Center 10290 Campus Point Dr. San Diego, CA 92121 [email protected]<mailto:[email protected]> CONFIDENTIALITY NOTICE: This email message (including all attachments) is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure, copying or distribution is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. From: Elias Harkins <[email protected]> Sent: Friday, November 15, 2019 11:33 AM To: matsengrp/linearham <[email protected]> Cc: William Smith - Network <[email protected]>; Author <[email protected]> Subject: [EXTERNAL] Re: [matsengrp/linearham] Can't seem to get linearham to run -- probably an issue understanding inputs (#57) EXTERNAL EMAIL: Use caution before replying, clicking links, and opening attachments. Regarding : I've mapped in the linearham directory with all executables to be just below the directory where I created the partis outputs , if you are able to share your docker command, that will help me try to understand if it is an issue understanding inputs as you say. Thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#57?email_source=notifications&email_token=ANU5HJS6ZUPRIE5JV6VB3G3QT32P3A5CNFSM4JNWOH52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEGPM7Q#issuecomment-554497662>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANU5HJV45SSDHSUPUTLL5JDQT32P3ANCNFSM4JNWOH5Q>.