msk-access / access_qc_generation Goto Github PK
View Code? Open in Web Editor NEWGeneration Quality Controls Files for MSK-ACCESS
Home Page: https://cmo-ci.gitbook.io/access-quality-control-v2/
License: Apache License 2.0
Generation Quality Controls Files for MSK-ACCESS
Home Page: https://cmo-ci.gitbook.io/access-quality-control-v2/
License: Apache License 2.0
Just putting this here to keep track, it appears to be sporadic and is fixed with a retry
<=========
[2021-07-01T14:58:39-0400] [Thread-261] [D] [toil.batchSystems.singleMachine] Launched job 240 as child 20646
Traceback (most recent call last):
File "/home/accessbot/miniconda3/envs/toil_test/bin/_toil_worker", line 8, in <module>
sys.exit(main())
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/worker.py", line 704, in main
with in_contexts(options.context):
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/worker.py", line 678, in in_contexts
with manager:
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/batchSystems/abstractBatchSystem.py", line 494, in __enter__
self.arena.enter()
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/lib/threading.py", line 448, in enter
fcntl.lockf(self.lockfileFD, fcntl.LOCK_EX) # type: ignore
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/lib/threading.py", line 366, in global_mutex
os.unlink(lock_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/juno/work/access/testing/users/johnsoni/access_qc/5500_HE/work/toil-mutex-cf0426ab-aa3d-4829-a54f-c264832f82aa-cleanup-arena-lock'
BAM QC:
Inputs:
RUN:
Collapsed FASTQ QC:
Input:
Run
...
self.add_file_or_directory_volume(runtime, vol, host_outdir_tgt)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/cwltool/singularity.py", line 310, in add_file_or_directory_volume
shutil.copytree(volume.resolved, host_outdir_tgt)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/shutil.py", line 318, in copytree
names = os.listdir(src)
FileNotFoundError: [Errno 2] No such file or directory: '_:4694b3bc-a86a-4be4-8d99-b6876552c8a8'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/worker.py", line 400, in workerScript
job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/job.py", line 2382, in _runner
returnValues = self._run(jobGraph=None, fileStore=fileStore)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/job.py", line 2303, in _run
return self.run(fileStore)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/cwl/cwltoil.py", line 1921, in run
logger=cwllogger,
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/cwltool/executors.py", line 149, in execute
self.run_jobs(process, job_order_object, logger, runtime_context)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/toil/cwl/cwltoil.py", line 848, in run_jobs
return super().run_jobs(process, job_order_object, logger, runtime_context)
File "/home/accessbot/miniconda3/envs/toil_test/lib/python3.7/site-packages/cwltool/executors.py", line 267, in run_jobs
raise WorkflowException(str(err)) from err
cwltool.errors.WorkflowException: [Errno 2] No such file or directory: '_:4694b3bc-a86a-4be4-8d99-b6876552c8a8'
[2021-07-01T15:49:09-0400] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host jx04
<=========
[2021-07-01T15:54:22-0400] [MainThread] [E] [root] Your workflow uses a CWL requirement that Toil does not support!
all_qc_files
/all_qc_files
duplicate dir name_mqc.html
files inside all_qc_files
folder (preferably with the associated sequence_qc / biometrics outputs)Create a fingerprinting python module, probably using click from python to have multiple subcommands.
Here are things that we talked about @maysunh
Things to do:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.