bxlab / metawrap Goto Github PK
View Code? Open in Web Editor NEWMetaWRAP - a flexible pipeline for genome-resolved metagenomic data analysis
License: MIT License
metawrap's People
Contributors
Stargazers
Watchers
Forkers
dgelsin xtmgah ichobits laam21 balays thexiyang yi1873 pythseq takadonet liupfskygre luanxiao wilsonyangliu brymerr921 migdesmarais dayueban yangkangyf qiongmeng-m ravinpoudel genetalks-fred lytlxy rayliu0115 jimaz jinqunhuang jacorwin dangchenyuan liaochenlanruo lizhiming11 xuzetan bestweicheng jianshu93 cloudoo qi-lee raissafedora george-githinji bayegy rnilsenhub drothen15 zwidny liaoherui boulund tankmermaid shafieif anuragpassi yyuangy2019 kilaza kenkeni-zju rdejonge xushaoyi rheannna shicheng-guo neptuneyt mortonjt arooney13 teresau sknishtala microbiaki jchenpku gibsramen davidcalfran rriyuzo bharat1912 salvoc81 yinx843 hj1994412 mita2020 ditag chasefor rli-study kyoreth slaperriere mikesqc vipmenon jameslz zjyzjjzmt janedy tiehan heyhuru zhaoxia413 bridgeto mol529 rhysnewell laurichi13 ebi-metagenomics seedpcseed ramakrishnas haihuawang-hub 25280841 xiangrong131 leornardzhou smdabdoub alexandrasimas biotovarx sahutt hbtang25 ryan10937 dyxstat jianghexiliu shuqili jbuongio fly-he-20metawrap's Issues
additional installation step question
I did a fresh install of the newly named conda package into a newly created conda environment.
After installation, I exited my ssh session and then re-entered. I copied my copy of the configuration file into the new conda environment. I tried to test the installation and got the following error:
(metawrap2-env) [[email protected]@bigmem0009 Scripts]$ which config-metawrap
~/scratch/miniconda2/envs/metawrap2-env/bin/config-metawrap
(metawrap2-env) [[email protected]@bigmem0009 Scripts]$ metaWRAP read_qc -h
/home-3/[email protected]/scratch/miniconda2/envs/metawrap2-env/bin/metaWRAP: line 33: config-metawrap: No such file or directory
/read_qc.sh -h
/home-3/[email protected]/scratch/miniconda2/envs/metawrap2-env/bin/metaWRAP: line 53: /read_qc.sh: No such file or directory
I got this error the first time I installed metawrap and got around it by adding the ~/scratch/miniconda2/envs/metawrap2-env/bin directory & metawrap-modules subdirectory to $PATH.
I think this is not an ideal solution and was wondering if you had any better ideas? Otherwise, if this step is required for installation, you might want to include it in the installation instructions.
kraken won't build + other stuff
one issue i noted was that the link to the checkM database changed and the autoupdate function doesn't work anymore (probs for that reason)
also, kraken won't build on MARCC for whatever reason. I submitted the issue to them as well. did you get around this issue?
[[email protected]@login-node03 ~]$ cd scratch/metaWRAP_DBs/
[[email protected]@login-node03 kraken_db]$ interact -p lrgmem -t 480 -n 24 -m 600G
Tasks: 24
Cores/task: 1
Total cores: 24
Walltime: 480
Reservation:
Queue: lrgmem
Command submitted: salloc -J interact -N 1-1 -n 24 -c 1 --mem=600G --time=480 -p lrgmem srun --pty bash
salloc: Granted job allocation 19524778
Using standard path: /cm/shared/apps/gsl/1.16
Using standard path: /cm/shared/apps/gsl/1.16
[email protected] currently using 6G of 20G available in home directory.
Filling home quota will prevent the ability to log in.
[[email protected]@bigmem0041 kraken_db]$ source ~/.bash_profile
Using standard path: /cm/shared/apps/gsl/1.16
Using standard path: /cm/shared/apps/gsl/1.16
[email protected] currently using 6G of 20G available in home directory.
Filling home quota will prevent the ability to log in.
[[email protected]@bigmem0041 kraken_db]$ source activate metawrap-env
(metawrap-env) [[email protected]@bigmem0041 kraken_db]$ kraken-build --standard --threads 24 --db MY_KRAKEN_DATABASE
Possible unintended interpolation of @jhu in string at /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/kraken-build line 28.
Global symbol "@jhu" requires explicit package name (did you forget to declare "my @jhu"?) at /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/kraken-build line 28.
BEGIN not safe after errors--compilation aborted at /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/kraken-build line 32.
summarize_checkm.py doesn't fill final column of stats
in most instances, len(sys.argv) is <2, so a column of NAs is added to reassembled_bins.stats.
but in bin refinement binsO.checkm is added to all columns of metaWRAP_bins.stats, which doesn't really show where the refined bins came from. maybe this info is elsewhere?
Confirm that use of BLAST's `-max_target_seqs` is intentional
Hi there,
This is a semi-automated message from a fellow bioinformatician. Through a GitHub search, I found that the following source files make use of BLAST's -max_target_seqs parameter:
Based on the recently published report, Misunderstood parameter of NCBI BLAST impacts the correctness of bioinformatics workflows, there is a strong chance that this parameter is misused in your repository.
If the use of this parameter was intentional, please feel free to ignore and close this issue but I would highly recommend to add a comment to your source code to notify others about this use case. If this is a duplicate issue, please accept my apologies for the redundancy as this simple automation is not smart enough to identify such issues.
Thank you!
-- Arman (armish/blast-patrol)
FYI: error in shebang in summarize_check_m.py
Running the bin-refiner module resulting in an error. Upon inspecting the log file, I found at least part of the problem (see below). Simply modifying the shebang to #!usr/bin/env python seemed to take care of it. I checked the rest of the of the scripts in bin/metawrap-scripts and this was the only one that seemed to suffer from this problem.
/var/lib/condor/execute/slot1/dir_30291/python/bin/metawrap-modules/bin_refinement.sh: /var/lib/condor/execute/slot1/dir_30291/python/bin/metawrap-scripts/summarize_checkm.py: /home-2/[email protected]/scratch/miniconda2/bin/python: bad interpreter: No such file or directory
Cheers!
metawrap reassemble_bins does not work
Hi, I am using the metawrap reassemble_bins module. There are two issues.
(1) --parallel option does not work: unrecognized option '--parallel'.
(2) the procedure got stuck at the closing bins step and did not proceed further. The command I used: metawrap reassemble_bins -o 08_reassemble_bins -b 05_BIN_REFINEMENT/metaWRAP_bins/metaWRAPbins/ -1 all_reads/ALL_READS_1.fastq -2 all_reads/ALL_READS_2.fastq -t 20 -m 800 -c 50 -x 10.
Thanks!
path issue with concoct
Similar issue to the previous one, this time on line 282.
Yields the following:
Traceback (most recent call last):
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/concoct", line 4, in <module>
__import__('pkg_resources').run_script('concoct==0.4.0', 'concoct')
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 750, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1527, in run_script
exec(code, namespace, namespace)
File "/scratch/users/[email protected]/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/EGG-INFO/scripts/concoct", line 76, in <module>
results = main(args)
File "/scratch/users/[email protected]/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/EGG-INFO/scripts/concoct", line 20, in main
composition, cov, cov_range = load_data(args)
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/concoct/input.py", line 16, in load_data
args.length_threshold
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/concoct/input.py", line 70, in load_composition
kmer_len
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/concoct/input.py", line 40, in _calculate_composition
for seq in SeqIO.parse(comp_file,"fasta"):
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/Bio/SeqIO/__init__.py", line 581, in parse
with as_handle(handle, mode) as fp:
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/Bio/File.py", line 88, in as_handle
with open(handleish, mode, **kwargs) as fp:
IOError: [Errno 2] No such file or directory: '/home-3/[email protected]/scratch/THRASH_LIBS/Scripts//home-3/[email protected]/scratch/THRASH_LIBS/DNA_BINS/work_files/assembly.fa'
error on metaWRAP bin refinement command
I used three bins sets to run this command, apparently some sets of bins didn't yield any refined bin results and that makes that checkm part goes in an halt. Bellow a copy of all log.
This program seems so promising, but I have been struggling to work with it.
Thank for any help.
(metawrap-env) igor@ubuntu:~/metaWRAP/Binning_Results$ metawrap bin_refinement -o BIN_REFINEMENT -t 96 -A /home/igor/metaWRAP/Binning_Results/AC3_bulkcontigs_renamed.fas.metaBAT2_bins/ -B /home/igor/metaWRAP/Binning_Results/AC3_Binning_MyCC_cov/ -C /home/igor/metaWRAP/Binning_Results/metaBAT1/ -c 50 -x 10
/home/igor/miniconda2/envs/metawrap-env/bin/metawrap-modules/bin_refinement.sh -o BIN_REFINEMENT -t 96 -A /home/igor/metaWRAP/Binning_Results/AC3_bulkcontigs_renamed.fas.metaBAT2_bins/ -B /home/igor/metaWRAP/Binning_Results/AC3_Binning_MyCC_cov/ -C /home/igor/metaWRAP/Binning_Results/metaBAT1/ -c 50 -x 10
########################################################################################################################
##### BEGIN PIPELINE! #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- setting up output folder and copything over bins... -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- There are 3 bin sets! -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### BEGIN BIN REFINEMENT #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- There are three bin folders, so there 4 ways we can refine the bins (A+B, B+C, A+C, -----
----- A+B+C). Will try all four in parallel! -----
------------------------------------------------------------------------------------------------------------------------
Specified 2 input bin sets: -1 binsA -2 binsC
Specified 2 input bin sets: -1 binsC -2 binsB
Specified 2 input bin sets: -1 binsA -2 binsB
Specified 3 input bin sets: -1 binsA -2 binsB -3 binsC
Add folder/bin name to contig name for binsA bins
Add folder/bin name to contig name for binsC bins
Add folder/bin name to contig name for binsA bins
Add folder/bin name to contig name for binsA bins
Add folder/bin name to contig name for binsB bins
Add folder/bin name to contig name for binsC bins
Add folder/bin name to contig name for binsB bins
Add folder/bin name to contig name for binsB bins
Combine all bins together
Add folder/bin name to contig name for binsC bins
Combine all bins together
Combine all bins together
The number of refined bins: 0
Exporting refined bins...
Combine all bins together
Deleting temporary files
The number of refined bins: 0
All done!
Exporting refined bins...
The number of refined bins: 33
Deleting temporary files
Exporting refined bins...
All done!
The number of refined bins: 0
Exporting refined bins...
Deleting temporary files
All done!
Extracting refined bin: Refined_33.fasta
Deleting temporary files
All done!
------------------------------------------------------------------------------------------------------------------------
----- Bin refinement finished successfully! -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- fixing bin naming to .fa convention for consistancy... -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### RUNNING CHECKM ON ALL SETS OF BINS #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- Running CheckM on binsA bins -----
------------------------------------------------------------------------------------------------------------------------
*******************************************************************************
[CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************
Identifying marker genes in 41 bins with 96 threads:
Finished processing 41 of 41 (100.00%) bins.
Saving HMM info to file.
Calculating genome statistics for 41 bins with 96 threads:
Finished processing 41 of 41 (100.00%) bins.
Extracting marker genes to align.
Parsing HMM hits to marker genes:
Finished parsing hits for 41 of 41 (100.00%) bins.
Extracting 43 HMMs with 96 threads:
Finished extracting 43 of 43 (100.00%) HMMs.
Aligning 43 marker genes with 96 threads:
Finished aligning 43 of 43 (100.00%) marker genes.
Reading marker alignment files.
Concatenating alignments.
Placing 41 bins into the genome tree with pplacer (be patient).
{ Current stage: 0:45:15.447 || Total: 0:45:15.447 }
*******************************************************************************
[CheckM - lineage_set] Inferring lineage-specific marker sets.
*******************************************************************************
Reading HMM info from file.
Parsing HMM hits to marker genes:
Finished parsing hits for 41 of 41 (100.00%) bins.
Determining marker sets for each genome bin.
Finished processing 41 of 41 (100.00%) bins (current: bin.26).
Marker set written to: binsA.checkm/lineage.ms
{ Current stage: 0:00:17.557 || Total: 0:45:33.005 }
*******************************************************************************
[CheckM - analyze] Identifying marker genes in bins.
*******************************************************************************
Identifying marker genes in 41 bins with 96 threads:
Finished processing 41 of 41 (100.00%) bins.
Saving HMM info to file.
{ Current stage: 0:09:51.599 || Total: 0:55:24.605 }
Parsing HMM hits to marker genes:
Finished parsing hits for 41 of 41 (100.00%) bins.
Aligning marker genes with multiple hits in a single bin:
Finished processing 41 of 41 (100.00%) bins.
{ Current stage: 0:00:24.292 || Total: 0:55:48.897 }
Calculating genome statistics for 41 bins with 96 threads:
Finished processing 41 of 41 (100.00%) bins.
{ Current stage: 0:00:03.498 || Total: 0:55:52.396 }
*******************************************************************************
[CheckM - qa] Tabulating genome statistics.
*******************************************************************************
Calculating AAI between multi-copy marker genes.
Reading HMM info from file.
Parsing HMM hits to marker genes:
Finished parsing hits for 41 of 41 (100.00%) bins.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Bin Id Marker lineage # genomes # markers # marker sets 0 1 2 3 4 5+ Completeness Contamination Strain heterogeneity
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
bin.unbinned root (UID1) 5656 56 24 0 0 0 1 1 54 100.00 905.48 1.22
bin.3 p__Firmicutes (UID1022) 100 293 157 6 283 4 0 0 0 97.27 0.64 0.00
bin.38 k__Bacteria (UID203) 5449 103 58 2 29 60 12 0 0 96.55 87.71 51.04
bin.1 k__Bacteria (UID203) 5449 104 58 2 82 20 0 0 0 96.55 15.52 20.00
bin.37 k__Bacteria (UID1452) 924 161 108 9 151 0 1 0 0 93.36 0.37 0.00
bin.18 p__Firmicutes (UID239) 1324 175 101 11 152 1 11 0 0 90.10 0.79 32.35
bin.22 c__Clostridia (UID1118) 387 223 124 19 186 18 0 0 0 89.80 4.96 16.67
bin.8 p__Firmicutes (UID241) 930 213 118 23 176 13 1 0 0 88.42 8.33 12.50
bin.11 p__Firmicutes (UID241) 930 213 118 25 183 5 0 0 0 87.57 3.25 80.00
bin.21 k__Bacteria (UID2495) 2993 143 89 14 129 0 0 0 0 86.41 0.00 0.00
bin.40 p__Actinobacteria (UID1454) 732 206 120 29 162 14 1 0 0 84.93 6.81 41.18
bin.25 p__Firmicutes (UID241) 930 213 118 33 174 6 0 0 0 81.75 2.64 16.67
bin.10 p__Firmicutes (UID241) 930 213 118 30 174 8 1 0 0 81.07 4.87 0.00
bin.9 p__Firmicutes (UID239) 1324 175 101 29 137 9 0 0 0 80.74 4.49 44.44
bin.7 o__Burkholderiales (UID4000) 193 427 214 74 332 21 0 0 0 80.10 5.82 71.43
bin.5 p__Euryarchaeota (UID49) 95 228 153 38 187 3 0 0 0 78.27 0.78 0.00
bin.28 c__Deltaproteobacteria (UID3216) 83 247 155 50 189 8 0 0 0 76.73 2.84 25.00
bin.13 p__Actinobacteria (UID1454) 732 206 120 48 152 5 1 0 0 75.35 5.83 25.00
bin.29 k__Bacteria (UID3187) 2258 181 110 39 129 13 0 0 0 74.50 6.65 69.23
bin.16 k__Bacteria (UID203) 5449 104 58 19 49 33 3 0 0 72.41 22.16 69.05
bin.20 k__Bacteria (UID1452) 924 151 101 33 115 3 0 0 0 71.93 2.48 66.67
bin.2 k__Bacteria (UID203) 5449 104 58 31 73 0 0 0 0 69.80 0.00 0.00
bin.36 p__Firmicutes (UID241) 930 213 118 67 146 0 0 0 0 68.01 0.00 0.00
bin.17 p__Euryarchaeota (UID3) 148 188 125 66 113 8 1 0 0 62.85 6.00 0.00
bin.19 k__Bacteria (UID203) 5449 104 58 27 32 20 19 5 1 62.04 52.58 15.38
bin.6 p__Euryarchaeota (UID3) 148 188 125 69 116 3 0 0 0 58.13 2.00 0.00
bin.4 k__Bacteria (UID1452) 924 151 101 54 89 8 0 0 0 55.98 6.44 12.50
bin.30 k__Bacteria (UID1452) 924 151 101 53 96 2 0 0 0 55.98 1.49 50.00
bin.26 k__Bacteria (UID2569) 434 278 186 123 149 6 0 0 0 55.33 2.71 33.33
bin.12 k__Bacteria (UID2565) 2921 152 93 59 93 0 0 0 0 53.62 0.00 0.00
bin.14 c__Clostridia (UID1085) 35 420 196 193 213 14 0 0 0 52.39 2.64 0.00
bin.27 k__Bacteria (UID1453) 901 171 117 63 106 2 0 0 0 52.32 0.16 100.00
bin.35 k__Bacteria (UID203) 5449 104 58 57 38 8 1 0 0 43.04 9.95 0.00
bin.32 p__Euryarchaeota (UID3) 148 188 125 111 77 0 0 0 0 40.83 0.00 0.00
bin.23 o__Burkholderiales (UID4000) 193 427 214 244 165 15 3 0 0 36.61 5.02 12.50
bin.31 k__Archaea (UID2) 207 149 107 104 44 1 0 0 0 35.05 0.47 100.00
bin.34 k__Bacteria (UID2495) 2993 147 91 109 38 0 0 0 0 29.90 0.00 0.00
bin.33 k__Bacteria (UID203) 5449 104 58 86 18 0 0 0 0 23.04 0.00 0.00
bin.39 k__Bacteria (UID2495) 2993 143 89 111 31 1 0 0 0 19.63 1.12 100.00
bin.15 k__Bacteria (UID203) 5449 102 56 87 15 0 0 0 0 9.99 0.00 0.00
bin.24 root (UID1) 5656 56 24 56 0 0 0 0 0 0.00 0.00 0.00
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
{ Current stage: 0:00:11.678 || Total: 0:56:04.074 }
------------------------------------------------------------------------------------------------------------------------
----- There are 27 'good' bins found in binsA! (>50% completion and <10% contamination) -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Running CheckM on binsAB bins -----
------------------------------------------------------------------------------------------------------------------------
*******************************************************************************
[CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************
[Error] No bins found. Check the extension (-x) used to identify bins.
Controlled exit resulting from an unrecoverable error or warning.
************************************************************************************************************************
***** Something went wrong with running CheckM. Exiting... *****
************************************************************************************************************************
error in metaWRAP binning command
I have this error message after the metaWRAP binning command
.......
------------------------------------------------------------------------------------------------------------------------
----- Aligning AC3_2MetaGenMerged_paired_1.fastq and AC3_2MetaGenMerged_paired_2.fastq back to -----
----- assembly, sorting the alignment, and gathering statistics on insert lengths -----
------------------------------------------------------------------------------------------------------------------------
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 9577972 sequences (960000077 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (107, 114, 93, 97)
[M::mem_pestat] analyzing insert size distribution for orientation FF...
[M::mem_pestat] (25, 50, 75) percentile: (648, 2386, 5914)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 16446)
[M::mem_pestat] mean and std.dev: (3388.33, 3002.29)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 21712)
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (456, 3134, 6082)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 17334)
[M::mem_pestat] mean and std.dev: (3338.57, 2922.67)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 22960)
[M::mem_pestat] analyzing insert size distribution for orientation RF...
[M::mem_pestat] (25, 50, 75) percentile: (517, 2597, 5356)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 15034)
[M::mem_pestat] mean and std.dev: (3399.18, 2976.25)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 19873)
[M::mem_pestat] analyzing insert size distribution for orientation RR...
[M::mem_pestat] (25, 50, 75) percentile: (1056, 2708, 6118)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 16242)
[M::mem_pestat] mean and std.dev: (3590.84, 2932.69)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 21304)
[mem_sam_pe] paired reads have different names: "D4ZHLFP1:60:D2HDNACXX:2:1101:1279:2201", "D4ZHLFP1:60:D2HDNACXX:2:1311:3692:20373"
awk: cmd. line:1: (FILENAME=- FNR=78735) fatal: division by zero attempted
samtools sort: couldn't allocate memory for bam_mem
************************************************************************************************************************
***** Something went wrong with aligning/sorting the reads to the assembly! *****
************************************************************************************************************************
metawrap binning bwa mem error
Hello,
I am using metawrap binning after having already assembled the contigs using another program.
I have been using paired-end reads file, so I deinterleaved the file before running metawrap binning step. However I get the following log:
----- Aligning reads_1.fastq and reads_2.fastq back to assembly, sorting the alignment, and -----
----- gathering statistics on insert lengths -----
------------------------------------------------------------------------------------------------------------------------
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 2317434 sequences (310000090 bp)...
[M::process] read 2322074 sequences (310000257 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 2317434 reads in 1669.619 CPU sec, 119.478 real sec
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::process] read 2319900 sequences (310000013 bp)...
[M::mem_process_seqs] Processed 2322074 reads in 1278.945 CPU sec, 53.233 real sec
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::process] read 2332844 sequences (310000136 bp)...
[M::mem_process_seqs] Processed 2319900 reads in 1193.532 CPU sec, 45.801 real sec
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::process] read 2315030 sequences (310000082 bp)...
[M::mem_process_seqs] Processed 2332844 reads in 1281.751 CPU sec, 52.724 real sec
[M::process] read 2318854 sequences (310000047 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_process_seqs] Processed 2315030 reads in 1230.817 CPU sec, 47.844 real sec
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::process] read 2322620 sequences (310000139 bp)...
[M::mem_process_seqs] Processed 2318854 reads in 1241.409 CPU sec, 47.085 real sec
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (0, 0, 0, 0)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] skip orientation FR as there are not enough pairs
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::process] read 2321694 sequences (310000112 bp)..`
and so on ..
Looking at the heads of the reads files, it looks like they are synced properly:
(metawrap-env) [yunha@smsx10srw-srcf-d15-35 7.metawrap_binning]$ head reads_1.fastq
@E00558:228:HMVT5CCXY:2:1103:9820:60870
AAAAAAATTATAATCTGTTGTTGATTAATCCCTGGATTTATGATTTTACGGCTTATGATTTCTGGTCTAAACCCTTAGGGCTATTATATATTGCTAGTATCTTGAAGGAACAGGGTTATGAGATTAG
+
JJJJJJJJJJJJJJJJJFJJJJJJJJJJJAJJJJJJJJJJJJJJJFJJJFJJJJJJFJJJJJJJJJJJJJJFFJFJJJJJJJJJJJJJJJJJJJF7JJJJFJJ<A77AFJAAA7F-AF<F<AAFF<F
@E00558:228:HMVT5CCXY:2:2124:13372:69133
GCTACCGGCAGGAGTGGAGCAGGAATTAGACAGCAATTTGGGACTGAGATGAATTGTATCCTGGCCCGAGAAAGTATAAAAATATTTGAGAATTTAAGTCAAGAATTGGATTATGATATCGAACTGA
+
JJJJJJJJJJJJ<JJJJJJJFJJJJJJJJJJFJFJJJJJJJJFJFJFJAJJJJJJJJJJJJJJJJJJFFJFJJFFJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJFJFFJJFFJFJFFJFJAAFJ
@E00558:228:HMVT5CCXY:2:1207:26230:43519
GGTAAACTCTAGGTAGGCACAACCTGGCGTTGTGCCCCTCACAACCTGTGGTTGTACTAATTAAATAAGATTAATTAAATAAGTGCAGAATTCC
(metawrap-env) [yunha@smsx10srw-srcf-d15-35 7.metawrap_binning]$ head reads_2.fastq
@E00558:228:HMVT5CCXY:2:1103:9820:60870
AATTATCTCAATCGCCCTGAACACTCCGAGATACCAGTAGGTCATAATAGAGGTAATCAATACT
+
AAAFFFJJAJJJFFJFJJJJJJJJJJJFFJJJJJAJFJFJF7JFJJ-J-7J-<J7FF<FJFF<J
@E00558:228:HMVT5CCXY:2:2124:13372:69133
CCTTTACGGTTATGTATCTTACATCAATCCCTAATGATTGTTCTAAAGCCACATTTTTCTTAAACTGGTTGACTTCTTTTTCGGTATAAGCCAGCATAAGATATCCGCCTTGATTCAGTTCGATATCATAATCCAATTCTTGACTTAAAT
+
AAFFFJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJFJJJJJJJJJJJJJJJJJFJJJJJJJJJJJJJJJJJJJJJJJJJJJJJFFJJJJJJJ
@E00558:228:HMVT5CCXY:2:1207:26230:43519
ATGGAATTCTGCACTTATTTAATTAATCTTATTTAATTAGTACAACCACAGGTTGTGAGGGGCACAACGCCAGGTTGTGCCTACCTAGAGTTTACCAGCTAATCCCTTAAAGGTTTTTTA
Any idea what is going on here?
Thanks!
error in metaWRAP read_qc --skip-bmtagger
Hi
I'm trying to run the metaWRAP read_qc --skip-bmtagger command, but it gives this error:
************************************************************************************************************************
***** /home/igor/metaWRAP/BMTAGGER_INDEX/hg38.bitmask file doesnt exist. Please configure your *****
***** bmtagger human genome index *****
************************************************************************************************************************
the flag --skip-bmtagger command seems to be not working.
Other question is, is it possible to use a bam file, previously produced to do the metawrap binning command? I'm trying to do the analyses on a 24 GB computer and it seems to go out of memory, but I was able to produce a bam file in the same pc using bam...
Thanks for any help.
unable to start device 'png'
Hi
At the end of the blobology module I got the following error:
Something went wrong with making the order bloblot image. Exiting
It seems the problem is from the makeblobplot.R part. When I tried to run it like this:
makeblobplot.R test.blobplot 0.005 taxlevel_order
I got the following error.
Error in png(paste(arg_input_file, taxlevel, "png", sep = "."), (numcols * :
unable to start device 'png'
In addition: Warning message:
In png(paste(arg_input_file, taxlevel, "png", sep = "."), (numcols * :
cairo error 'invalid value (typically too big) for the size of the input (surface, pattern, etc.)'
Execution halted
I installed metawrap and all dependency using the conda.
Any comments on how to solve this?
Regards
Mahdi
MetaWRAP cant find perl5 libraries for Binning
This is a question that was privately emailed to me, but I would like to put this discussion on here in case anyone else has similar issues. I want to keep the development as transparent as possible!
"first of, I would like to thank you for setting up metawrap, it is currently the easisest to use and setup environment for metagenome analysis.
I got stuck in the binning process. It seems that maxbin2 is missing the following:
----- split master contig depth file into individual files for maxbin2 input -----
------------------------------------------------------------------------------------------------------------------------
processing final_pure_reads.bam depth file...
************************************************************************************************************************
***** /usr/miniconda2/lib/perl5/site_perl/5.22.0 does not exixt. Cannot set manual path to *****
***** perl5 libraries. Exiting... *****
************************************************************************************************************************
I have tried to work around by copying
perl5/site_perl/5.22.0 into /usr/miniconda2/lib/
however, without success.
What is the syntax to define the path to the perl5 libraries in config-metwrap?"
Binning with single end reads
Dear German,
I tried to run the binning module with several single end reads but it gave an error.
Is there any possibility to run this module with single end reads?
Error at the end of Bin_refinement module
I am getting an error at the end of the Bin_refinement pipeline.
When I run...
metawrap bin_refinement -o BIN_REFINEMENT -t 56 -A INITIAL_BINNING/metabat2_bins/ -B INITIAL_BINNING/maxbin2_bins/ -C INITIAL_BINNING/concoct_bins/ -c 50 -x 10
I get ...
------------------------------------------------------------------------------------------------------------------------
----- There are 14 'good' bins found in binsO.checkm! (>50% completion and <10% contamination) -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Removing bins that are inadequate quality... -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Re-evaluating bin quality after contig de-replication is complete! There are still 14 -----
----- high quality bins. -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- making completion and contamination ranking plots for all refinement iterations -----
------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/a1654837/.conda/envs/metawrap-env/bin/metawrap-scripts/plot_binning_results.py", line 6, in <module>
import matplotlib.pyplot as plt
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/pyplot.py", line 114, in <module>
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
globals(),locals(),[backend_name],0)
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/backend_qt4agg.py", line 18, in <module>
from .backend_qt5agg import FigureCanvasQTAggBase as _FigureCanvasQTAggBase
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5agg.py", line 15, in <module>
from .backend_qt5 import QtCore
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5.py", line 31, in <module>
from .qt_compat import QtCore, QtGui, QtWidgets, _getSaveFileName, __version__
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/qt_compat.py", line 124, in <module>
from PyQt4 import QtCore, QtGui
ImportError: libXrender.so.1: cannot open shared object file: No such file or directory
mv: cannot stat `binning_results.eps': No such file or directory
mv: cannot stat `binning_results.png': No such file or directory
------------------------------------------------------------------------------------------------------------------------
----- Finalizing bin set naming, and moving over intermediate files into 'work_files' -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- making completion and contamination ranking plots of final outputs -----
------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/a1654837/.conda/envs/metawrap-env/bin/metawrap-scripts/plot_binning_results.py", line 6, in <module>
import matplotlib.pyplot as plt
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/pyplot.py", line 114, in <module>
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
globals(),locals(),[backend_name],0)
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/backend_qt4agg.py", line 18, in <module>
from .backend_qt5agg import FigureCanvasQTAggBase as _FigureCanvasQTAggBase
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5agg.py", line 15, in <module>
from .backend_qt5 import QtCore
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5.py", line 31, in <module>
from .qt_compat import QtCore, QtGui, QtWidgets, _getSaveFileName, __version__
File "/home/a1654837/.conda/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/backends/qt_compat.py", line 124, in <module>
from PyQt4 import QtCore, QtGui
ImportError: libXrender.so.1: cannot open shared object file: No such file or directory
mv: cannot stat `binning_results.eps': No such file or directory
mv: cannot stat `binning_results.png': No such file or directory
########################################################################################################################
##### BIN_REFINEMENT PIPELINE FINISHED SUCCESSFULLY! #####
######################################################################################################################## ```
Not find the Binning_refiner.stats as mentioned in Usage_tutorial.md
Hi, I am checking the data from Bin_refinement module. But I did not find the Binning_refiner.stats, Binning_refiner as mentioned in Usage_tutorial.md. But others are all there. And there are two empty bins in the metaWRAP_bins, which I think should be removed. let me know if this can be an issue.
Thanks,
Binning module not working properly
When I run:
metawrap /home-2/[email protected]/scratch/metaWRAP/pipelines/binning -a AS.contigs.fasta -o AS_binning --metabat2 /home-2/[email protected]/data/BAC_Metagenomes/CLEAN_READS/AS_1.fastq /home-2/[email protected]/data/BAC_Metagenomes/CLEAN_READS/AS_2.fastq
I get the error:
------------------------------------------------------------------------------------------------------------------------
----- 1 forward and 1 reverse read files detected -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### ALIGNING READS TO MAKE COVERAGE FILES #####
########################################################################################################################
mkdir: cannot create directory `AS_binning': File exists
mkdir: cannot create directory `AS_binning/work_files': File exists
------------------------------------------------------------------------------------------------------------------------
----- making copy of assembly file AS.contigs.fasta -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Indexing assembly file -----
------------------------------------------------------------------------------------------------------------------------
[bwa_index] Pack FASTA... 0.21 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 8.00 seconds elapse.
[bwa_index] Update BWT... 0.16 sec
[bwa_index] Pack forward-only FASTA... 0.14 sec
[bwa_index] Construct SA from BWT and Occ... 2.98 sec
[main] Version: 0.7.15-r1140
[main] CMD: bwa index AS_binning/work_files/assembly.fa
[main] Real time: 11.600 sec; CPU: 11.517 sec
------------------------------------------------------------------------------------------------------------------------
----- Aligning /home-2/[email protected]/data/BAC_Metagenomes/CLEAN_READS/AS_1.fastq and -----
----- /home-2/[email protected]/data/BAC_Metagenomes/CLEAN_READS/AS_2.fastq back to assembly, -----
----- sorting the alignment, and gathering statistics on insert lengths -----
------------------------------------------------------------------------------------------------------------------------
sort: invalid option -- 'T'
sort: invalid option -- 'O'
open: No such file or directory
[bam_sort_core] fail to open file AS_binning/work_files/tmp-samtools
[M::bwa_idx_load_from_disk] read 0 ALT contigs
[M::process] read 55292 sequences (10000106 bp)...
[M::process] read 54180 sequences (10000381 bp)...
[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (20, 24564, 34, 27)
[M::mem_pestat] analyzing insert size distribution for orientation FF...
[M::mem_pestat] (25, 50, 75) percentile: (156, 261, 354)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 750)
[M::mem_pestat] mean and std.dev: (221.41, 101.89)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 948)
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (232, 313, 409)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 763)
[M::mem_pestat] mean and std.dev: (326.34, 136.61)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 940)
[M::mem_pestat] analyzing insert size distribution for orientation RF...
[M::mem_pestat] (25, 50, 75) percentile: (16, 30, 49)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 115)
[M::mem_pestat] mean and std.dev: (32.42, 23.78)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 148)
[M::mem_pestat] analyzing insert size distribution for orientation RR...
[M::mem_pestat] (25, 50, 75) percentile: (158, 247, 349)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 731)
[M::mem_pestat] mean and std.dev: (249.54, 136.95)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 922)
[M::mem_pestat] skip orientation FF
[M::mem_pestat] skip orientation RF
[M::mem_pestat] skip orientation RR
[M::mem_process_seqs] Processed 55292 reads in 5.132 CPU sec, 5.052 real sec
[samopen] SAM header is present: 2012 sequences.
########################################################################################################################
##### RUNNING METABAT2 #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- making contig depth file... -----
------------------------------------------------------------------------------------------------------------------------
Output depth matrix to AS_binning/work_files/metabat2_depth.txt
Output matrix to AS_binning/work_files/metabat2_depth.txt
Opening 1 bams
[E::hts_open] fail to open file 'AS_binning/work_files/*.bam'
Consolidating headers
/home-2/[email protected]/scratch/metaWRAP/pipelines/binning.sh: line 200: 139724 Segmentation fault jgi_summarize_bam_contig_depths --outputDepth ${out}/work_files/metabat2_depth.txt ${out}/work_files/*.bam
************************************************************************************************************************
***** Something went wrong with making contig depth file. Exiting. *****
************************************************************************************************************************
What may be the issue with this?
Thanks!
Error binning module
Hi,
I have run the binning module.
MetaBat2 run without problems and output around 200 bins, but both MaxBin2 and Concoct called for errors, see below the error file:
Creating depth matrix file: binning/work_files/mb2_master_depth.txt
Closing most bam files
Closing last bam file
Finished
readline() on closed filehandle FILE at /home/people/alpal/.conda/envs/metawrap-env/bin/run_MaxBin.pl line 1334.
cp: cannot stat 'binning/work_files/maxbin2_out/bin*.fasta': No such file or directoryOutput depth matrix to binning/work_files/tmp
Calculating intra contig depth variance
Output matrix to binning/work_files/tmp
Opening 24 bams
Consolidating headers
Processing bam files
Creating depth matrix file: binning/work_files/tmp
Closing most bam files
Closing last bam file
Finished
Traceback (most recent call last):
File "/home/people/alpal/.conda/envs/metawrap-env/bin/concoct", line 4, in <module>
__import__('pkg_resources').run_script('concoct==0.4.0', 'concoct')
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 748, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1517, in run_script
exec(code, namespace, namespace)
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/EGG-INFO/scripts/concoct", line 10, in <module>
from concoct.cluster import cluster
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/concoct-0.4.0-py2.7-linux-x86_64.egg/concoct/cluster.py", line 4, in <module>
from sklearn.mixture import GMM
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/sklearn/__init__.py", line 134, in <module>
from .base import clone
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/sklearn/base.py", line 11, in <module>
from scipy import sparse
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/scipy/sparse/__init__.py", line 229, in <module>
from .csr import *
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/scipy/sparse/csr.py", line 15, in <module>
from ._sparsetools import csr_tocsc, csr_tobsr, csr_count_blocks, \
ImportError: /home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/scipy/sparse/../../../../libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/scipy/sparse/_sparsetools.soIn the log file for MaxBin2, it says:
----- Starting binning with MaxBin2... -----
------------------------------------------------------------------------------------------------------------------------
Cannot run IDBA-UD. Please indicate the file directory in 'setting' file.
------------------------------------------------------------------------------------------------------------------------
----- MaxBin2 finished successfully, and found 0 bins! -----Both MaxBin2 and concoct abundance files seems to be fine:
head mb2_abund_list.txt
mb2_AAGAGGCATCGCATAA_merged_TM_paired.txt
mb2_ACTCGCTATCGCATAA_merged_TM_paired.txt
...
head mb2_AAGAGGCATCGCATAA_merged_TM_paired.txt
contig-1 53.4517
contig-2 1.2245
contig-3 18.5064
head concoct_depth.txt
contig-1 53.4517 0.103837 0.025839 0.21196 0.135306 0.112696 0.0226196 23.9448 0.0491105 0.0210677 0.0967119 0.082113 0.0973345 0.137699 0.088772 0.0479484 0.176484 0.129474 0.0901337 0.0602444 0.0725666 0.603433 0.131319 0.0282016
contig-2 1.2245 2.09769 1.36187 0.96547 1.05846 1.11671 5.11016 0.975225 0.149114 6.51347 5.00844 0.588666 3.05451 0.642295 0.947799 2.12208 1.30022 1.26318 1.13096 0.930848 1.14926 4.7246 1.36526 1.47942
contig-3 18.5064 0.194334 0.0958584 0.162577 0.224753 0.175527 0.103175 18.4421 0.0364323 0.0595865 0.479316 0.17967 0.20679 0.303299 0.164022 0.462451 0.129024 0.173942 0.190543 0.104959 0.118599 0.744656 0.907991 0.0647205
Do you know what could it be going on?
Thanks in advance.
ClobberError issues during installation - harmless
I got some messages about errors during installation of metaWRAP with Conda (as seen bellow). The errors are caused by a few conda dependencies of metaWRAP having the same binary files installed at the same time. For example, Taxator-tk comes with BLAST binaries, but so does the official BLAST installation, so they clash.
I realize this is not ideal, and I try to eliminate some of the redundancies. As far as I can tell, these errors are harmless, so feel free to ignore the errors. If you finds legitimate issues with metaWRAP functioning in relation to this, please comment so I can address it.
ClobberError: The package 'conda-forge::gsl-1.16-0' cannot be installed due to a
path collision for 'share/info/dir'.
This path already exists in the target prefix, and it won't be removed by
an uninstall action in this transaction. The path appears to be coming from
the package 'conda-forge::libgcc-ng-7.2.0-hdf63c60_3', which is already installed in the prefix.
ClobberError: The package 'conda-forge::libidn11-1.33-0' cannot be installed due to a
path collision for 'share/info/dir'.
This path already exists in the target prefix, and it won't be removed by
an uninstall action in this transaction. The path appears to be coming from
the package 'conda-forge::libgcc-ng-7.2.0-hdf63c60_3', which is already installed in the prefix.
ClobberError: The package 'conda-forge::readline-6.2-0' cannot be installed due to a
path collision for 'share/info/dir'.
This path already exists in the target prefix, and it won't be removed by
an uninstall action in this transaction. The path appears to be coming from
the package 'conda-forge::libgcc-ng-7.2.0-hdf63c60_3', which is already installed in the prefix.
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge::gsl-1.16-0, conda-forge::libidn11-1.33-0, conda-forge::readline-6.2-0
path: 'share/info/dir'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: bioconda::krona-2.7-pl526_2, ursky::taxator-tk-1.3.3e-0
path: 'bin/ktGetLibPath'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: bioconda::krona-2.7-pl526_2, ursky::taxator-tk-1.3.3e-0
path: 'bin/ktImportText'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: bioconda::parallel-20160622-1, ursky::taxator-tk-1.3.3e-0
path: 'bin/parallel'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: bioconda::blast-2.7.1-boost1.64_1, ursky::taxator-tk-1.3.3e-0
path: 'bin/blastn'
ClobberError: This transaction has incompatible packages due to a shared path.
packages: bioconda::blast-2.7.1-boost1.64_1, ursky::taxator-tk-1.3.3e-0
path: 'bin/makeblastdb'
Error in bin refinement module
Hi,
sorry for late reply in the previous issue, I was on holidays.
Now concoct seems to work fine, however there is a problem when doing the bin refinement.
This is what I get in the bin_refinement folder:
1644 Jun 27 20:03 binsA
6470 Jun 27 20:03 binsB
3418 Jun 27 20:03 binsC
7124 Jun 27 21:04 binsAB
4340 Jun 27 21:04 binsABC
5588 Jun 27 21:04 binsAC
5396 Jun 27 21:04 binsBC
0 Jun 27 21:04 binsA.tmp
And here the error:
Traceback (most recent call last):
File "/home/people/alpal/.conda/envs/metawrap-env/bin/checkm", line 36, in <module>
from checkm import main
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 25, in <module>
from checkm.defaultValues import DefaultValues
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/checkm/defaultValues.py", line 26, in <module>
class DefaultValues():
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/checkm/defaultValues.py", line 29, in DefaultValues
__DBM = DBManager()
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/checkm/checkmData.py", line 114, in __init__
if not self.setRoot():
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/checkm/checkmData.py", line 177, in setRoot
path = self.confirmPath(path=path)
File "/home/people/alpal/.conda/envs/metawrap-env/lib/python2.7/site-packages/checkm/checkmData.py", line 199, in confirmPath
path = raw_input("Where should CheckM store it's data?\n" \
EOFError: EOF when reading a line
########################################################################################################################
##### RUNNING CHECKM ON ALL SETS OF BINS #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- Running CheckM on binsA bins -----
------------------------------------------------------------------------------------------------------------------------
It seems that the CheckM data folder has not been set yet or has been removed. Running: 'checkm data setRoot'.
Where should CheckM store it's data?
Please specify a location or type 'abort' to stop trying:
************************************************************************************************************************
***** Something went wrong with running CheckM. Exiting... *****
************************************************************************************************************************
I guess I did not update the CheckM database...
Support for gzipped FASTQ reads files
Hi,
I was wondering metaWRAP could be modified to support gzipped FASTQ reads files. From what I can see, it appears that most of the programs used by metaWRAP already support this (trim_galore, MEGAHIT, metaSPAdes, etc.). Thanks!
Best,
Bryan
Issue running quant_bins (making heatmap with Seaborn)
Hi,
I've found the following error when try to "make heatmap with Seaborn" in quant_bins module:
Command:
metawrap quant_bins -t 48 -b bin_refinement_all/metaWRAP_bins -o quant_bins_all -a assembly_all/final_assembly.fasta trimmomatic_mh/P*
Output:
------------------------------------------------------------------------------------------------------------------
------
----- 9 forward and 9 reverse read files detected
-----
------------------------------------------------------------------------------------------------------------------
------
##################################################################################################################
######
##### SETTING UP OUTPUT AND INDEXING ASSEMBLY
#####
##################################################################################################################
######
------------------------------------------------------------------------------------------------------------------
------
----- Indexing assembly file with salmon. Ignore any warnings
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
index ["quant_bins_all/assembly_index"] did not previously exist . . . creating it
[2018-09-28 13:04:25.881] [jLog] [info] building index
RapMap Indexer
[Step 1 of 4] : counting k-mers
[2018-09-28 13:04:25.981] [jointLog] [warning] Entry with header [NODE_1_length_473967_cov_14.670517] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:25.997] [jointLog] [warning] Entry with header [NODE_2_length_468876_cov_10.781396] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.012] [jointLog] [warning] Entry with header [NODE_3_length_457119_cov_12.974187] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.027] [jointLog] [warning] Entry with header [NODE_4_length_445486_cov_15.241851] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.043] [jointLog] [warning] Entry with header [NODE_5_length_444045_cov_15.168857] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.056] [jointLog] [warning] Entry with header [NODE_6_length_417663_cov_8.725199] was longer th
an 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.071] [jointLog] [warning] Entry with header [NODE_7_length_405658_cov_18.802842] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.084] [jointLog] [warning] Entry with header [NODE_8_length_390756_cov_6.348980] was longer th
an 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.096] [jointLog] [warning] Entry with header [NODE_9_length_378840_cov_15.887049] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.111] [jointLog] [warning] Entry with header [NODE_10_length_370522_cov_7.786999] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.122] [jointLog] [warning] Entry with header [NODE_11_length_349221_cov_10.086798] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.133] [jointLog] [warning] Entry with header [NODE_12_length_325204_cov_32.530707] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.143] [jointLog] [warning] Entry with header [NODE_13_length_308193_cov_34.368601] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.153] [jointLog] [warning] Entry with header [NODE_14_length_307617_cov_22.357645] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.166] [jointLog] [warning] Entry with header [NODE_15_length_284254_cov_8.260831] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.175] [jointLog] [warning] Entry with header [NODE_16_length_281858_cov_14.636118] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.184] [jointLog] [warning] Entry with header [NODE_17_length_267102_cov_15.598119] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.192] [jointLog] [warning] Entry with header [NODE_18_length_259202_cov_34.374637] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.200] [jointLog] [warning] Entry with header [NODE_19_length_251337_cov_3.969091] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.208] [jointLog] [warning] Entry with header [NODE_20_length_249924_cov_11.292621] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.216] [jointLog] [warning] Entry with header [NODE_21_length_249550_cov_36.618441] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.224] [jointLog] [warning] Entry with header [NODE_22_length_248299_cov_35.847448] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.232] [jointLog] [warning] Entry with header [NODE_23_length_247289_cov_8.208248] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.240] [jointLog] [warning] Entry with header [NODE_24_length_243878_cov_8.937360] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.254] [jointLog] [warning] Entry with header [NODE_25_length_243469_cov_7.771550] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.261] [jointLog] [warning] Entry with header [NODE_26_length_206742_cov_4.478317] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.267] [jointLog] [warning] Entry with header [NODE_27_length_206428_cov_16.668057] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.274] [jointLog] [warning] Entry with header [NODE_28_length_206329_cov_12.448583] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.280] [jointLog] [warning] Entry with header [NODE_29_length_206304_cov_15.781468] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.287] [jointLog] [warning] Entry with header [NODE_30_length_205493_cov_15.385771] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.294] [jointLog] [warning] Entry with header [NODE_31_length_204493_cov_7.767979] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.300] [jointLog] [warning] Entry with header [NODE_32_length_203487_cov_33.184155] was longer
than 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
[2018-09-28 13:04:26.307] [jointLog] [warning] Entry with header [NODE_33_length_200060_cov_8.786300] was longer t
han 200000 nucleotides. Are you certain that we are indexing a transcriptome and not a genome?
counted k-mers for 120000 transcripts
counted k-mers for 200000 transcriptsElapsed time: 22.0589s
Replaced 1547532 non-ATCG nucleotides
Clipped poly-A tails from 9 transcripts
Building rank-select dictionary and saving to disk done
Elapsed time: 0.0357691s
Writing sequence data to file . . . done
Elapsed time: 0.327935s
[info] Building 32-bit suffix array (length of generalized text is 621818567)
Building suffix array . . . success
saving to disk . . . done
Elapsed time: 1.29636s
done
Elapsed time: 85.2951s
processed 15000000 positions
processed 30000000 positions
processed 45000000 positions
processed 60000000 positions
processed 76000000 positions
processed 91000000 positions
processed 106000000 positions
processed 120000000 positions
processed 135000000 positions
processed 150000000 positions
processed 165000000 positions
processed 179000000 positions
processed 194000000 positions
processed 209000000 positions
processed 223000000 positions
processed 238000000 positions
processed 253000000 positions
processed 268000000 positions
processed 282000000 positions
processed 297000000 positions
processed 312000000 positions
processed 326000000 positions
processed 341000000 positions
processed 356000000 positions
processed 370000000 positions
processed 385000000 positions
processed 400000000 positions
processed 415000000 positions
processed 429000000 positions
processed 444000000 positions
processed 459000000 positions
processed 4
processed 488000000 positions
processed 503000000 positions
processed 518000000 positions
processed 532000000 positions
processed 547000000 positions
processed 562000000 positions
processed 576000000 positions
processed 591000000 positions
processed 606000000 positions
processed 621000000 positions
0000 positions
khash had 610359776 keys
saving hash to disk . . . done
Elapsed time: 112.633s
[2018-09-28 13:28:15.438] [jLog] [info] done building index
##################################################################################################################
######
##### ALIGNING READS FROM ALL SAMPLES BACK TO BINS WITH SALMON
#####
##################################################################################################################
######
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P1-1_trim_mh_PE with reads trimmomatic_mh/P1-1_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P1-1_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P1-1_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P1-1_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P1-1_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P1-1_trim_mh_PE.quant/logs
[2018-09-28 13:28:27.535] [jointLog] [info] parsing read library format
[2018-09-28 13:28:27.535] [jointLog] [info] There is 1 library.
[2018-09-28 13:28:27.690] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:28:27.689] [jointLog] [info] Loading Quasi index
[2018-09-28 13:28:27.690] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:28:31.798] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:28:32.641] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:28:32.926] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:28:33.030] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:28:33.031] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:29:52.339] [stderrLog] [info] Done loading index
[2018-09-28 13:29:52.339] [jointLog] [info] done
[2018-09-28 13:29:52.339] [jointLog] [info] Index contained 209578 targets
processed 11000000 fragments
hits: 6462669, hits per frag: 0.597948
[2018-09-28 13:30:44.029] [jointLog] [info] Computed 208456 rich equivalence classes for further processing
[2018-09-28 13:30:44.029] [jointLog] [info] Counted 6490323 total reads in the equivalence classes
[2018-09-28 13:30:44.115] [jointLog] [info] Mapping rate = 58.043%
[2018-09-28 13:30:44.115] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:30:44.116] [jointLog] [info] Starting optimizer
[2018-09-28 13:30:44.279] [jointLog] [info] Marked 1 weighted equivalence classes as degenerate
[2018-09-28 13:30:44.285] [jointLog] [info] iteration = 0 | max rel diff. = 2.5237
[2018-09-28 13:30:44.761] [jointLog] [info] iteration = 100 | max rel diff. = 0.03118
[2018-09-28 13:30:45.160] [jointLog] [info] iteration = 187 | max rel diff. = 0.00518404
[2018-09-28 13:30:45.169] [jointLog] [info] Finished optimizer
[2018-09-28 13:30:45.169] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P1-2_trim_mh_PE with reads trimmomatic_mh/P1-2_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P1-2_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P1-2_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P1-2_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P1-2_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P1-2_trim_mh_PE.quant/logs
[2018-09-28 13:30:57.197] [jointLog] [info] parsing read library format
[2018-09-28 13:30:57.197] [jointLog] [info] There is 1 library.
[2018-09-28 13:30:57.353] [jointLog] [info] Loading Quasi index
[2018-09-28 13:30:57.353] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:30:57.353] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:31:02.484] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:31:03.373] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:31:03.675] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:31:03.800] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:31:03.800] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:32:20.343] [jointLog] [info] done
[2018-09-28 13:32:20.343] [jointLog] [info] Index contained 209578 targets
[2018-09-28 13:32:20.343] [stderrLog] [info] Done loading index
processed 10500000 fragments
hits: 6137723, hits per frag: 0.596444
[2018-09-28 13:33:09.658] [jointLog] [info] Computed 208801 rich equivalence classes for further processing
[2018-09-28 13:33:09.658] [jointLog] [info] Counted 6099600 total reads in the equivalence classes
[2018-09-28 13:33:09.746] [jointLog] [info] Mapping rate = 57.7531%
[2018-09-28 13:33:09.746] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:33:09.746] [jointLog] [info] Starting optimizer
[2018-09-28 13:33:09.917] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-28 13:33:09.927] [jointLog] [info] iteration = 0 | max rel diff. = 1.96239
[2018-09-28 13:33:10.508] [jointLog] [info] iteration = 100 | max rel diff. = 0.0324912
[2018-09-28 13:33:10.953] [jointLog] [info] iteration = 200 | max rel diff. = 0.0284976
[2018-09-28 13:33:11.177] [jointLog] [info] iteration = 242 | max rel diff. = 0.00502761
[2018-09-28 13:33:11.188] [jointLog] [info] Finished optimizer
[2018-09-28 13:33:11.188] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P1-3_trim_mh_PE with reads trimmomatic_mh/P1-3_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P1-3_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P1-3_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P1-3_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P1-3_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P1-3_trim_mh_PE.quant/logs
[2018-09-28 13:33:24.883] [jointLog] [info] parsing read library format
[2018-09-28 13:33:24.883] [jointLog] [info] There is 1 library.
[2018-09-28 13:33:25.038] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:33:25.038] [jointLog] [info] Loading Quasi index
[2018-09-28 13:33:25.038] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:33:28.147] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:33:28.847] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:33:29.083] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:33:29.174] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:33:29.175] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:33:59.524] [stderrLog] [info] Done loading index
[2018-09-28 13:33:59.524] [jointLog] [info] done
[2018-09-28 13:33:59.524] [jointLog] [info] Index contained 209578 targets
processed 10500000 fragments
hits: 6279147, hits per frag: 0.606834
[2018-09-28 13:34:21.477] [jointLog] [info] Computed 209357 rich equivalence classes for further processing
[2018-09-28 13:34:21.477] [jointLog] [info] Counted 6447403 total reads in the equivalence classes
[2018-09-28 13:34:21.567] [jointLog] [info] Mapping rate = 59.0754%
[2018-09-28 13:34:21.567] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:34:21.567] [jointLog] [info] Starting optimizer
[2018-09-28 13:34:21.723] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-28 13:34:21.734] [jointLog] [info] iteration = 0 | max rel diff. = 1.31483
[2018-09-28 13:34:22.416] [jointLog] [info] iteration = 100 | max rel diff. = 0.0400291
[2018-09-28 13:34:23.054] [jointLog] [info] iteration = 200 | max rel diff. = 0.0166825
[2018-09-28 13:34:23.523] [jointLog] [info] iteration = 300 | max rel diff. = 0.0167376
[2018-09-28 13:34:24.073] [jointLog] [info] iteration = 395 | max rel diff. = 0.0046768
[2018-09-28 13:34:24.084] [jointLog] [info] Finished optimizer
[2018-09-28 13:34:24.084] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P2-1_trim_mh_PE with reads trimmomatic_mh/P2-1_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P2-1_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P2-1_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P2-1_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P2-1_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P2-1_trim_mh_PE.quant/logs
[2018-09-28 13:34:35.153] [jointLog] [info] parsing read library format
[2018-09-28 13:34:35.153] [jointLog] [info] There is 1 library.
[2018-09-28 13:34:35.311] [jointLog] [info] Loading Quasi index
[2018-09-28 13:34:35.311] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:34:35.311] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:34:40.032] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:34:40.979] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:34:41.292] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:34:41.418] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:34:41.419] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:36:05.097] [jointLog] [info] done
[2018-09-28 13:36:05.097] [jointLog] [info] Index contained 209578 targets
[2018-09-28 13:36:05.097] [stderrLog] [info] Done loading index
processed 10000000 fragments
hits: 5454328, hits per frag: 0.572538
[2018-09-28 13:36:56.415] [jointLog] [info] Computed 220377 rich equivalence classes for further processing
[2018-09-28 13:36:56.415] [jointLog] [info] Counted 5533306 total reads in the equivalence classes
[2018-09-28 13:36:56.522] [jointLog] [info] Mapping rate = 53.8543%
[2018-09-28 13:36:56.522] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:36:56.522] [jointLog] [info] Starting optimizer
[2018-09-28 13:36:56.717] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-28 13:36:56.726] [jointLog] [info] iteration = 0 | max rel diff. = 1.76942
[2018-09-28 13:36:57.259] [jointLog] [info] iteration = 100 | max rel diff. = 0.0445611
[2018-09-28 13:36:57.298] [jointLog] [info] iteration = 111 | max rel diff. = 0.00651675
[2018-09-28 13:36:57.316] [jointLog] [info] Finished optimizer
[2018-09-28 13:36:57.316] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P2-2_trim_mh_PE with reads trimmomatic_mh/P2-2_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P2-2_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P2-2_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P2-2_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P2-2_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P2-2_trim_mh_PE.quant/logs
[2018-09-28 13:37:09.732] [jointLog] [info] parsing read library format
[2018-09-28 13:37:09.732] [jointLog] [info] There is 1 library.
[2018-09-28 13:37:09.879] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:37:09.879] [jointLog] [info] Loading Quasi index
[2018-09-28 13:37:09.879] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:37:14.720] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:37:15.707] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:37:16.021] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:37:16.147] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:37:16.148] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:38:33.846] [jointLog] [info] done
[2018-09-28 13:38:33.846] [jointLog] [info] Index contained 209578 targets
[2018-09-28 13:38:33.846] [stderrLog] [info] Done loading index
processed 9500000 fragments
hits: 5189923, hits per frag: 0.569149
[2018-09-28 13:39:20.358] [jointLog] [info] Computed 220200 rich equivalence classes for further processing
[2018-09-28 13:39:20.358] [jointLog] [info] Counted 5383830 total reads in the equivalence classes
[2018-09-28 13:39:20.723] [jointLog] [info] Mapping rate = 53.95%
[2018-09-28 13:39:20.723] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:39:20.724] [jointLog] [info] Starting optimizer
[2018-09-28 13:39:20.911] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-28 13:39:20.919] [jointLog] [info] iteration = 0 | max rel diff. = 1.29261
[2018-09-28 13:39:21.638] [jointLog] [info] iteration = 100 | max rel diff. = 0.0293809
[2018-09-28 13:39:22.212] [jointLog] [info] iteration = 200 | max rel diff. = 0.0297914
[2018-09-28 13:39:22.815] [jointLog] [info] iteration = 300 | max rel diff. = 0.0161888
[2018-09-28 13:39:23.368] [jointLog] [info] iteration = 400 | max rel diff. = 0.0161762
[2018-09-28 13:39:23.929] [jointLog] [info] iteration = 500 | max rel diff. = 0.010421
[2018-09-28 13:39:24.106] [jointLog] [info] iteration = 535 | max rel diff. = 0.000873772
[2018-09-28 13:39:24.129] [jointLog] [info] Finished optimizer
[2018-09-28 13:39:24.129] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P2-3_trim_mh_PE with reads trimmomatic_mh/P2-3_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P2-3_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P2-3_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P2-3_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P2-3_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P2-3_trim_mh_PE.quant/logs
[2018-09-28 13:39:36.599] [jointLog] [info] parsing read library format
[2018-09-28 13:39:36.599] [jointLog] [info] There is 1 library.
[2018-09-28 13:39:36.746] [jointLog] [info] Loading Quasi index
[2018-09-28 13:39:36.746] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:39:36.746] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:39:40.715] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:39:41.632] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:39:41.941] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:39:42.076] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:39:42.077] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:40:57.928] [stderrLog] [info] Done loading index
[2018-09-28 13:40:57.928] [jointLog] [info] done
[2018-09-28 13:40:57.928] [jointLog] [info] Index contained 209578 targets
processed 9000000 fragments
hits: 4898598, hits per frag: 0.570022
[2018-09-28 13:41:39.684] [jointLog] [info] Computed 217309 rich equivalence classes for further processing
[2018-09-28 13:41:39.684] [jointLog] [info] Counted 5034639 total reads in the equivalence classes
[2018-09-28 13:41:39.767] [jointLog] [info] Mapping rate = 53.7368%
[2018-09-28 13:41:39.768] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:41:39.768] [jointLog] [info] Starting optimizer
[2018-09-28 13:41:39.916] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-28 13:41:39.925] [jointLog] [info] iteration = 0 | max rel diff. = 1.32755
[2018-09-28 13:41:40.549] [jointLog] [info] iteration = 100 | max rel diff. = 0.0440606
[2018-09-28 13:41:41.165] [jointLog] [info] iteration = 200 | max rel diff. = 0.0345308
[2018-09-28 13:41:41.170] [jointLog] [info] iteration = 202 | max rel diff. = 0.00522146
[2018-09-28 13:41:41.187] [jointLog] [info] Finished optimizer
[2018-09-28 13:41:41.187] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P3-1_trim_mh_PE with reads trimmomatic_mh/P3-1_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P3-1_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P3-1_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P3-1_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P3-1_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P3-1_trim_mh_PE.quant/logs
[2018-09-28 13:41:51.831] [jointLog] [info] parsing read library format
[2018-09-28 13:41:51.831] [jointLog] [info] There is 1 library.
[2018-09-28 13:41:51.980] [jointLog] [info] Loading Quasi index
[2018-09-28 13:41:51.980] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:41:51.980] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:41:57.842] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:41:58.754] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:41:59.059] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:41:59.183] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:41:59.184] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:43:15.159] [jointLog] [info] done
[2018-09-28 13:43:15.159] [jointLog] [info] Index contained 209578 targets
[2018-09-28 13:43:15.159] [stderrLog] [info] Done loading index
processed 9000000 fragments
hits: 4859395, hits per frag: 0.561233
[2018-09-28 13:44:02.472] [jointLog] [info] Computed 189305 rich equivalence classes for further processing
[2018-09-28 13:44:02.472] [jointLog] [info] Counted 5043210 total reads in the equivalence classes
[2018-09-28 13:44:02.547] [jointLog] [info] Mapping rate = 53.214%
[2018-09-28 13:44:02.547] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:44:02.547] [jointLog] [info] Starting optimizer
[2018-09-28 13:44:02.815] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-28 13:44:02.820] [jointLog] [info] iteration = 0 | max rel diff. = 2.96946
[2018-09-28 13:44:03.466] [jointLog] [info] iteration = 100 | max rel diff. = 0.0331629
[2018-09-28 13:44:04.035] [jointLog] [info] iteration = 200 | max rel diff. = 0.0127142
[2018-09-28 13:44:04.586] [jointLog] [info] iteration = 300 | max rel diff. = 0.013338
[2018-09-28 13:44:05.116] [jointLog] [info] iteration = 400 | max rel diff. = 0.0115014
[2018-09-28 13:44:05.717] [jointLog] [info] iteration = 500 | max rel diff. = 0.0114869
[2018-09-28 13:44:06.284] [jointLog] [info] iteration = 600 | max rel diff. = 0.0114823
[2018-09-28 13:44:06.570] [jointLog] [info] iteration = 652 | max rel diff. = 0.00833909
[2018-09-28 13:44:06.588] [jointLog] [info] Finished optimizer
[2018-09-28 13:44:06.588] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P3-2_trim_mh_PE with reads trimmomatic_mh/P3-2_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P3-2_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P3-2_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P3-2_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P3-2_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P3-2_trim_mh_PE.quant/logs
[2018-09-28 13:44:18.593] [jointLog] [info] parsing read library format
[2018-09-28 13:44:18.593] [jointLog] [info] There is 1 library.
[2018-09-28 13:44:18.796] [jointLog] [info] Loading Quasi index
[2018-09-28 13:44:18.796] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:44:18.796] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:44:23.479] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:44:24.550] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:44:24.854] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:44:24.979] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:44:24.980] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:45:42.049] [stderrLog] [info] Done loading index
[2018-09-28 13:45:42.049] [jointLog] [info] done
[2018-09-28 13:45:42.049] [jointLog] [info] Index contained 209578 targets
processed 9500000 fragments
hits: 5194439, hits per frag: 0.566982
[2018-09-28 13:46:27.949] [jointLog] [info] Computed 190868 rich equivalence classes for further processing
[2018-09-28 13:46:27.949] [jointLog] [info] Counted 5344888 total reads in the equivalence classes
[2018-09-28 13:46:28.025] [jointLog] [info] Mapping rate = 53.8928%
[2018-09-28 13:46:28.025] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:46:28.026] [jointLog] [info] Starting optimizer
[2018-09-28 13:46:28.168] [jointLog] [info] Marked 1 weighted equivalence classes as degenerate
[2018-09-28 13:46:28.180] [jointLog] [info] iteration = 0 | max rel diff. = 1.92492
[2018-09-28 13:46:28.809] [jointLog] [info] iteration = 100 | max rel diff. = 0.0444149
[2018-09-28 13:46:29.417] [jointLog] [info] iteration = 200 | max rel diff. = 0.0160686
[2018-09-28 13:46:30.017] [jointLog] [info] iteration = 300 | max rel diff. = 0.0115261
[2018-09-28 13:46:30.647] [jointLog] [info] iteration = 400 | max rel diff. = 0.0117537
[2018-09-28 13:46:31.236] [jointLog] [info] iteration = 500 | max rel diff. = 0.0118256
[2018-09-28 13:46:31.276] [jointLog] [info] iteration = 508 | max rel diff. = 0.00238785
[2018-09-28 13:46:31.315] [jointLog] [info] Finished optimizer
[2018-09-28 13:46:31.315] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- processing sample P3-3_trim_mh_PE with reads trimmomatic_mh/P3-3_trim_mh_PE_1.fastq and
-----
----- trimmomatic_mh/P3-3_trim_mh_PE_2.fastq...
-----
------------------------------------------------------------------------------------------------------------------
------
Version Info: ### A newer version of Salmon is available. ####
###
The newest version, available at https://github.com/COMBINE-lab/salmon/releases
contains new features, improvements, and bug fixes; please upgrade at your
earliest convenience.
###
### salmon (mapping-based) v0.9.1
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { quant_bins_all/assembly_index }
### [ libType ] => { IU }
### [ mates1 ] => { trimmomatic_mh/P3-3_trim_mh_PE_1.fastq }
### [ mates2 ] => { trimmomatic_mh/P3-3_trim_mh_PE_2.fastq }
### [ output ] => { quant_bins_all/alignment_files/P3-3_trim_mh_PE.quant }
Logs will be written to quant_bins_all/alignment_files/P3-3_trim_mh_PE.quant/logs
[2018-09-28 13:46:42.651] [jointLog] [info] parsing read library format
[2018-09-28 13:46:42.651] [jointLog] [info] There is 1 library.
[2018-09-28 13:46:42.930] [stderrLog] [info] Loading Suffix Array
[2018-09-28 13:46:42.930] [jointLog] [info] Loading Quasi index
[2018-09-28 13:46:42.930] [jointLog] [info] Loading 32-bit quasi index
[2018-09-28 13:46:50.531] [stderrLog] [info] Loading Transcript Info
[2018-09-28 13:46:52.164] [stderrLog] [info] Loading Rank-Select Bit Array
[2018-09-28 13:46:52.528] [stderrLog] [info] There were 209578 set bits in the bit array
[2018-09-28 13:46:52.669] [stderrLog] [info] Computing transcript lengths
[2018-09-28 13:46:52.670] [stderrLog] [info] Waiting to finish loading hash
[2018-09-28 13:48:10.269] [jointLog] [info] done
[2018-09-28 13:48:10.269] [jointLog] [info] Index contained 209578 targets
[2018-09-28 13:48:10.269] [stderrLog] [info] Done loading index
processed 9000001 fragments
hits: 4927052, hits per frag: 0.557441
[2018-09-28 13:48:26.816] [jointLog] [info] Computed 187291 rich equivalence classes for further processing
[2018-09-28 13:48:26.816] [jointLog] [info] Counted 4941741 total reads in the equivalence classes
[2018-09-28 13:48:26.881] [jointLog] [warning] Only 4941741 fragments were mapped, but the number of burn-in fragm
ents was set to 5000000.
The effective lengths have been computed using the observed mappings.
[2018-09-28 13:48:26.881] [jointLog] [info] Mapping rate = 53.9743%
[2018-09-28 13:48:26.881] [jointLog] [info] finished quantifyLibrary()
[2018-09-28 13:48:26.881] [jointLog] [info] Starting optimizer
[2018-09-28 13:48:26.961] [jointLog] [info] Marked 1 weighted equivalence classes as degenerate
[2018-09-28 13:48:26.966] [jointLog] [info] iteration = 0 | max rel diff. = 1.66144
[2018-09-28 13:48:27.312] [jointLog] [info] iteration = 100 | max rel diff. = 0.032761
[2018-09-28 13:48:27.620] [jointLog] [info] iteration = 200 | max rel diff. = 0.0192132
[2018-09-28 13:48:27.877] [jointLog] [info] iteration = 300 | max rel diff. = 0.0121217
[2018-09-28 13:48:28.133] [jointLog] [info] iteration = 400 | max rel diff. = 0.0123326
[2018-09-28 13:48:28.207] [jointLog] [info] iteration = 430 | max rel diff. = 0.00798077
[2018-09-28 13:48:28.220] [jointLog] [info] Finished optimizer
[2018-09-28 13:48:28.220] [jointLog] [info] writing output
------------------------------------------------------------------------------------------------------------------
------
----- summarize salmon files...
-----
------------------------------------------------------------------------------------------------------------------
------
Starting in: /localData/mecantao/metawrap/dani_edu/quant_bins_all/alignment_files
Loading counts from: ./P1-1_trim_mh_PE.quant quant.sf
Loading counts from: ./P1-2_trim_mh_PE.quant quant.sf
Loading counts from: ./P1-3_trim_mh_PE.quant quant.sf
Loading counts from: ./P2-1_trim_mh_PE.quant quant.sf
Loading counts from: ./P2-2_trim_mh_PE.quant quant.sf
Loading counts from: ./P2-3_trim_mh_PE.quant quant.sf
Loading counts from: ./P3-1_trim_mh_PE.quant quant.sf
Loading counts from: ./P3-2_trim_mh_PE.quant quant.sf
Loading counts from: ./P3-3_trim_mh_PE.quant quant.sf
"P1-1_trim_mh_PE.quant.counts",
"P1-2_trim_mh_PE.quant.counts",
"P1-3_trim_mh_PE.quant.counts",
"P2-1_trim_mh_PE.quant.counts",
"P2-2_trim_mh_PE.quant.counts",
"P2-3_trim_mh_PE.quant.counts",
"P3-1_trim_mh_PE.quant.counts",
"P3-2_trim_mh_PE.quant.counts",
"P3-3_trim_mh_PE.quant.counts"
##################################################################################################################
######
##### EXTRACTING AVERAGE ABUNDANCE OF EACH BIN
#####
##################################################################################################################
######
------------------------------------------------------------------------------------------------------------------
------
----- There were 9 samples detected. Making abundance table!
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- Average bin abundance table stored in quant_bins_all/abundance_table.tab
-----
------------------------------------------------------------------------------------------------------------------
------
##################################################################################################################
######
##### MAKING GENOME ABUNDANCE HEATMAP WITH SEABORN
#####
##################################################################################################################
######
------------------------------------------------------------------------------------------------------------------
------
----- counting reads in each sample for standardization, and storing in
-----
----- quant_bins_all/sample_read_count.tab
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P3-3_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P2-3_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P3-1_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P3-2_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P2-2_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P2-1_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P1-2_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P1-1_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- finished counting reads in P1-3_trim_mh_PE_1.fastq
-----
------------------------------------------------------------------------------------------------------------------
------
------------------------------------------------------------------------------------------------------------------
------
----- making heatmap with Seaborn
-----
------------------------------------------------------------------------------------------------------------------
------
loading libs...
loading abundance data...
drawing clustermap...
Traceback (most recent call last):
File "/dados/programas/miniconda2/envs/metawrap-env/bin/metawrap-scripts/make_heatmap.py", line 78, in <module>
draw_clustermap(df, lut)
File "/dados/programas/miniconda2/envs/metawrap-env/bin/metawrap-scripts/make_heatmap.py", line 56, in draw_clus
termap
g = sns.clustermap(df, figsize=(14,8), col_colors=lut, col_cluster=True, yticklabels=True, cmap="magma")
File "/dados/programas/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/seaborn/matrix.py", line 1295, i
n clustermap
mask=mask)
File "/dados/programas/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/seaborn/matrix.py", line 775, in
__init__
self._preprocess_colors(data, col_colors, axis=1)
File "/dados/programas/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/seaborn/matrix.py", line 841, in
_preprocess_colors
colors = _convert_colors(colors)
File "/dados/programas/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/seaborn/matrix.py", line 53, in
_convert_colors
to_rgb(colors[0])
TypeError: 'bool' object has no attribute '__getitem__'
******************************************************************************************************************
******
***** something went wrong with making the heatmap. Exiting...
*****
******************************************************************************************************************
******
real 44m20.781s
user 212m55.085s
sys 21m40.326s
Thanks
Errors when running Kraken and Maxbin2
Hi,
I tried to run some modules of metaWRAP on the tutorial data and got the following error messages:
- Similar to the other issue reported by karoraw1, when I run...
metawrap binning -o INITIAL_BINNING -t 56 -a ASSEMBLY/final_assembly.fasta --maxbin2 CLEAN_READS/ERR*fastq
I get the following error...
----- Starting binning with MaxBin2... ——
Can't locate LWP/Simple.pm in @ INC (@ INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/a1654837/.conda/envs/metawrap-env/bin/run_MaxBin.pl line 4. BEGIN failed--compilation aborted at /home/a1654837/.conda/envs/metawrap-env/bin/run_MaxBin.pl line 4. ************************************************************************************************************************
***** Something went wrong with running MaxBin2. Exiting. ***** ************************************************************************************************************************
Metabat2 and concoct seem to run without problem.
I was able to get around it by setting the @ INC as follows:
export PERL5LIB=/home/a1654837/.conda/envs/metawrap-env/lib/perl5/site_perl/5.22.0/
- (minor) When running Kraken, I get this error at the end..
##### RUNNING KRAKEN-TRANSLATE ON OUTPUT #####
########################################################################################################################
##### MAKING KRONAGRAM OF ALL FILES #####
########################################################################################################################
**Unknown option: a**
************************************************************************************************************************
***** Something went wrong with running KronaTools to make kronagram. Exiting... *****
************************************************************************************************************************
Checking the script “ktImportText" there is no option “-a”.
Error when using Quant_bins
Note: this error was submitted by a user, and I am just relaying it here so everyone can see it.
I have got an error while running the quant_bins module:
300 Feb 7 20:09 assembly_index
1536 Feb 7 20:22 quant_files
1368 Feb 7 20:22 alignment_files
0 Feb 7 20:22 abundance_table.tab
The error:
Loading counts from: ./A1.quant quant.sf
Loading counts from: ./A2.quant quant.sf
Traceback (most recent call last):
File "/home/people/alpal/.conda/envs/metawrap-env/bin/metawrap-scripts/split_salmon_out_into_bins.py", line 39, in <module>
bin_abundances[bin]["total_len"]+=contig_lengths[line.split("\t")[0]]
KeyError: 'NTT_9-2948'
one of the quant.sf files looks like:
Name Length EffectiveLength TPM NumReads
contig-1673 33461 33276.054 8.732845 55.526894
contig-2011 30366 30181.054 42.073555 242.638354
....
....
....
NTT_9-2527 1225 1040.054 934.421424 185.701066
NTT_9-2625 1190 1005.054 1201.641601 230.770382
NTT_9-2661 1178 993.054 1712.753167 325.000000
NTT_9-2814 1132 947.054 1992.883028 360.638644
NTT_9-2828 1128 943.054 1337.801299 241.070398
NTT_9-2947 614 431.962 2231.053931 184.149826
NTT_9-2948 591 409.974 957.389568 75.000000
Thanks
Error quant_bins module
Hi,
I have run quant_bins but it has stopped with the following error:
`Starting in: /home/projects/env_10000/people/alpal/WaterWorks/F17FTSEUET0039_BACggoR/Clean/Alex_1/IDBA_all_TM_meta/all_merged_TM_paired_IDBA_OK/metaWRAP/binning_all_OK/bin_refinement/quantification/alignment_files
Loading counts from: ./CGAGGCTGTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./TACGCTGCTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GTAGAGGATCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./CGGAGCCTTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./ATGCGCAGTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./AAGAGGCATCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./AGGCAGAAAGAGGATA_merged_TM_paired.quant quant.sf
Loading counts from: ./CGTACTAGAGAGGATA_merged_TM_paired.quant quant.sf
Loading counts from: ./ATCTCAGGTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./TAGCGCTCTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./AGGCAGAATCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GCTCATGATCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GGAGCTACTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./CGAGGCTGCTAGTCGA_merged_TM_paired.quant quant.sf
Loading counts from: ./ACTCGCTATCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GGACTCCTAGAGGATA_merged_TM_paired.quant quant.sf
Loading counts from: ./TCCTGAGCAGAGGATA_merged_TM_paired.quant quant.sf
Loading counts from: ./CTCTCTACCTAGTCGA_merged_TM_paired.quant quant.sf
Loading counts from: ./TAGGCATGTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GCTACGCTTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GGACTCCTTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./CTCTCTACTCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./GCGTAGTATCGCATAA_merged_TM_paired.quant quant.sf
Loading counts from: ./TAGGCATGAGAGGATA_merged_TM_paired.quant quant.sf
Traceback (most recent call last):
File "/home/people/alpal/.conda/envs/metawrap-env/bin/metawrap-scripts/split_salmon_out_into_bins.py", line 27, in <module>
bin_abundances[bin]["total_len"]+=int(line.split("\t")[0].split("_")[3])
IndexError: list index out of range
It finished like this:
########################################################################################################################
##### EXTRACTING AVERAGE ABUNDANCE OF EACH BIN #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- There were 24 samples detected. Making abundance table! -----
------------------------------------------------------------------------------------------------------------------------
************************************************************************************************************************
something went wrong with making summary abundance table. Exiting...
This is how the output folder looks:
61 Jan 15 12:15 all_bin_contigs
1536 Jan 15 13:20 quant_files
1368 Jan 15 13:20 alignment_files
0 Jan 15 13:20 abundance_table.tab
Each quant.sf looks like:
(metawrap-env) head quant.sf
Name Length EffectiveLength TPM NumReads
contig-1 420936 420800.156 183.796403 109528.269851
contig-2 412056 411920.156 3.191064 1861.495250
contig-3 406383 406247.156 65.965626 37950.836252
contig-4 389993 389857.156 5.964740 3293.141174
contig-5 382469 382333.156 3.212324 1739.300499
contig-6 338179 338043.156 3.197438 1530.691453
contig-7 326651 326515.156 5.777906 2671.696687
contig-8 316073 315937.156 6.080464 2720.512609
contig-9 310613 310477.156 5.813615 2556.167111
Any idea about what is going on?
Thanks in advance.
list of library depencies for reassembly module on HPC cluster
Hello Gherman,
I am running all steps from the metaWRAP tool on a HPC cluster. That means that not only all resources (time, memory, cores) should be requested before the run (based on prediction) but also all of the dependencies.
So far, every dependency was satisfied by the installation and simply loading the conda metawrap_env was enough to get everything running, except for the reassembly module. Apparently there libraries that are not available and still they are being required.
Example:
When running the reassembly module for a given set of samples. I got the following line on my *err output file:
$ more seawater_reassembly_bins*.err | grep Import
ImportError: libXext.so.6: cannot open shared object file: No such file or directory
So what I did was to " manually" load this library for the run with "module load libxext" and run again. Then I got a second error:
$ more seawater_reassembly_bins*.err | grep Import
ImportError: libSM.so.6: cannot open shared object file: No such file or directory
I assume, then, there are a few libraries that are not available.
I will append more details from the last run .err file:
$ tail -n 40 sponge_reassembly_bins-4808018.err
Calculating AAI between multi-copy marker genes.
Reading HMM info from file.
Parsing HMM hits to marker genes:
Finished parsing hits for 14 of 14 (100.00%) bins.
{ Current stage: 0:00:08.592 || Total: 0:12:47.236 }
*******************************************************************************
[CheckM - bin_qa_plot] Creating bar plot of bin quality.
*******************************************************************************
Calculating AAI between multi-copy marker genes.
Plotting bin completeness, contamination, and strain heterogeneity.
Plot written to: /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins.plot/bin_qa_plot.png
{ Current stage: 0:03:05.834 || Total: 0:03:05.834 }
Traceback (most recent call last):
File "/data/msb/tools/metawrap/metawrap_env/bin/metawrap-scripts/plot_reassembly.py", line 6, in <module>
import matplotlib.pyplot as plt
File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/pyplot.py", line 114, in <module>
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
globals(),locals(),[backend_name],0)
File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/backend_qt4agg.py", line 18, in <module>
from .backend_qt5agg import FigureCanvasQTAggBase as _FigureCanvasQTAggBase
File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5agg.py", line 15, in <module>
from .backend_qt5 import QtCore
File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/backend_qt5.py", line 31, in <module>
from .qt_compat import QtCore, QtGui, QtWidgets, _getSaveFileName, __version__
File "/data/msb/tools/metawrap/metawrap_env/lib/python2.7/site-packages/matplotlib/backends/qt_compat.py", line 124, in <module>
from PyQt4 import QtCore, QtGui
ImportError: libSM.so.6: cannot open shared object file: No such file or directory
real 601m30.487s
user 1709m57.309s
sys 81m37.775s
And for the same run, the .out file
$ tail -n 30 sponge_reassembly_bins-4808018.out
------------------------------------------------------------------------------------------------------------------------
----- Finalizing CheckM stats... -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Making CheckM plot of /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins -----
----- bins -----
------------------------------------------------------------------------------------------------------------------------
Plotting bin 14 of 14 (100.00%) bins.
------------------------------------------------------------------------------------------------------------------------
----- you will find the info on the final reassembled bins in -----
----- /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins.stats, and a figure -----
----- summarizing it in /data/msb/LEMAM/rita_polonia/BIN_REASSEMBLY/sponge/reassembled_bins.png -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- making reassembly N50, compleiton, and contamination summary plots. -----
------------------------------------------------------------------------------------------------------------------------
************************************************************************************************************************
***** Something went wrong with plotting the reassembly summary plots. Exiting... *****
************************************************************************************************************************
end reassembly bins
Wed Sep 26 02:38:54 CEST 2018
Again, I assume this is the problem. Could I maybe get a list of the dependencies that are required? I tried to find the list of the libraries needed but it was a little challenging for me.
Thanks again for support!!!
Best,
Rodolfo
issue running bin reassembly
Hi,
I'm running bin reassembly in several bins. I have tried with several reads as an input but I always get the same no output.
I have done a test with a set of bins that were improved using bin reassembly (with a previous metWRAP version) but I also get no output.
Output with previous version:
108 Apr 5 21:58 original_bins
130 Apr 5 23:48 reassembled_bins
216 Apr 5 23:48 work_files
75 Apr 5 23:55 reassembled_bins.checkm
262 Apr 5 23:57 reassembled_bins.stats
106615 Apr 5 23:57 reassembled_bins.png
260 Apr 5 23:57 original_bins.stats
44246 Apr 5 23:57 reassembly_results.eps
1036345 Apr 5 23:57 reassembly_results.png
New output:
108 Jul 1 16:40 original_bins
193 Jul 1 16:40 binned_assembly
656 Jul 1 16:40 reads_for_reassembly
560 Jul 1 16:57 reassemblies
0 Jul 1 16:58 reassembled_bins
And the reassemblies folder looks like:
144 Jul 1 16:42 bin.10.permissive
0 Jul 1 16:43 bin.10.permissive.tmp
144 Jul 1 16:43 bin.10.strict
0 Jul 1 16:47 bin.10.strict.tmp
244 Jul 1 16:51 bin.20.permissive
0 Jul 1 16:51 bin.20.permissive.tmp
244 Jul 1 16:55 bin.20.strict
0 Jul 1 16:55 bin.20.strict.tmp
274 Jul 1 16:56 bin.52.permissive
0 Jul 1 16:56 bin.52.permissive.tmp
274 Jul 1 16:56 bin.52.strict
0 Jul 1 16:56 bin.52.strict.tmp
274 Jul 1 16:57 bin.64.permissive
0 Jul 1 16:57 bin.64.permissive.tmp
274 Jul 1 16:58 bin.64.strict
0 Jul 1 16:58 bin.64.strict.tmp
Here I upload the log file. It seems it might be related with SPAdes?
The abundance_table.tab is blank
Dear Gherman
I've been trying to use metawrap to calculate bins abundance, but for some reason in some cases, I cannot understand which, the abundance table result is blank, although I have a sample_read_count.tab and AC3_2MetaGenMerged_R1_val.quant.counts results.
As I do some editing in windows, and some times some programs have problems reading the files I do dos2unix command in all files before using them in linux.
As I'm using just one sample, the heatmap gives error, but before that I have the error bellow.
I would like to ask for some help again.
Thanks
Igor
[2018-09-24 17:49:29.302] [jointLog] [info] Computed 103949 rich equivalence classes for further processing
[2018-09-24 17:49:29.302] [jointLog] [info] Counted 10278288 total reads in the equivalence classes
[2018-09-24 17:49:29.303] [jointLog] [warning] 0.000125587% of fragments were shorter than the k used to build the index (31).
If this fraction is too large, consider re-building the index with a smaller k.
The minimum read size found was 22.
[2018-09-24 17:49:29.303] [jointLog] [info] Mapping rate = 43.0271%
[2018-09-24 17:49:29.303] [jointLog] [info] finished quantifyLibrary()
[2018-09-24 17:49:29.304] [jointLog] [info] Starting optimizer
[2018-09-24 17:49:29.354] [jointLog] [info] Marked 0 weighted equivalence classes as degenerate
[2018-09-24 17:49:29.357] [jointLog] [info] iteration = 0 | max rel diff. = 169.784
[2018-09-24 17:49:29.614] [jointLog] [info] iteration = 100 | max rel diff. = 4.38914e-16
[2018-09-24 17:49:29.616] [jointLog] [info] Finished optimizer
[2018-09-24 17:49:29.616] [jointLog] [info] writing output
[2018-09-24 17:49:29.715] [jointLog] [warning] NOTE: Read Lib [( /home/igor/AC3_Data_Metagenomes/TrimGalore_fastqc_files/AC3_READ_QC/AC3_2MetaGenMerged_R1_val_1.fastq, /home/igor/AC3_Data_Metagenomes/TrimGalore_fastqc_files/AC3_READ_QC/AC3_2MetaGenMerged_R1_val_2.fastq )] :
Detected a *potential* strand bias > 1% in an unstranded protocol check the file: /home/igor/AC3_BINNING_GENES_COVERAGE_ANALYSES/genes_Coverage/alignment_files/AC3_2MetaGenMerged_R1_val.quant/lib_format_counts.json for details
------------------------------------------------------------------------------------------------------------------------
----- summarize salmon files... -----
------------------------------------------------------------------------------------------------------------------------
Starting in: /home/igor/AC3_BINNING_GENES_COVERAGE_ANALYSES/genes_Coverage/alignment_files
Loading counts from: ./AC3_2MetaGenMerged_R1_val.quant quant.sf
"AC3_2MetaGenMerged_R1_val.quant.counts"
########################################################################################################################
##### EXTRACTING AVERAGE ABUNDANCE OF EACH BIN #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- There were 1 samples detected. Making abundance table! -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Average bin abundance table stored in -----
----- /home/igor/AC3_BINNING_GENES_COVERAGE_ANALYSES/genes_Coverage/abundance_table.tab -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### MAKING GENOME ABUNDANCE HEATMAP WITH SEABORN #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- counting reads in each sample for standardization, and storing in -----
----- /home/igor/AC3_BINNING_GENES_COVERAGE_ANALYSES/genes_Coverage/sample_read_count.tab -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- finished counting reads in AC3_2MetaGenMerged_R1_val_1.fastq -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- make heatmap -----
------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/igor/miniconda2/envs/metawrap-env/bin/metawrap-scripts/make_heatmap.py", line 23, in <module>
z=pd.read_csv(sys.argv[2], sep='\t', index_col=0)
File "/home/igor/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pandas/io/parsers.py", line 709, in parser_f
return _read(filepath_or_buffer, kwds)
File "/home/igor/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pandas/io/parsers.py", line 449, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/home/igor/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pandas/io/parsers.py", line 818, in __init__
self._make_engine(self.engine)
File "/home/igor/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pandas/io/parsers.py", line 1049, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/home/igor/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/pandas/io/parsers.py", line 1695, in __init__
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 565, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file
************************************************************************************************************************
***** something went wrong with making the heatmap. Exiting... *****
************************************************************************************************************************
real 18m50.216s
user 34m54.451s
sys 0m44.535s
Error classify_bins module
Hi,
I have run classify_bins but it has stopped with the following error:
Log:
/home/people/name/.conda/envs/metawrap-env/bin/metawrap-modules/classify_bins.sh -b metaWRAP_bins -o classify_bins -t 28
########################################################################################################################
##### ALIGN CONTIGS TO DATABASE WITH MEGABLAST #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- setting up ouput folder classify_bins and merging contigs from all bins... -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- aligning classify_bins/all_contigs.fa to -----
----- /home/projects/env_10000/people/alpal/scripts/NCBI_nt database with MEGABLAST. This is the -----
----- longest step - please be patient. You may look at the classification progress in -----
----- classify_bins/megablast_out.tab -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- removing unnecessary lines that lead to bad tax IDs (without a proper rank) -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- making mapping file -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### GET TAXONOMY FROM MEGABLAST OUTPUT WITH TAXATOR-TK #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- pulling out classifications with taxator -----
------------------------------------------------------------------------------------------------------------------------
************************************************************************************************************************
***** Failed to run taxator. Exiting... *****
************************************************************************************************************************
Error:
An unrecoverable error occurred: std::exception
Here is some debugging information to locate the problem:
/scratch/users/[email protected]/miniconda2/conda-bld/taxator-kt_1514604714830/work/src/fileparser.hh(52): Throw in function FileParser<FactoryType>::RecordType* FileParser<FactoryType>::next() [with FactoryType = AlignmentRecordFactory<AlignmentRecordTaxonomy>; FileParser<FactoryType>::RecordType = AlignmentRecordTaxonomy]
Dynamic exception type: boost::exception_detail::clone_impl<Exception>
std::exception::what: std::exception
[exception_tag_line*] = 1
[exception_tag_general*] = bad number of fields in alignment line
cat: write error: Broken pipe
output folder:
458459200 Jan 15 14:11 all_contigs.fa
794859069 Jan 15 16:34 megablast_out.tab
299814468 Jan 15 16:34 mapping.tax
0 Jan 15 16:34 predictions.gff3
Looking forward to any help.
Thanks in advance.
indexed BAM files
Hi, I am locating the mapping file. Just curious about why there is no .bam.bai file in the work-file?
Thanks.
PackageNotFoundError: Packages missing in current channels
Hi,
I'm trying to install metaWRAP but I got this error:
conda create -n metawrap-env python=2.7
Fetching package metadata ...........
Solving package specifications: .
Package plan for installation in environment /home/people/name/.conda/envs/metawrap-env:
The following NEW packages will be INSTALLED:
ca-certificates: 2017.08.26-h1d4fec5_0
certifi: 2017.11.5-py27h71e7faf_0
libedit: 3.1-heed3624_0
libffi: 3.2.1-hd88cf55_4
libgcc-ng: 7.2.0-h7cc24e2_2
libstdcxx-ng: 7.2.0-h7a57d05_2
ncurses: 6.0-h9df7e31_2
openssl: 1.0.2n-hb7f436b_0
pip: 9.0.1-py27ha730c48_4
python: 2.7.14-h1571d57_29
readline: 7.0-ha6073c6_4
setuptools: 36.5.0-py27h68b189e_0
sqlite: 3.20.1-hb898158_2
tk: 8.6.7-hc745277_3
wheel: 0.30.0-py27h2bc6bb2_1
zlib: 1.2.11-ha838bed_2
Proceed ([y]/n)? y
conda install -c ursky metawrap-binningy-env
Proceed ([y]/n)? y
ca-certificate 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 11.99 MB/s
libgcc-ng-7.2. 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 29.42 MB/s
libstdcxx-ng-7 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 24.99 MB/s
libffi-3.2.1-h 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 3.55 MB/s
ncurses-6.0-h9 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 29.51 MB/s
openssl-1.0.2n 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 29.16 MB/s
tk-8.6.7-hc745 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 38.64 MB/s
zlib-1.2.11-ha 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 10.87 MB/s
libedit-3.1-he 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 21.76 MB/s
readline-7.0-h 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 35.47 MB/s
sqlite-3.20.1- 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 36.59 MB/s
python-2.7.14- 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 39.36 MB/s
certifi-2017.1 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 21.09 MB/s
setuptools-36. 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 30.40 MB/s
wheel-0.30.0-p 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 12.62 MB/s
pip-9.0.1-py27 100% |#############################################################################################################################################################################################################################| Time: 0:00:00 37.77 MB/s
#
# To activate this environment, use:
# > source activate metawrap-env
#
# To deactivate an active environment, use:
# > source deactivate
#
source activate metawrap-env
(metawrap-env) metaWRAP read_qc -h
-bash: metaWRAP: command not found
(metawrap-env) conda install -c ursky metawrap-binningy-env
Fetching package metadata .............
PackageNotFoundError: Packages missing in current channels:
- metawrap-binningy-env
We have searched for the packages in the following channels:
- https://conda.anaconda.org/ursky/linux-64
- https://conda.anaconda.org/ursky/noarch
- https://repo.continuum.io/pkgs/main/linux-64
- https://repo.continuum.io/pkgs/main/noarch
- https://repo.continuum.io/pkgs/free/linux-64
- https://repo.continuum.io/pkgs/free/noarch
- https://repo.continuum.io/pkgs/r/linux-64
- https://repo.continuum.io/pkgs/r/noarch
- https://repo.continuum.io/pkgs/pro/linux-64
- https://repo.continuum.io/pkgs/pro/noarchDo you know what could be the error?
Thanks in advance.
How to run these commands with multiple nodes in server cluster?
The server cluster I used had 32 CPUs in each node. How to run these commands with multiple nodes for more CPUs?
Error checkm in Binning procedure
Hi,
I have run the binning module, using metabat2, maxbin2, concoct, plus checkm. here is the error message. it is still running. i would like to know if this is a big problem, as it looks the final checkm results looks all fine.
during metabat2 binning:
*******************************************************************************
[CheckM - analyze] Identifying marker genes in bins.
*******************************************************************************
Identifying marker genes in 536 bins with 40 threads:
Finished processing 536 of 536 (100.00%) bins.
Traceback (most recent call last):
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/multiprocessing/util.py", line 274, in _run_finalizers
finalizer()
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/multiprocessing/util.py", line 207, in __call__
res = self._callback(*self._args, **self._kwargs)
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 16] Device or resource busy: '.tmp/pymp-NioJzr/.nfs000000002876a5af00000527'
Saving HMM info to file.
during maxbin2 binning:
[CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************
Identifying marker genes in 549 bins with 40 threads:
Finished processing 549 of 549 (100.00%) bins.
Traceback (most recent call last):
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/multiprocessing/util.py", line 274, in _run_finalizers
finalizer()
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/multiprocessing/util.py", line 207, in __call__
res = self._callback(*self._args, **self._kwargs)
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/export/data/programs/pyenv/versions/2.7.9/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 16] Device or resource busy: 'bin.549.fasta.tmp/pymp-GrvYbX/.nfs000000002827d9320000074f'
Saving HMM info to file.
Thanks in advance.
classify bins catches on weird taxa
Apparently there is a bit of hamster in my mapping.tax file.
An unrecoverable error occurred: std::exception
Here is some debugging information to locate the problem:
/home/johdro/projects/taxator-tk_default.git/src/fileparser.hh(52): Throw in function FileParser<FactoryType>::RecordType* FileParser<FactoryType>::next() [with FactoryType = AlignmentRecordFactory<AlignmentRecordTaxonomy>; FileParser<FactoryType>::RecordType = AlignmentRecordTaxonomy]
Dynamic exception type: boost::exception_detail::clone_impl<Exception>
std::exception::what: std::exception
[exception_tag_line*] = 1488404
[exception_tag_taxid*] = 10026
[exception_tag_general*] = bad alignment reference taxon
I fixed this by using
taxknife -f 2 --mode traverse -r species genus family order class phylum superkingdom < mapping.tax > newmapping.tax
as referenced here:
fungs/taxator-tk#51
i am using an older version of metawrap so i am not sure if you've corrected for this already, but i thought i would pass it along.
Possible to allow user assign directory for tmp files e.g. in BIN_REFINEMENT module?
It would be better to allow user to custom directory for temporary files generated by the metaWRAP modules, otherwise one could encounter "no space left on device" error, and no output for Refined_AB for instance in BIN_REFINEMENT module.
When running this module, some tmp files would be written into /tmp/ with no cautions, it may be by default according to python environment, and some other tmp files would be written into user-customed output directory. My suggestion is that it would be better to write all tmp files into output directory temporarily.
Thank you in advance!
Read_qc module error
When I run the read_qc module
metawrap /home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/metawrap-modules/read_qc -1 /home-2/[email protected]/data/Emine_gypsum_metagenomes/RAW_READS/SAG-NEG-2_1.fastq -2 /home-2/[email protected]/data/Emine_gypsum_metagenomes/RAW_READS/SAG-NEG-2_2.fastq -o SAG-NEG-2
I get the error:
########################################################################################################################
MAKING PRE-QC REPORT
########################################################################################################################
Error occurred during initialization of VM
Unable to load native library: libverify.so: cannot open shared object file: No such file or directory
***** Something went wrong with making pre-QC fastqc report. Exiting. *****
parallelization in annotate_bins breaks conda environmental variables
I run this:
source activate metawrap-env
MW_O=/home-3/[email protected]/scratch/metaWRAP_Out
metaWRAP annotate_bins \
-o $MW_O/Mystic_Bin_Fxns \
-t 48 \
-b $MW_O/Mystic_Bin_Reassembly/reassembled_bins
I get a copy of this for every prokka call in STDERR:
Can't locate XML/Simple.pm in @INC (you may need to install the XML::Simple module) (@INC contains: /home-3/[email protected]/perl5/lib/perl5/5.22.0/x86_64-linux /home-3/[email protected]/perl5/lib/perl5/5.22.0 /home-3/[email protected]/perl5/lib/perl5/x86_64-linux /home-3/[email protected]/perl5/lib/perl5 /cm/shared/apps/perl/5.22.0/lib/site_perl/5.22.0/x86_64-linux /cm/shared/apps/perl/5.22.0/lib/site_perl/5.22.0 /cm/shared/apps/perl/5.22.0/lib/5.22.0/x86_64-linux /cm/shared/apps/perl/5.22.0/lib/5.22.0 .) at /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/prokka line 25.
BEGIN failed--compilation aborted at /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/prokka line 25.
If I load the conda environment and run prokka manually, there is no problem. i.e.
[[email protected]@compute0117 metaWRAP_bins]$ source activate metawrap-env
(metawrap-env) [[email protected]@compute0117 metaWRAP_bins]$ prokka --outdir ~/scratch/test_bin_35_annot --prefix Bin35_test bin.35.fa --force
[13:22:26] This is prokka 1.12
....
[13:25:36] Thank you, come again.
The only way I can reproduce the error is by using module restore in SLURM. It seems to change where prokka searches for dependencies. blastp breaks, java breaks, and perl breaks as seen above.
It seems like calls to prokka within the parallelization scheme don't inherit the environmental variable preference that manual calls to prokka do.
Let me know if you need any other info from me.
Blobology figures
Hi,
I ran the module blobology of metaWRAP and it generates figures with "00000" instead of headers, bin names, and values.
The section "MAKE FINAL BLOBPLOT IMAGES" produces the following warnings:
########################################################################################################################
##### MAKE FINAL BLOBPLOT IMAGES #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- making blobplots with phylogeny annotations (order, phylum, and superkingdom). -----
------------------------------------------------------------------------------------------------------------------------
There were 26 warnings (use warnings() to see them)
null device
1
There were 22 warnings (use warnings() to see them)
null device
1
Warning messages:
1: Transformation introduced infinite values in continuous y-axis
2: Transformation introduced infinite values in continuous y-axis
3: Transformation introduced infinite values in continuous y-axis
4: Removed 21455 rows containing missing values (geom_point).
5: Removed 26829 rows containing missing values (geom_point).
6: Removed 1765 rows containing missing values (geom_point).
null device
1
------------------------------------------------------------------------------------------------------------------------
----- making blobplot images with bin annotations -----
------------------------------------------------------------------------------------------------------------------------
There were 26 warnings (use warnings() to see them)
null device
1
Warning messages:
1: Transformation introduced infinite values in continuous y-axis
2: Transformation introduced infinite values in continuous y-axis
3: Removed 29885 rows containing missing values (geom_point).
4: Removed 20164 rows containing missing values (geom_point).
null device
1
There were 20 warnings (use warnings() to see them)
null device
1
------------------------------------------------------------------------------------------------------------------------
----- making blobplot images of only the contigs that were binned -----
------------------------------------------------------------------------------------------------------------------------
There were 24 warnings (use warnings() to see them)
null device
1
------------------------------------------------------------------------------------------------------------------------
----- making blobplots of only binned contigs with phylogeny annotations (order, phylum, and -----
----- superkingdom). -----
------------------------------------------------------------------------------------------------------------------------
There were 20 warnings (use warnings() to see them)
null device
1
There were 18 warnings (use warnings() to see them)
null device
1
Warning messages:
1: Transformation introduced infinite values in continuous y-axis
2: Transformation introduced infinite values in continuous y-axis
3: Transformation introduced infinite values in continuous y-axis
4: Removed 5028 rows containing missing values (geom_point).
5: Removed 13854 rows containing missing values (geom_point).
6: Removed 1282 rows containing missing values (geom_point).
null device
1
------------------------------------------------------------------------------------------------------------------------
----- cleaning up... -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### BLOBPLOT PIPELINE FINISHED SUCCESSFULLY!!! #####
########################################################################################################################
``
error for metaWRAP read_qc after update to v0.7.7
I did the up date to v0.7.7 as bellow, and then tried to run metaWRAP read_qc...
Thanks for any help.
igor@ubuntu:~$ source activate metawrap-env
(metawrap-env) igor@ubuntu:~$ conda update -c ursky metawrap-mg
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.4.10
latest version: 4.4.11
Please update conda by running
$ conda update -n base conda
## Package Plan ##
environment location: /home/igor/miniconda2/envs/metawrap-env
added / updated specs:
- metawrap-mg
The following packages will be downloaded:
package | build
---------------------------|-----------------
metawrap-mg-0.7.7 | py27pl5_0 912 KB ursky
The following packages will be UPDATED:
metawrap-mg: 0.7.6-py27pl5_0 ursky --> 0.7.7-py27pl5_0 ursky
Proceed ([y]/n)? y
Downloading and Extracting Packages
metawrap-mg 0.7.7: ##################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(metawrap-env) igor@ubuntu:~$ metaWRAP read_qc --skip-bmtagger --skip-pre-qc-report --skip-post-qc-report -1 /home/igor/Metagenomes/AC3_2MetaGenMerged_R_1.fastq -2 /home/igor/Metagenomes/AC3_2MetaGenMerged_R_2.fastq -o /home/igor/metaWRAP/AC3_metagenome/READ_QC
metawrap /home/igor/miniconda2/envs/metawrap-env/bin/metawrap-modules/read_qc --skip-bmtagger --skip-pre-qc-report --skip-post-qc-report -1 /home/igor/Metagenomes/AC3_2MetaGenMerged_R_1.fastq -2 /home/igor/Metagenomes/AC3_2MetaGenMerged_R_2.fastq -o /home/igor/metaWRAP/AC3_metagenome/READ_QC
########################################################################################################################
##### RUNNING TRIM-GALORE #####
########################################################################################################################
/home/igor/miniconda2/envs/metawrap-env/bin/metawrap-modules/read_qc.sh: line 124: trim_galore: command not found
mv: cannot stat '/home/igor/metaWRAP/AC3_metagenome/READ_QC/AC3_2MetaGenMerged_R_1_val_1.fq': No such file or directory
mv: cannot stat '/home/igor/metaWRAP/AC3_metagenome/READ_QC/AC3_2MetaGenMerged_R_2_val_2.fq': No such file or directory
************************************************************************************************************************
***** Something went wrong with trimming the reads. Exiting. *****
************************************************************************************************************************
real 0m0.123s
user 0m0.052s
sys 0m0.028s
Technical question
Hi
I'm trying to figure how to build a server to do the analyses and I wanted to ask if any one know if I can work with a blade system or if I have to have a "super computer"?
Is there any program that does the management of resources available, if they are scattered in several machines, or if that is not possible?
Thanks for any advice.
Best
Igor
concoct run time?
Hi!
What is the expected runtime for concoct? I am starting off with an assembly file of 470 MB and bam file of 60GB. Concoct started running about 12 hours ago and I initially gave the job 32 cpus but looks like its only using 10 threads right now. The log file hasn't updated itself since it began:
2018-09-26 23:14:18,242:INFO:root:Results created at XXX/concoct_out
2018-09-26 23:18:26,255:INFO:root:Successfully loaded composition data.
2018-09-26 23:18:27,400:INFO:root:Successfully loaded coverage data.
2018-09-26 23:18:33,108:INFO:root:Performed PCA, resulted in 65 dimensions
2018-09-26 23:19:59,223:INFO:root:Wrote original filtered data file.
2018-09-26 23:20:41,312:INFO:root:Wrote PCA transformed file.
2018-09-26 23:20:41,342:INFO:root:Wrote PCA components file.
2018-09-26 23:20:41,342:INFO:root:PCA transformed data.
2018-09-26 23:20:41,342:INFO:root:Will call vbgmm with parameters: ./, 400, 1000
Do you know a way of checking the progress and estimating runtime?
Thanks!
error in bin_refinement
When I run : metawrap /home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/metawrap-modules/bin_refinement -o /home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref -A /home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binning/concoct_bins -B /home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binning/maxbin2_bins -C /home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binning/metabat2_bins
I get the error:
########################################################################################################################
##### BEGIN PIPELINE! #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- setting up output folder and copything over bins... -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- there are 13 bins in binsA -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- there are 4 bins in binsB -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- there are 5 bins in binsC -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- There are 3 bin sets! -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Fix contig naming by removing special characters... -----
------------------------------------------------------------------------------------------------------------------------
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.0.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.10.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.11.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.1.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.2.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.3.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.4.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.5.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.6.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.7.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.8.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/bin.9.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsA/unbinned.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsB/bin.0.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsB/bin.1.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsB/bin.2.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsB/bin.3.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsC/bin.1.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsC/bin.2.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsC/bin.3.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsC/bin.4.fa
/home-2/[email protected]/scratch/Binning/BAC_metagenomes_binning/AS_binref/binsC/bin.unbinned.fa
########################################################################################################################
##### BEGIN BIN REFINEMENT #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- There are three bin folders, so there 4 ways we can refine the bins (A+B, B+C, A+C, -----
----- A+B+C). Will try all four in parallel! -----
------------------------------------------------------------------------------------------------------------------------
Specified 2 input bin sets: -1 binsA -2 binsB
Add folder/bin name to contig name for binsA bins
Add folder/bin name to contig name for binsB bins
Combine all bins together
The number of refined bins: 4
Exporting refined bins...
Extracting refined bin: Refined_4.fasta
Deleting temporary files
All done!
Specified 2 input bin sets: -1 binsA -2 binsC
Add folder/bin name to contig name for binsA bins
Add folder/bin name to contig name for binsC bins
Combine all bins together
The number of refined bins: 6
Exporting refined bins...
Extracting refined bin: Refined_6.fasta
Deleting temporary files
All done!
Specified 2 input bin sets: -1 binsC -2 binsB
Add folder/bin name to contig name for binsC bins
Add folder/bin name to contig name for binsB bins
Combine all bins together
The number of refined bins: 6
Exporting refined bins...
Extracting refined bin: Refined_6.fasta
Deleting temporary files
All done!
Specified 3 input bin sets: -1 binsA -2 binsB -3 binsC
Add folder/bin name to contig name for binsA bins
Add folder/bin name to contig name for binsB bins
Add folder/bin name to contig name for binsC bins
Combine all bins together
The number of refined bins: 5
Exporting refined bins...
Extracting refined bin: Refined_5.fasta
Deleting temporary files
All done!
------------------------------------------------------------------------------------------------------------------------
----- there are 4 refined bins in binsAB -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- there are 6 refined bins in binsBC -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- there are 6 refined bins in binsAC -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- there are 5 refined bins in binsABC -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- Bin refinement finished successfully! -----
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----- fixing bin naming to .fa convention for consistancy... -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### RUNNING CHECKM ON ALL SETS OF BINS #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- Running CheckM on binsA bins -----
------------------------------------------------------------------------------------------------------------------------
/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
*******************************************************************************
[CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************
Identifying marker genes in 13 bins with 1 threads:
Process Process-2:ssing 0 of 13 (0.00%) bins.
Traceback (most recent call last):
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/markerGeneFinder.py", line 122, in __processBin
hmmModelFile = markerSetParser.createHmmModelFile(binId, markerFile)
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/markerSets.py", line 318, in createHmmModelFile
markerFileType = self.markerFileType(markerFile)
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/markerSets.py", line 418, in markerFileType
with open(markerFile, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/home-2/[email protected]/scratch/Database/CheckM/hmms/phylo.hmm'
Saving HMM info to file.
Calculating genome statistics for 13 bins with 1 threads:
Finished processing 13 of 13 (100.00%) bins.
Extracting marker genes to align.
[Error] Models must be parsed before identifying HMM hits.
Unexpected error: <type 'exceptions.AttributeError'>
Traceback (most recent call last):
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/bin/checkm", line 712, in <module>
checkmParser.parseOptions(args)
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 1244, in parseOptions
self.tree(options)
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 156, in tree
os.path.join(options.out_folder, 'storage', 'tree')
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/hmmerAligner.py", line 104, in makeAlignmentToPhyloMarkers
resultsParser.parseBinHits(outDir, hmmTableFile, False, bIgnoreThresholds, evalueThreshold, lengthThreshold)
File "/home-2/[email protected]/scratch/miniconda2/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 1205, in parseOptions
if options.bVerbose:
AttributeError: 'Namespace' object has no attribute 'bVerbose'
************************************************************************************************************************
***** Something went wrong with running CheckM. Exiting... *****
************************************************************************************************************************
real 0m48.700s
user 0m19.135s
sys 0m4.592s
assembly sop
My current understanding of how the metaWRAP pipeline should work involves doing QC on the demultiplexed libraries (I receive the data demultiplexed from the sequencing people), concatenating the outputs, and assembling.
I concatenate by repeatedly running this command on the output of the QC module:
cat $Trim_/Site-D1-DNA/final_pure_reads_1.fastq >> $Trim_/thrashLibs_1.fastq & \
cat $Trim_/Site-D1-DNA/final_pure_reads_2.fastq >> $Trim_/thrashLibs_2.fastq;
cat $Trim_/Site-D2-DNA/final_pure_reads_1.fastq >> $Trim_/thrashLibs_1.fastq & \
cat $Trim_/Site-D2-DNA/final_pure_reads_2.fastq >> $Trim_/thrashLibs_2.fastq;
However, when I assemble the huge pair of files produced by the above, I inconsistently get this error, depending on the input files:
Verification of expression 'irsl.eof() && irsr.eof()' failed in function 'void hammer::CorrectPairedReadFiles(const KMerData&, size_t&, size_t&, size_t&, size_t&, const string&, const string&, std::ofstream*, std::ofstream*, std::ofstream*, std::ofstream*, std::ofstream*)'. In file '/spades/src/projects/hammer/hammer_tools.cpp' on line 188. Message 'Pair of read files /home-3/[email protected]/scratch/THRASH_LIBS/TRIM/thrashLibs_1.fastq and /home-3/[email protected]/scratch/THRASH_LIBS/TRIM/thrashLibs_2.fastq contain unequal amount of reads'.
Verification of expression 'irsl.eof() && irsr.eof()' failed in function 'void hammer::CorrectPairedReadFiles(const KMerData&, size_t&, size_t&, size_t&, size_t&, const string&, const string&, std::ofstream*, std::ofstream*, std::ofstream*, std::ofstream*, std::ofstream*)'. In file '/spades/src/projects/hammer/hammer_tools.cpp' on line 188. Message 'Pair of read files /home-3/[email protected]/scratch/THRASH_LIBS/TRIM/thrashLibs_1.fastq and /home-3/[email protected]/scratch/THRASH_LIBS/TRIM/thrashLibs_2.fastq contain unequal amount of reads'.
hammer: /spades/src/projects/hammer/hammer_tools.cpp:188: void hammer::CorrectPairedReadFiles(const KMerData&, size_t&, size_t&, size_t&, size_t&, const string&, const string&, std::ofstream*, std::ofstream*, std::ofstream*, std::ofstream*, std::ofstream*): Assertion `irsl.eof() && irsr.eof()' failed.
== Error == system call for: "['/home-3/[email protected]/SPAdes-3.9.0-Linux/bin/hammer', '/home-3/[email protected]/scratch/THRASH_LIBS/DNA_ASSEMBLY/metaspades/corrected/configs/config.info']" finished abnormally, err code: -6
This is the assembly job script:
I assume this result is produced because of orphaned read pairs and am in the process of trying to resync the pairs using BBMap's repair.sh, but I'm not sure this is the right approach.
My questions are as follows:
- Is concatenation and "co-assembly" the correct way of creating contigs for binning?
- Is there any reason that orphaned reads would be in the "final pure read" files?
- If not, what is the source of this error?
Thanks!
Keith
issue with classification output
Hi,
I have run the Classify_bins module on over 100 bins. The output in some of them are correct (as I already double checked with another tool) but at least in one bin that I was expecting a specific taxonomy, the output is different. I have checked the contig_taxonomy.tab and the bin_taxonomy.tab files and this is what I have found for that specific bin.
contig_taxonomy.tab file:
RSF33_CG26_100 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_104
RSF33_CG26_105 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_106 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_107 Bacteria
RSF33_CG26_108
RSF33_CG26_109 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_110 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_111 Bacteria;Proteobacteria;Alphaproteobacteria;Rhodobacterales;Rhodobacteraceae
RSF33_CG26_12 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_13 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_17 Bacteria;Proteobacteria;Alphaproteobacteria;Rickettsiales;Rickettsiaceae
RSF33_CG26_18 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_19
RSF33_CG26_21 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_22 Bacteria
RSF33_CG26_24 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_28 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_3 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_30 Bacteria;Proteobacteria;Betaproteobacteria
RSF33_CG26_33 Bacteria;Proteobacteria;Alphaproteobacteria;Sphingomonadales;Sphingomonadaceae
RSF33_CG26_34 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_36 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_37 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_4
RSF33_CG26_44 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_45 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_47 Bacteria;Proteobacteria
RSF33_CG26_51 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_53 Bacteria;Proteobacteria
RSF33_CG26_57 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_58 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_59 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae;Nitrosomonas
RSF33_CG26_6 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_60 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_61 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_64
RSF33_CG26_7 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_72
RSF33_CG26_73
RSF33_CG26_74 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_75
RSF33_CG26_78 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae;Nitrosomonas
RSF33_CG26_8 Bacteria;Proteobacteria;Gammaproteobacteria;Methylococcales;Methylococcaceae
RSF33_CG26_83 Bacteria;Proteobacteria;Gammaproteobacteria;Alteromonadales;Alteromonadaceae
RSF33_CG26_84
RSF33_CG26_86 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_87 Bacteria
RSF33_CG26_89 Bacteria;Proteobacteria;Gammaproteobacteria;Alteromonadales;Pseudoalteromonadaceae
RSF33_CG26_9 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_90 Bacteria;Proteobacteria;Gammaproteobacteria;Methylococcales;Methylococcaceae;Methylomonas
RSF33_CG26_91 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_94 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae;Nitrosomonas;Nitrosomonas sp. Is79A3
RSF33_CG26_95 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_97 Bacteria;Proteobacteria;Betaproteobacteria;Nitrosomonadales;Nitrosomonadaceae
RSF33_CG26_99 Bacteria;Proteobacteria;Betaproteobacteria;Burkholderiales;Comamonadaceae
bin_taxonomy.tab file:
RSF33_CG26.fa Bacteria;Proteobacteria;Alphaproteobacteria;Rhodobacterales;Rhodobacteraceae
I'm pretty sure this bin belongs to Nitrosomonadaceae (ANI of 89% with other Nitrosomonas spp.), and that seems to be also the classfication for most of the contigs, however, the output says a totally different taxonomy.
I have checked in the contig_taxonomy.tab file and there are 319 contigs classified as Bacteria;Proteobacteria;Alphaproteobacteria;Rhodobacterales;Rhodobacteraceae, but any of them belongs to bin RSF33_CG26.
Any idea of what is going on? I'm not sure if this issue is also happening in other bins...
I'm using metaWRAP v=0.9.
normalize the relative abundance for each bin
Hi
I'm sorry, but could you please help me on this, so I have determined the relative abundance for each bin, but I wanted to normalize the values according to the overall metagenome, but I really not seeing the maths involved...
Genomic bins SampleMetaGenMerged_R1_val
bin.9 10.5060241613
bin.20 17.5038193854
bin.4 3.14227878024
bin.22 3.1807582388
bin.6 5.34662812579
bin.15 16.0117605048
bin.17 20.7550530435
bin.3 5.5673562303
bin.2 8.66532434566
#samples reads
SampleMetaGenMerged_R1_val 23887914
I did 10.5060241613 * 100 / 23887914, but i gives me a very low number like 4E-05
Is this correct?
Thanks for any help
Best
Igor
Maxbin fails but without informative error message
Hi German,
Thanks for making this software publicly available - I think it has a lot of promise! I'm especially interested in the potential for automation on the binning and refinement steps. Right now, I'm testing the binning module on a number of concatenated individual assemblies (done with metaSPAdes) from a time-series dataset. Here is my command:
metaWRAP binning --metabat2 --maxbin2 --concoct --run-checkm -t 30 -a 180429_all_anotop_contigs_concatenated_renamed.fasta -o 180429_all_anotop_contigs_concatenated_renamed_metaWRAP_binning_all3 paired-ano.312_1.fastq paired-ano.312_2.fastq paired-ano.317_1.fastq paired-ano.317_2.fastq paired-ano.322_1.fastq paired-ano.322_2.fastq paired-ano.323_1.fastq paired-ano.323_2.fastq paired-ano.324_1.fastq paired-ano.324_2.fastq paired-ano.327_1.fastq paired-ano.327_2.fastq paired-ano.415_1.fastq paired-ano.415_2.fastq paired-ano.5.21.2011_1.fastq paired-ano.5.21.2011_2.fastq paired-ano.5.22.2011_1.fastq paired-ano.5.22.2011_2.fastq paired-ano.6.16.2011_1.fastq paired-ano.6.16.2011_2.fastq paired-ano.6.2.2011_1.fastq paired-ano.6.2.2011_2.fastq > 180430_metaWRAP_out 2> 180430_metaWRAP_err
All of the initial steps (indexing, read mapping, metabat2) work without a problem but it fails upon getting to MaxBin. Stdout from metaWRAP is:
Get 114 seeds.
Start EM process.
Iteration 1
Error encountered while running core MaxBin program. Error recorded in 180429_all_anotop_contigs_concatenated_renamed_metaWRAP_binning_all3/work_files/maxbin2_out/bin.log.
Program Stop.
************************************************************************************************************************
***** Something went wrong with running MaxBin2. Exiting. *****
************************************************************************************************************************
The log file it refers to is not very informative, but here it is:
Reading seed list...
Looking for seeds in sequences.
David_TS_ano_5 [48.953000] [4.242860] [6.469040] [1.423680] [16.001400] [0.120895] [0.028697] [0.000100] [0.004699] [0.000100] [0.000100]
David_TS_ano_355 [22.467800] [0.713816] [3.164640] [2.159610] [4.285410] [0.179324] [0.289469] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1386 [30.206000] [1.599050] [3.217550] [0.707424] [1.211990] [0.000100] [0.096070] [0.000100] [0.000100] [0.092100] [0.000100]
David_TS_ano_1391 [11.291300] [3.972940] [0.663351] [0.596896] [0.118981] [0.432551] [0.119379] [0.000100] [0.146041] [0.000100] [0.000100]
David_TS_ano_1522 [4.764560] [2.847340] [0.622296] [0.623128] [2.102750] [0.605241] [0.124792] [0.080283] [0.000100] [0.000100] [0.000100]
David_TS_ano_1898 [18.409600] [10.645100] [2.775370] [1.616330] [0.107126] [0.000100] [0.283152] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1916 [18.180300] [1.485770] [2.793170] [0.454934] [4.000470] [0.124763] [0.209677] [0.000100] [0.000100] [0.000100] [0.041746]
David_TS_ano_3159 [13.834000] [1.337670] [0.264560] [0.152416] [0.737918] [0.092317] [0.000100] [0.000100] [0.106568] [0.000100] [0.000100]
David_TS_ano_7799 [5.840960] [0.307692] [0.764033] [0.647609] [0.461538] [0.019751] [0.154886] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_730436 [12.417700] [12.874100] [2.259670] [0.772464] [0.694795] [0.148441] [0.013257] [0.006420] [0.028862] [0.000100] [0.000100]
David_TS_ano_730560 [1.133360] [17.841400] [0.541113] [0.569778] [0.028537] [0.002809] [0.000100] [0.000100] [0.001341] [0.000100] [0.000100]
David_TS_ano_730579 [6.971000] [40.592300] [1.047130] [1.055060] [0.248085] [0.020451] [0.020451] [0.176402] [0.063338] [0.000100] [0.000100]
David_TS_ano_730768 [15.208200] [14.105200] [2.154660] [1.529330] [0.838760] [0.581611] [0.278230] [0.000100] [0.147255] [0.000100] [0.044458]
David_TS_ano_730963 [45.674700] [4.419970] [5.681600] [1.170600] [16.737800] [0.077328] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_730967 [0.356859] [2.836070] [1.043560] [2.866260] [1.808130] [0.158104] [0.119174] [0.000100] [0.060514] [0.117850] [0.081965]
David_TS_ano_731740 [4.923630] [41.596900] [0.853969] [0.446193] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.004860]
David_TS_ano_732618 [6.562280] [5.217590] [2.446320] [1.387070] [0.138898] [0.000100] [0.000100] [0.005734] [0.000100] [0.000100] [0.000100]
David_TS_ano_733044 [0.074100] [14.007100] [0.021171] [0.164785] [0.000100] [0.387438] [0.000100] [0.049400] [0.000100] [0.216655] [0.000100]
David_TS_ano_733561 [5.450100] [10.509600] [0.632877] [1.174170] [0.463405] [0.930724] [0.057534] [0.185519] [0.162035] [0.000100] [0.000100]
David_TS_ano_733722 [0.738833] [16.212900] [0.281690] [0.300201] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_733811 [3.525560] [20.677300] [0.239673] [0.965644] [0.384458] [0.122290] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_736609 [6.832650] [7.645990] [0.937975] [0.589819] [0.732592] [0.146284] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_736788 [1.539060] [11.271300] [0.443649] [0.177698] [0.176506] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_736919 [0.935464] [6.343790] [65.405900] [25.005400] [2.642340] [0.000100] [0.000100] [0.000100] [0.270808] [0.176719] [0.000100]
David_TS_ano_737877 [2.293930] [9.789030] [0.709340] [0.536904] [0.629001] [0.377531] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_742775 [16.885900] [6.485410] [0.859416] [1.580900] [0.194518] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_743205 [0.135257] [5.862940] [0.043282] [0.957619] [0.465284] [0.133454] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1625367 [47.922500] [4.223440] [6.224410] [1.519820] [15.331500] [0.148549] [0.021295] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1625468 [2.199630] [0.776684] [12.704100] [4.797500] [1.567900] [1.607030] [0.086989] [0.254188] [0.396910] [0.378929] [0.503491]
David_TS_ano_1625531 [0.704612] [0.064001] [13.563700] [1.666700] [0.323570] [0.030099] [0.019706] [0.455825] [0.905121] [0.879784] [1.784130]
David_TS_ano_1625591 [2.082400] [0.866084] [11.224300] [8.430510] [1.244620] [0.104982] [0.010235] [0.607653] [1.759120] [0.953823] [0.230086]
David_TS_ano_1625871 [2.570850] [1.315770] [18.255600] [13.810800] [1.209910] [0.073592] [0.016409] [0.761991] [1.652120] [0.714576] [0.481231]
David_TS_ano_1625990 [1.702610] [0.193277] [8.731030] [7.393290] [1.903460] [0.083136] [0.000100] [0.552441] [0.668292] [0.180191] [0.000449]
David_TS_ano_1626048 [1.315390] [0.347329] [14.703400] [4.824550] [0.959389] [0.074408] [0.102905] [0.276569] [0.574752] [0.538064] [0.348431]
David_TS_ano_1626060 [1.105890] [0.135190] [8.141720] [5.655880] [1.092070] [0.155881] [0.055131] [0.360922] [0.957645] [0.287391] [0.071657]
David_TS_ano_1626094 [0.931670] [0.000100] [10.279600] [2.867180] [1.222330] [0.058248] [0.000100] [0.854780] [1.242200] [1.162340] [0.605324]
David_TS_ano_1626186 [2.147400] [0.466688] [12.757000] [4.210840] [1.353870] [1.276200] [0.136906] [0.000100] [0.168071] [0.502783] [0.231038]
David_TS_ano_1628432 [4.416110] [3.971800] [113.177000] [48.290400] [4.402720] [0.179254] [0.000100] [0.504541] [0.758843] [0.436902] [0.150813]
David_TS_ano_1628534 [1.648300] [0.000100] [7.043530] [5.213480] [0.073783] [0.000100] [0.073783] [0.000100] [0.000100] [0.000100] [0.114363]
David_TS_ano_1629555 [0.024722] [0.758035] [4.465700] [1.696540] [0.813041] [0.138133] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1630322 [0.851904] [0.000100] [18.212600] [2.535970] [0.447814] [0.051834] [0.125529] [0.083921] [0.173131] [0.000100] [0.000100]
David_TS_ano_1630453 [1.295280] [0.795463] [9.289160] [6.244870] [1.403670] [0.612171] [0.108030] [0.776017] [0.359381] [0.426719] [0.168887]
David_TS_ano_1631573 [3.349310] [0.704077] [2.765870] [0.501892] [1.836070] [0.115595] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1632567 [0.829280] [0.000100] [11.491600] [3.400370] [0.420954] [0.000100] [0.000100] [0.202058] [0.204864] [0.575772] [0.724977]
David_TS_ano_1634503 [1.085410] [0.132002] [1.130340] [1.376590] [2.186910] [0.584027] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1637379 [1.591800] [0.371217] [9.006730] [2.099530] [0.967048] [0.000100] [0.000100] [0.088769] [0.322125] [1.225960] [0.271688]
David_TS_ano_1638017 [7.718860] [7.583160] [2.022270] [0.832985] [0.000100] [0.262352] [0.000100] [0.000100] [0.000100] [0.107864] [0.000100]
David_TS_ano_1639157 [4.522900] [3.198670] [4.265140] [0.000100] [1.328660] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_1640668 [0.817965] [0.000100] [2.441180] [1.666930] [0.476948] [0.000100] [0.000100] [0.395866] [0.000100] [0.000100] [0.000100]
David_TS_ano_1645458 [0.000100] [0.862632] [5.763690] [0.499520] [0.713737] [0.144092] [0.000100] [0.000100] [0.399616] [0.112392] [0.921230]
David_TS_ano_1649317 [0.000100] [0.000100] [1.533620] [1.529280] [1.471800] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_2996080 [2.409130] [1.559460] [10.009600] [12.373200] [1.212540] [0.044605] [0.007689] [0.634747] [1.782000] [0.616522] [0.481620]
David_TS_ano_2996122 [4.513190] [1.304410] [18.821500] [12.637100] [3.000980] [1.943800] [0.076998] [0.559758] [0.334133] [0.435885] [0.265847]
David_TS_ano_2996131 [0.594413] [0.773317] [3.739300] [40.224900] [3.799600] [1.284560] [0.102769] [0.015445] [0.008549] [0.041868] [0.008549]
David_TS_ano_2996142 [2.962690] [0.967779] [8.961520] [11.072500] [1.251280] [0.077384] [0.011386] [0.692550] [1.643360] [0.946184] [0.284337]
David_TS_ano_2996268 [1.173970] [0.279530] [6.433150] [9.904810] [1.633460] [0.126735] [0.016349] [0.417732] [0.929610] [0.103199] [0.079443]
David_TS_ano_2996426 [1.855080] [0.855780] [2.541570] [13.289600] [3.043380] [0.938231] [0.000100] [0.083148] [0.118802] [0.030989] [0.000100]
David_TS_ano_2996570 [1.793620] [0.763929] [9.653150] [5.592950] [1.242840] [1.043890] [0.048454] [0.437230] [0.141202] [0.352639] [0.433641]
David_TS_ano_2996625 [3.636250] [2.032790] [90.469000] [47.295700] [2.701760] [0.217077] [0.000100] [0.460320] [0.802462] [0.507833] [0.176020]
David_TS_ano_2996945 [0.469199] [3.147340] [0.959702] [3.922660] [1.819600] [0.148599] [0.125826] [0.039878] [0.146815] [0.043866] [0.052367]
David_TS_ano_2996974 [1.047300] [0.367731] [0.837364] [3.426100] [3.385680] [0.902397] [0.000100] [0.000100] [0.000100] [0.000100] [0.005912]
David_TS_ano_2998698 [0.389658] [0.877739] [2.634370] [6.273360] [1.490200] [0.113995] [0.000100] [0.000100] [0.000100] [0.175509] [0.000100]
David_TS_ano_2998791 [2.385500] [1.805090] [4.457200] [6.871890] [0.757884] [0.000100] [0.000100] [0.178452] [0.279530] [0.188051] [0.175906]
David_TS_ano_2998985 [44.288100] [4.036640] [5.901670] [1.733710] [12.851000] [0.122150] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_3001135 [0.525567] [0.000100] [6.337980] [5.273170] [0.368608] [0.000100] [0.081488] [0.000100] [0.114301] [0.132622] [0.130435]
David_TS_ano_3002581 [2.565540] [0.012395] [8.622560] [4.818410] [0.000100] [0.000100] [0.092966] [0.000100] [0.289123] [0.056089] [0.000100]
David_TS_ano_3003342 [1.990180] [0.392799] [2.861870] [5.333550] [1.532240] [0.104092] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_3004893 [0.128510] [1.570550] [0.754860] [7.016200] [0.821094] [0.214903] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_3007960 [1.293680] [0.391847] [13.658100] [5.320300] [0.487937] [1.376040] [0.224210] [0.206739] [0.534110] [0.000100] [0.177621]
David_TS_ano_3017611 [0.000100] [0.000100] [14.637800] [7.203820] [0.500281] [0.000100] [0.677709] [0.243683] [0.117911] [0.000100] [0.000100]
David_TS_ano_3036232 [0.239968] [0.000100] [0.239968] [2.114770] [2.156500] [0.677368] [0.000100] [0.000100] [0.000100] [0.000100] [0.016854]
David_TS_ano_3039232 [0.000100] [0.000100] [2.190440] [14.988300] [0.772651] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_3045704 [1.055150] [0.275735] [0.915441] [7.683820] [1.376840] [1.042280] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_3047761 [3.268180] [1.356940] [0.000100] [2.435320] [1.404150] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_3056215 [20.788000] [0.944386] [1.264430] [2.679960] [1.259180] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4929455 [47.502000] [4.200500] [6.500870] [1.439550] [16.006200] [0.137754] [0.030227] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4929716 [1.383080] [0.490530] [0.813763] [2.430430] [4.724490] [0.856818] [0.075758] [0.185606] [0.000100] [0.000100] [0.000100]
David_TS_ano_4929855 [0.931383] [0.581028] [1.188770] [1.742920] [5.138020] [1.317150] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4930322 [0.491078] [0.639208] [3.234660] [5.115860] [5.369840] [0.477389] [0.000100] [0.209484] [0.060132] [0.000100] [0.118553]
David_TS_ano_4930783 [3.934200] [0.884544] [1.594350] [0.513967] [6.167600] [0.139354] [0.000100] [0.091248] [0.000100] [0.000100] [0.000100]
David_TS_ano_4930919 [1.248520] [1.256730] [0.802364] [0.145108] [6.967500] [0.640840] [0.196980] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4931742 [0.128921] [0.000100] [1.669100] [25.076900] [2.181780] [0.227761] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4931962 [2.206060] [3.972440] [83.522800] [46.845000] [4.866240] [0.000100] [0.000100] [0.203344] [0.548125] [0.000100] [0.108450]
David_TS_ano_4934218 [4.309520] [1.683200] [0.569444] [1.162700] [1.544310] [0.382275] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4934865 [4.328330] [1.372680] [0.374821] [0.520744] [5.935620] [1.362660] [0.716023] [0.000100] [0.180258] [0.000100] [0.000100]
David_TS_ano_4937834 [0.000100] [1.806150] [0.755825] [1.581550] [1.881640] [0.000100] [0.000100] [0.000100] [0.417521] [0.000100] [0.000100]
David_TS_ano_4940995 [0.340522] [1.021570] [0.295119] [2.018160] [4.413170] [0.824064] [0.281498] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_4941615 [6.249710] [4.132470] [2.073860] [1.905040] [3.092610] [0.466588] [0.253224] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_5690863 [1.325330] [0.505689] [6.877370] [6.787610] [1.329540] [0.561736] [0.126422] [0.259166] [0.210282] [0.000100] [0.210704]
David_TS_ano_5692495 [1.120260] [1.173480] [9.255790] [3.915230] [0.947265] [2.072450] [0.098571] [0.000100] [0.194184] [0.376047] [0.000100]
David_TS_ano_5692722 [2.584130] [0.539930] [0.000100] [0.000100] [2.176290] [5.940230] [0.000100] [0.171271] [0.157207] [0.000100] [0.000100]
David_TS_ano_5693524 [0.000100] [0.000100] [11.042800] [2.700160] [2.866920] [2.470870] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_5699769 [0.084604] [0.000100] [0.046494] [0.044207] [1.326980] [6.060210] [0.474848] [0.000100] [0.000100] [0.622713] [0.000100]
David_TS_ano_5702422 [1.773650] [0.669764] [0.842905] [0.978885] [3.451860] [1.772800] [0.000100] [0.000100] [0.000100] [0.000100] [0.000100]
David_TS_ano_7044969 [0.927915] [0.159011] [1.473500] [0.000100] [1.626150] [0.896820] [3.605650] [0.345583] [0.000100] [0.000100] [0.000100]
David_TS_ano_7045639 [0.000100] [0.000100] [0.000100] [0.000100] [0.000100] [0.127680] [2.701750] [0.000100] [0.000100] [0.000100] [0.092593]
David_TS_ano_7045698 [0.000100] [0.059761] [0.225100] [0.000100] [2.295820] [0.525896] [4.408370] [0.000100] [0.455179] [0.000100] [0.000100]
David_TS_ano_7385320 [4.079170] [2.528180] [0.655148] [0.104362] [3.755630] [7.109840] [0.513444] [10.478300] [2.525990] [0.645389] [0.679546]
David_TS_ano_7414014 [4.331560] [0.454115] [7.010070] [5.013030] [0.598579] [0.000100] [0.000100] [2.865010] [0.878626] [0.000100] [0.293665]
David_TS_ano_8563571 [0.545433] [0.000100] [8.193900] [4.855100] [0.996725] [0.104666] [0.000100] [0.971699] [2.217990] [0.455385] [0.272015]
David_TS_ano_8568829 [1.433500] [0.746838] [12.561000] [7.939210] [0.393921] [0.000100] [0.060996] [0.629335] [1.531010] [0.282334] [0.442268]
David_TS_ano_8574267 [2.167960] [1.445030] [5.954470] [8.519430] [0.605775] [0.000100] [0.000100] [1.223490] [3.785120] [0.657135] [0.084398]
David_TS_ano_8580210 [2.015710] [2.452860] [59.946600] [30.378100] [1.685410] [0.000100] [0.000100] [0.000100] [3.150840] [0.132682] [0.161662]
David_TS_ano_8618486 [0.000100] [1.303120] [0.977520] [2.253080] [1.666420] [0.556925] [0.000100] [0.000100] [3.151560] [0.000100] [0.000100]
David_TS_ano_8668687 [0.000100] [0.000100] [3.766400] [0.278481] [0.315305] [0.000100] [0.000100] [0.000100] [2.470660] [0.000100] [1.714610]
David_TS_ano_10152906 [1.811570] [0.724210] [6.663570] [8.926120] [1.271610] [0.066682] [0.000100] [0.523699] [1.022300] [2.042520] [0.753717]
David_TS_ano_10157705 [1.761970] [0.729738] [5.676450] [3.873010] [0.260370] [0.000100] [0.000100] [0.293874] [1.314930] [1.847160] [0.000100]
David_TS_ano_10170959 [5.131900] [1.025750] [12.684700] [3.316340] [0.393064] [0.157120] [0.000100] [0.000100] [1.581710] [3.009460] [0.706253]
David_TS_ano_10204684 [0.000100] [0.000100] [4.405190] [1.043910] [0.743513] [0.000100] [0.000100] [0.000100] [0.422156] [2.522950] [0.000100]
David_TS_ano_10205216 [1.505530] [1.538690] [6.958790] [4.671360] [1.809050] [1.389950] [0.301508] [0.000100] [0.000100] [0.495477] [0.000100]
David_TS_ano_11030825 [0.000100] [0.082632] [8.211620] [0.890330] [0.303628] [0.041385] [0.054628] [0.367499] [1.007040] [0.602980] [1.780110]
David_TS_ano_11060180 [1.418470] [0.286203] [4.901940] [3.092360] [0.171038] [0.000100] [0.000100] [0.189852] [0.358039] [1.708100] [2.313000]
David_TS_ano_11071092 [0.028011] [0.000100] [0.014006] [1.239500] [0.000100] [0.000100] [0.000100] [0.721289] [0.325630] [1.978290] [3.464290]
David_TS_ano_11100614 [0.000100] [0.000100] [0.000100] [0.000100] [0.560543] [1.479120] [0.000100] [5.543840] [0.000100]
I also checked the stderr from metaWRAP but it's also a very generic error (only showing the end of the stderr file here):
Creating depth matrix file: 180429_all_anotop_contigs_concatenated_renamed_metaWRAP_binning_all3/work_files/mb2_master_depth.txt
Closing most bam files
Closing last bam file
Finished
readline() on closed filehandle FILE at /home/anaconda/miniconda2/envs/metawrap-env/bin/run_MaxBin.pl line 1334.
terminate called after throwing an instance of 'std::system_error'
what(): Resource temporarily unavailable
real 1171m24.138s
user 9829m3.676s
sys 757m45.564s
Since there is no obvious error with memory or not finding scripts etc, I'm not sure what's going on. I installed with conda following your instructions and recently updated to 0.8.4 through conda as well so everything should be up to date. It's running on a server with Ubuntu 16.04 and 1 Tb of memory so I don't think resources will be an issue. Any idea what might be going on here?
Thanks,
Jesse
Bin_refinement module error. Process killed at finalizing the refined bins.
Hi,
I ran the bin refinement module and everything went smoothly until it was finalizing the refined bins.
I obtained the following error:
Uncaught exception: Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")
Fatal error: exception Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")
Below are the details during the last part of the bin refinement module.
------------------------------------------------------------------------------------------------------------------------
----- You will find the best non-reassembled versions of the bins in binsO -----
------------------------------------------------------------------------------------------------------------------------
########################################################################################################################
##### FINALIZING THE REFINED BINS #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- Re-running CheckM on binsO bins -----
------------------------------------------------------------------------------------------------------------------------
*******************************************************************************
[CheckM - tree] Placing bins in reference genome tree.
*******************************************************************************
Identifying marker genes in 6 bins with 11 threads:
Finished processing 6 of 6 (100.00%) bins.
Saving HMM info to file.
Calculating genome statistics for 6 bins with 11 threads:
Finished processing 6 of 6 (100.00%) bins.
Extracting marker genes to align.
Parsing HMM hits to marker genes:
Finished parsing hits for 6 of 6 (100.00%) bins.
Extracting 43 HMMs with 11 threads:
Finished extracting 43 of 43 (100.00%) HMMs.
Aligning 43 marker genes with 11 threads:
Finished aligning 43 of 43 (100.00%) marker genes.
Reading marker alignment files.
Concatenating alignments.
Placing 6 bins into the genome tree with pplacer (be patient).
Killed
Uncaught exception: Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")
Fatal error: exception Sys_error("binsO.checkm/storage/tree/concatenated.pplacer.json: No such file or directory")
{ Current stage: 0:06:31.217 || Total: 0:06:31.217 }
*******************************************************************************
[CheckM - lineage_set] Inferring lineage-specific marker sets.
*******************************************************************************
Reading HMM info from file.
Parsing HMM hits to marker genes:
Finished parsing hits for 6 of 6 (100.00%) bins.
Determining marker sets for each genome bin.
Unexpected error: <type 'exceptions.IOError'>
Traceback (most recent call last):
File "/mnt/miniconda3/envs/metawrap-env/bin/checkm", line 712, in <module>
checkmParser.parseOptions(args)
File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 1245, in parseOptions
self.lineageSet(options)
File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/checkm/main.py", line 230, in lineageSet
resultsParser, options.unique, options.multi)
File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/checkm/treeParser.py", line 477, in getBinMarkerSets
tree = dendropy.Tree.get_from_path(treeFile, schema='newick', rooting="force-rooted", preserve_underscores=True)
File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/dendropy/datamodel/basemodel.py", line 216, in get_from_path
with open(src, "r", newline=None) as fsrc:
File "/mnt/miniconda3/envs/metawrap-env/lib/python2.7/site-packages/dendropy/utility/filesys.py", line 61, in pre_py34_open
buffering=buffering)
IOError: [Errno 2] No such file or directory: 'binsO.checkm/storage/tree/concatenated.tre'
************************************************************************************************************************
***** Something went wrong with running CheckM. Exiting... *****
************************************************************************************************************************
Thanks in advance!
R dependencies for Blobology
So I ran into an issue rendering blobplot PNGs on MARCC.
Error in .External2(C_X11, paste("png::", filename, sep = ""), g$width, :
unable to start device PNG
Calls: png
In addition: Warning message:
In png(paste(arg_input_file, taxlevel, "png", sep = "."), (numcols * :
unable to open connection to X11 display ''
Execution halted
I tracked it down to the lack of PNG support on some installations of R i.e.
> capabilities()
jpeg png tiff tcltk X11 aqua
FALSE FALSE FALSE FALSE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE TRUE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE FALSE FALSE FALSE TRUE TRUE
I got around this by installing R and one other random package into my metawrap-env using conda:
$ conda install r-essentials
Fetching package metadata ...................
Solving package specifications: .
Package plan for installation in environment /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env:
The following NEW packages will be INSTALLED:
backports: 1.0-py27_1 conda-forge
backports.shutil_get_terminal_size: 1.0.0-py_3 conda-forge
backports_abc: 0.5-py27_0 conda-forge
bleach: 1.4.2-py27_0 bioconda
configparser: 3.5.0-py27_0 conda-forge
decorator: 4.1.2-py27_0 conda-forge
entrypoints: 0.2.3-py27_1 conda-forge
functools32: 3.2.3.2-py27_2 conda-forge
glib: 2.50.2-1
harfbuzz: 0.9.39-2
html5lib: 1.0.1-py_0 conda-forge
ipykernel: 4.8.1-py27_0 conda-forge
ipython: 5.5.0-py27_0 conda-forge
ipython_genutils: 0.2.0-py27_0 conda-forge
jsonschema: 2.6.0-py27_1 conda-forge
jupyter_client: 5.2.2-py27_0 conda-forge
jupyter_core: 4.4.0-py_0 conda-forge
libsodium: 1.0.15-1 conda-forge
mistune: 0.8.3-py_0 conda-forge
nbconvert: 5.3.1-py_1 conda-forge
nbformat: 4.4.0-py27_0 conda-forge
notebook: 5.4.0-py27_0 conda-forge
pandoc: 2.0.0.1-0 conda-forge
pandocfilters: 1.4.1-py27_0 conda-forge
pango: 1.40.3-1
pathlib2: 2.3.0-py27_0 conda-forge
pexpect: 4.4.0-py27_0 conda-forge
pickleshare: 0.7.4-py27_0 conda-forge
prompt_toolkit: 1.0.15-py27_0 conda-forge
ptyprocess: 0.5.2-py27_0 conda-forge
pyzmq: 16.0.1-py27_0 conda-forge
r-assertthat: 0.2.0-r3.4.1_0 r
r-backports: 1.1.0-r3.4.1_0 r
r-base: 3.4.1-0 conda-forge
r-base64enc: 0.1_3-r3.4.1_0 r
r-bh: 1.62.0_1-r3.4.1_0 r
r-bindr: 0.1-r3.4.1_0 r
r-bindrcpp: 0.2-r3.4.1_0 r
r-bitops: 1.0_6-r3.4.1_2 r
r-boot: 1.3_19-r3.4.1_0 r
r-broom: 0.4.2-r3.4.1_0 r
r-car: 2.1_4-r3.4.1_0 r
r-caret: 6.0_76-r3.4.1_0 r
r-catools: 1.17.1-r3.4.1_2 r
r-cellranger: 1.1.0-r3.4.1_0 r
r-class: 7.3_14-r3.4.1_0 r
r-cluster: 2.0.6-r3.4.1_0 r
r-codetools: 0.2_15-r3.4.1_0 r
r-colorspace: 1.3_2-r3.4.1_0 r
r-crayon: 1.3.2-r3.4.1_0 r
r-curl: 2.6-r3.4.1_0 r
r-data.table: 1.10.4-r3.4.1_0 r
r-dbi: 0.6_1-r3.4.1_0 r
r-dichromat: 2.0_0-r3.4.1_2 r
r-digest: 0.6.12-r3.4.1_0 r
r-dplyr: 0.7.0-r3.4.1_0 r
r-essentials: 1.6.0-r3.4.1_0 r
r-evaluate: 0.10-r3.4.1_0 r
r-forcats: 0.2.0-r3.4.1_0 r
r-foreach: 1.4.3-r3.4.1_0 r
r-foreign: 0.8_68-r3.4.1_0 r
r-formatr: 1.5-r3.4.1_0 r
r-ggplot2: 2.2.1-r3.4.1_0 r
r-gistr: 0.4.0-r3.4.1_0 r
r-glmnet: 2.0_10-r3.4.1_0 r
r-glue: 1.1.1-r3.4.1_0 r
r-gtable: 0.2.0-r3.4.1_0 r
r-haven: 1.0.0-r3.4.1_0 r
r-hexbin: 1.27.1-r3.4.1_0 r
r-highr: 0.6-r3.4.1_0 r
r-hms: 0.3-r3.4.1_0 r
r-htmltools: 0.3.6-r3.4.1_0 r
r-htmlwidgets: 0.8-r3.4.1_1 r
r-httpuv: 1.3.3-r3.4.1_0 r
r-httr: 1.2.1-r3.4.1_0 r
r-irdisplay: 0.4.4-r3.4.1_0 r
r-irkernel: 0.7.1-r3.4.1_0 r
r-iterators: 1.0.8-r3.4.1_0 r
r-jsonlite: 1.5-r3.4.1_0 r
r-kernsmooth: 2.23_15-r3.4.1_0 r
r-knitr: 1.16-r3.4.1_0 r
r-labeling: 0.3-r3.4.1_2 r
r-lattice: 0.20_35-r3.4.1_0 r
r-lazyeval: 0.2.0-r3.4.1_0 r
r-lme4: 1.1_13-r3.4.1_0 r
r-lubridate: 1.6.0-r3.4.1_0 r
r-magrittr: 1.5-r3.4.1_2 r
r-maps: 3.2.0-r3.4.1_0 r
r-markdown: 0.8-r3.4.1_0 r
r-mass: 7.3_47-r3.4.1_0 r
r-matrix: 1.2_10-r3.4.1_0 r
r-matrixmodels: 0.4_1-r3.4.1_0 r
r-mgcv: 1.8_17-r3.4.1_0 r
r-mime: 0.5-r3.4.1_0 r
r-minqa: 1.2.4-r3.4.1_2 r
r-mnormt: 1.5_5-r3.4.1_0 r
r-modelmetrics: 1.1.0-r3.4.1_0 r
r-modelr: 0.1.0-r3.4.1_0 r
r-munsell: 0.4.3-r3.4.1_0 r
r-nlme: 3.1_131-r3.4.1_0 r
r-nloptr: 1.0.4-r3.4.1_2 r
r-nnet: 7.3_12-r3.4.1_0 r
r-openssl: 0.9.6-r3.4.1_0 r
r-pbdzmq: 0.2_6-r3.4.1_0 r
r-pbkrtest: 0.4_7-r3.4.1_0 r
r-pkgconfig: 2.0.1-r3.4.1_0 r
r-plogr: 0.1_1-r3.4.1_0 r
r-plyr: 1.8.4-r3.4.1_0 r
r-pryr: 0.1.2-r3.4.1_0 r
r-psych: 1.7.5-r3.4.1_0 r
r-purrr: 0.2.2.2-r3.4.1_0 r
r-quantmod: 0.4_9-r3.4.1_0 r
r-quantreg: 5.33-r3.4.1_0 r
r-r6: 2.2.1-r3.4.1_0 r
r-randomforest: 4.6_12-r3.4.1_0 r
r-rbokeh: 0.5.0-r3.4.1_0 r
r-rcolorbrewer: 1.1_2-r3.4.1_3 r
r-rcpp: 0.12.11-r3.4.1_0 r
r-rcppeigen: 0.3.3.3.0-r3.4.1_0 r
r-readr: 1.1.1-r3.4.1_0 r
r-readxl: 1.0.0-r3.4.1_0 r
r-recommended: 3.4.1-r3.4.1_0 r
r-rematch: 1.0.1-r3.4.1_0 r
r-repr: 0.10-r3.4.1_0 r
r-reshape2: 1.4.2-r3.4.1_0 r
r-rlang: 0.1.1-r3.4.1_0 r
r-rmarkdown: 1.5-r3.4.1_0 r
r-rpart: 4.1_11-r3.4.1_0 r
r-rprojroot: 1.2-r3.4.1_0 r
r-rvest: 0.3.2-r3.4.1_0 r
r-scales: 0.4.1-r3.4.1_0 r
r-selectr: 0.3_1-r3.4.1_0 r
r-shiny: 1.0.3-r3.4.1_0 r
r-sourcetools: 0.1.6-r3.4.1_0 r
r-sparsem: 1.77-r3.4.1_0 r
r-spatial: 7.3_11-r3.4.1_0 r
r-stringi: 1.1.5-r3.4.1_0 r
r-stringr: 1.2.0-r3.4.1_0 r
r-survival: 2.41_3-r3.4.1_0 r
r-tibble: 1.3.3-r3.4.1_0 r
r-tidyr: 0.6.3-r3.4.1_0 r
r-tidyverse: 1.1.1-r3.4.1_0 r
r-ttr: 0.23_1-r3.4.1_0 r
r-uuid: 0.1_2-r3.4.1_0 r
r-xml2: 1.1.1-r3.4.1_0 r
r-xtable: 1.8_2-r3.4.1_0 r
r-xts: 0.9_7-r3.4.1_2 r
r-yaml: 2.1.14-r3.4.1_0 r
r-zoo: 1.8_0-r3.4.1_0 r
scandir: 1.6-py27_0 conda-forge
send2trash: 1.4.2-py_0 conda-forge
simplegeneric: 0.8.1-py27_0 conda-forge
singledispatch: 3.4.0.3-py27_0 conda-forge
ssl_match_hostname: 3.5.0.1-py27_1 conda-forge
terminado: 0.8.1-py27_0 conda-forge
testpath: 0.3.1-py27_0 conda-forge
tornado: 4.5.3-py27_0 conda-forge
traitlets: 4.3.2-py27_0 conda-forge
wcwidth: 0.1.7-py27_0 conda-forge
webencodings: 0.5-py27_0 conda-forge
zeromq: 4.1.5-0 conda-forge
The following packages will be UPDATED:
libedit: 3.1-heed3624_0 --> 3.1.20170329-0 conda-forge
The following packages will be SUPERSEDED by a higher-priority channel:
ncurses: 6.0-h9df7e31_2 --> 5.9-10 conda-forge
Proceed ([y]/n)? y
libsodium-1.0. 100% |#############################################################################################################| Time: 0:00:00 10.23 MB/s
libedit-3.1.20 100% |#############################################################################################################| Time: 0:00:00 36.34 MB/s
pandoc-2.0.0.1 100% |#############################################################################################################| Time: 0:00:00 51.95 MB/s
zeromq-4.1.5-0 100% |#############################################################################################################| Time: 0:00:00 62.14 MB/s
glib-2.50.2-1. 100% |#############################################################################################################| Time: 0:00:00 19.98 MB/s
configparser-3 100% |#############################################################################################################| Time: 0:00:00 15.36 MB/s
decorator-4.1. 100% |#############################################################################################################| Time: 0:00:00 8.81 MB/s
functools32-3. 100% |#############################################################################################################| Time: 0:00:00 11.31 MB/s
ipython_genuti 100% |#############################################################################################################| Time: 0:00:00 14.65 MB/s
mistune-0.8.3- 100% |#############################################################################################################| Time: 0:00:00 6.92 MB/s
pandocfilters- 100% |#############################################################################################################| Time: 0:00:00 5.59 MB/s
ptyprocess-0.5 100% |#############################################################################################################| Time: 0:00:00 10.42 MB/s
pyzmq-16.0.1-p 100% |#############################################################################################################| Time: 0:00:00 48.65 MB/s
scandir-1.6-py 100% |#############################################################################################################| Time: 0:00:00 19.13 MB/s
send2trash-1.4 100% |#############################################################################################################| Time: 0:00:00 4.85 MB/s
simplegeneric- 100% |#############################################################################################################| Time: 0:00:00 3.40 MB/s
wcwidth-0.1.7- 100% |#############################################################################################################| Time: 0:00:00 10.46 MB/s
webencodings-0 100% |#############################################################################################################| Time: 0:00:00 8.62 MB/s
backports.shut 100% |#############################################################################################################| Time: 0:00:00 3.65 MB/s
entrypoints-0. 100% |#############################################################################################################| Time: 0:00:00 4.64 MB/s
harfbuzz-0.9.3 100% |#############################################################################################################| Time: 0:00:00 21.12 MB/s
html5lib-1.0.1 100% |#############################################################################################################| Time: 0:00:00 25.29 MB/s
jsonschema-2.6 100% |#############################################################################################################| Time: 0:00:00 24.66 MB/s
pathlib2-2.3.0 100% |#############################################################################################################| Time: 0:00:00 14.20 MB/s
pexpect-4.4.0- 100% |#############################################################################################################| Time: 0:00:00 25.34 MB/s
traitlets-4.3. 100% |#############################################################################################################| Time: 0:00:00 29.04 MB/s
bleach-1.4.2-p 100% |#############################################################################################################| Time: 0:00:00 14.52 MB/s
jupyter_core-4 100% |#############################################################################################################| Time: 0:00:00 20.09 MB/s
pango-1.40.3-1 100% |#############################################################################################################| Time: 0:00:00 22.80 MB/s
pickleshare-0. 100% |#############################################################################################################| Time: 0:00:00 5.89 MB/s
testpath-0.3.1 100% |#############################################################################################################| Time: 0:00:00 27.16 MB/s
jupyter_client 100% |#############################################################################################################| Time: 0:00:00 30.45 MB/s
nbformat-4.4.0 100% |#############################################################################################################| Time: 0:00:00 33.34 MB/s
prompt_toolkit 100% |#############################################################################################################| Time: 0:00:00 38.40 MB/s
r-base-3.4.1-0 100% |#############################################################################################################| Time: 0:00:00 66.06 MB/s
terminado-0.8. 100% |#############################################################################################################| Time: 0:00:00 9.06 MB/s
ipython-5.5.0- 100% |#############################################################################################################| Time: 0:00:00 56.86 MB/s
nbconvert-5.3. 100% |#############################################################################################################| Time: 0:00:00 46.48 MB/s
r-assertthat-0 100% |#############################################################################################################| Time: 0:00:00 19.41 MB/s
r-backports-1. 100% |#############################################################################################################| Time: 0:00:00 16.38 MB/s
r-base64enc-0. 100% |#############################################################################################################| Time: 0:00:00 549.00 kB/s
r-bh-1.62.0_1- 100% |#############################################################################################################| Time: 0:00:00 61.51 MB/s
r-bindr-0.1-r3 100% |#############################################################################################################| Time: 0:00:00 8.78 MB/s
r-bitops-1.0_6 100% |#############################################################################################################| Time: 0:00:00 11.92 MB/s
r-boot-1.3_19- 100% |#############################################################################################################| Time: 0:00:00 52.50 MB/s
r-cluster-2.0. 100% |#############################################################################################################| Time: 0:00:00 51.62 MB/s
r-codetools-0. 100% |#############################################################################################################| Time: 0:00:00 17.43 MB/s
r-colorspace-1 100% |#############################################################################################################| Time: 0:00:00 50.84 MB/s
r-crayon-1.3.2 100% |#############################################################################################################| Time: 0:00:00 53.98 MB/s
r-curl-2.6-r3. 100% |#############################################################################################################| Time: 0:00:00 45.32 MB/s
r-data.table-1 100% |#############################################################################################################| Time: 0:00:00 55.59 MB/s
r-dbi-0.6_1-r3 100% |#############################################################################################################| Time: 0:00:00 55.34 MB/s
r-dichromat-2. 100% |#############################################################################################################| Time: 0:00:00 37.47 MB/s
r-digest-0.6.1 100% |#############################################################################################################| Time: 0:00:00 30.98 MB/s
r-foreign-0.8_ 100% |#############################################################################################################| Time: 0:00:00 43.51 MB/s
r-formatr-1.5- 100% |#############################################################################################################| Time: 0:00:00 33.97 MB/s
r-glue-1.1.1-r 100% |#############################################################################################################| Time: 0:00:00 3.40 MB/s
r-gtable-0.2.0 100% |#############################################################################################################| Time: 0:00:00 22.59 MB/s
r-highr-0.6-r3 100% |#############################################################################################################| Time: 0:00:00 15.04 MB/s
r-hms-0.3-r3.4 100% |#############################################################################################################| Time: 0:00:00 11.60 MB/s
r-iterators-1. 100% |#############################################################################################################| Time: 0:00:00 29.67 MB/s
r-jsonlite-1.5 100% |#############################################################################################################| Time: 0:00:00 41.70 MB/s
r-kernsmooth-2 100% |#############################################################################################################| Time: 0:00:00 28.16 MB/s
r-labeling-0.3 100% |#############################################################################################################| Time: 0:00:00 18.85 MB/s
r-lattice-0.20 100% |#############################################################################################################| Time: 0:00:00 16.25 MB/s
r-lazyeval-0.2 100% |#############################################################################################################| Time: 0:00:00 31.38 MB/s
r-magrittr-1.5 100% |#############################################################################################################| Time: 0:00:00 39.30 MB/s
r-maps-3.2.0-r 100% |#############################################################################################################| Time: 0:00:00 60.46 MB/s
r-mass-7.3_47- 100% |#############################################################################################################| Time: 0:00:00 57.30 MB/s
r-mime-0.5-r3. 100% |#############################################################################################################| Time: 0:00:00 13.51 MB/s
r-mnormt-1.5_5 100% |#############################################################################################################| Time: 0:00:00 23.97 MB/s
r-nloptr-1.0.4 100% |#############################################################################################################| Time: 0:00:00 55.29 MB/s
r-nnet-7.3_12- 100% |#############################################################################################################| Time: 0:00:00 32.01 MB/s
r-openssl-0.9. 100% |#############################################################################################################| Time: 0:00:00 49.30 MB/s
r-pkgconfig-2. 100% |#############################################################################################################| Time: 0:00:00 9.96 MB/s
r-plogr-0.1_1- 100% |#############################################################################################################| Time: 0:00:00 8.15 MB/s
r-r6-2.2.1-r3. 100% |#############################################################################################################| Time: 0:00:00 39.74 MB/s
r-randomforest 100% |#############################################################################################################| Time: 0:00:00 34.95 MB/s
r-rcolorbrewer 100% |#############################################################################################################| Time: 0:00:00 13.77 MB/s
r-rcpp-0.12.11 100% |#############################################################################################################| Time: 0:00:00 53.11 MB/s
r-rematch-1.0. 100% |#############################################################################################################| Time: 0:00:00 7.33 MB/s
r-repr-0.10-r3 100% |#############################################################################################################| Time: 0:00:00 24.42 MB/s
r-rlang-0.1.1- 100% |#############################################################################################################| Time: 0:00:00 49.76 MB/s
r-rpart-4.1_11 100% |#############################################################################################################| Time: 0:00:00 57.68 MB/s
r-sourcetools- 100% |#############################################################################################################| Time: 0:00:00 24.51 MB/s
r-sparsem-1.77 100% |#############################################################################################################| Time: 0:00:00 56.19 MB/s
r-spatial-7.3_ 100% |#############################################################################################################| Time: 0:00:00 36.04 MB/s
r-stringi-1.1. 100% |#############################################################################################################| Time: 0:00:00 52.05 MB/s
r-uuid-0.1_2-r 100% |#############################################################################################################| Time: 0:00:00 11.07 MB/s
r-xtable-1.8_2 100% |#############################################################################################################| Time: 0:00:00 53.88 MB/s
r-yaml-2.1.14- 100% |#############################################################################################################| Time: 0:00:00 26.65 MB/s
ipykernel-4.8. 100% |#############################################################################################################| Time: 0:00:00 25.02 MB/s
r-bindrcpp-0.2 100% |#############################################################################################################| Time: 0:00:00 28.34 MB/s
r-catools-1.17 100% |#############################################################################################################| Time: 0:00:00 25.80 MB/s
r-class-7.3_14 100% |#############################################################################################################| Time: 0:00:00 27.74 MB/s
r-foreach-1.4. 100% |#############################################################################################################| Time: 0:00:00 34.97 MB/s
r-hexbin-1.27. 100% |#############################################################################################################| Time: 0:00:00 43.33 MB/s
r-htmltools-0. 100% |#############################################################################################################| Time: 0:00:00 34.89 MB/s
r-httpuv-1.3.3 100% |#############################################################################################################| Time: 0:00:00 44.50 MB/s
r-httr-1.2.1-r 100% |#############################################################################################################| Time: 0:00:00 41.23 MB/s
r-irdisplay-0. 100% |#############################################################################################################| Time: 0:00:00 12.64 MB/s
r-markdown-0.8 100% |#############################################################################################################| Time: 0:00:00 31.47 MB/s
r-matrix-1.2_1 100% |#############################################################################################################| Time: 0:00:00 57.97 MB/s
r-minqa-1.2.4- 100% |#############################################################################################################| Time: 0:00:00 4.07 MB/s
r-modelmetrics 100% |#############################################################################################################| Time: 0:00:00 5.10 MB/s
r-munsell-0.4. 100% |#############################################################################################################| Time: 0:00:00 5.46 MB/s
r-nlme-3.1_131 100% |#############################################################################################################| Time: 0:00:00 18.71 MB/s
r-pbdzmq-0.2_6 100% |#############################################################################################################| Time: 0:00:00 48.12 MB/s
r-plyr-1.8.4-r 100% |#############################################################################################################| Time: 0:00:00 1.04 MB/s
r-rprojroot-1. 100% |#############################################################################################################| Time: 0:00:00 12.93 MB/s
r-stringr-1.2. 100% |#############################################################################################################| Time: 0:00:00 310.75 kB/s
r-tibble-1.3.3 100% |#############################################################################################################| Time: 0:00:00 10.97 MB/s
r-xml2-1.1.1-r 100% |#############################################################################################################| Time: 0:00:00 18.39 MB/s
r-zoo-1.8_0-r3 100% |#############################################################################################################| Time: 0:00:00 39.56 MB/s
notebook-5.4.0 100% |#############################################################################################################| Time: 0:00:00 62.24 MB/s
r-cellranger-1 100% |#############################################################################################################| Time: 0:00:00 27.39 MB/s
r-dplyr-0.7.0- 100% |#############################################################################################################| Time: 0:00:00 59.05 MB/s
r-evaluate-0.1 100% |#############################################################################################################| Time: 0:00:00 19.41 MB/s
r-forcats-0.2. 100% |#############################################################################################################| Time: 0:00:00 37.66 MB/s
r-glmnet-2.0_1 100% |#############################################################################################################| Time: 0:00:00 57.75 MB/s
r-htmlwidgets- 100% |#############################################################################################################| Time: 0:00:00 54.40 MB/s
r-lubridate-1. 100% |#############################################################################################################| Time: 0:00:00 55.23 MB/s
r-matrixmodels 100% |#############################################################################################################| Time: 0:00:00 42.61 MB/s
r-mgcv-1.8_17- 100% |#############################################################################################################| Time: 0:00:00 45.85 MB/s
r-pryr-0.1.2-r 100% |#############################################################################################################| Time: 0:00:00 41.01 MB/s
r-psych-1.7.5- 100% |#############################################################################################################| Time: 0:00:00 61.56 MB/s
r-purrr-0.2.2. 100% |#############################################################################################################| Time: 0:00:00 37.68 MB/s
r-rcppeigen-0. 100% |#############################################################################################################| Time: 0:00:00 61.56 MB/s
r-readr-1.1.1- 100% |#############################################################################################################| Time: 0:00:00 53.70 MB/s
r-reshape2-1.4 100% |#############################################################################################################| Time: 0:00:00 33.50 MB/s
r-scales-0.4.1 100% |#############################################################################################################| Time: 0:00:00 46.36 MB/s
r-selectr-0.3_ 100% |#############################################################################################################| Time: 0:00:00 38.46 MB/s
r-shiny-1.0.3- 100% |#############################################################################################################| Time: 0:00:00 60.12 MB/s
r-survival-2.4 100% |#############################################################################################################| Time: 0:00:00 61.03 MB/s
r-xts-0.9_7-r3 100% |#############################################################################################################| Time: 0:00:00 52.26 MB/s
r-ggplot2-2.2. 100% |#############################################################################################################| Time: 0:00:00 36.45 MB/s
r-haven-1.0.0- 100% |#############################################################################################################| Time: 0:00:00 42.58 MB/s
r-irkernel-0.7 100% |#############################################################################################################| Time: 0:00:00 33.89 MB/s
r-knitr-1.16-r 100% |#############################################################################################################| Time: 0:00:00 55.56 MB/s
r-lme4-1.1_13- 100% |#############################################################################################################| Time: 0:00:00 46.00 MB/s
r-quantreg-5.3 100% |#############################################################################################################| Time: 0:00:00 24.62 MB/s
r-readxl-1.0.0 100% |#############################################################################################################| Time: 0:00:00 55.54 MB/s
r-recommended- 100% |#############################################################################################################| Time: 0:00:00 1.33 MB/s
r-rvest-0.3.2- 100% |#############################################################################################################| Time: 0:00:00 53.30 MB/s
r-tidyr-0.6.3- 100% |#############################################################################################################| Time: 0:00:00 48.68 MB/s
r-ttr-0.23_1-r 100% |#############################################################################################################| Time: 0:00:00 5.41 MB/s
r-broom-0.4.2- 100% |#############################################################################################################| Time: 0:00:00 59.32 MB/s
r-pbkrtest-0.4 100% |#############################################################################################################| Time: 0:00:00 39.01 MB/s
r-quantmod-0.4 100% |#############################################################################################################| Time: 0:00:00 52.00 MB/s
r-rmarkdown-1. 100% |#############################################################################################################| Time: 0:00:00 60.17 MB/s
r-car-2.1_4-r3 100% |#############################################################################################################| Time: 0:00:00 59.15 MB/s
r-gistr-0.4.0- 100% |#############################################################################################################| Time: 0:00:00 59.79 MB/s
r-modelr-0.1.0 100% |#############################################################################################################| Time: 0:00:00 32.78 MB/s
r-caret-6.0_76 100% |#############################################################################################################| Time: 0:00:00 58.57 MB/s
r-rbokeh-0.5.0 100% |#############################################################################################################| Time: 0:00:00 55.61 MB/s
r-tidyverse-1. 100% |#############################################################################################################| Time: 0:00:00 16.83 MB/s
r-essentials-1 100% |#############################################################################################################| Time: 0:00:00 2.32 MB/s
$ conda install readline
Fetching package metadata ...................
Solving package specifications: .
Package plan for installation in environment /home-3/[email protected]/scratch/miniconda2/envs/metawrap-env:
The following packages will be SUPERSEDED by a higher-priority channel:
readline: 6.2-2 --> 6.2-0 conda-forge
Proceed ([y]/n)? y
readline-6.2-0 100% |#############################################################################################################| Time: 0:00:00 10.34 MB/s
And tada:
> capabilities()
jpeg png tiff tcltk X11 aqua
TRUE TRUE TRUE TRUE FALSE FALSE
http/ftp sockets libxml fifo cledit iconv
TRUE TRUE TRUE TRUE TRUE TRUE
NLS profmem cairo ICU long.double libcurl
TRUE TRUE TRUE TRUE TRUE TRUE
I am currently testing the fix, but it seems to be working fine. I just figured I'd post this here to let others know my solution and put the idea of including R dependencies in the conda installation process in your head. ✌️
checkm error in bin_refinement
In the bin_refinement module log I am seeing this checkm error:
[CheckM - analyze] Identifying marker genes in bins.
*******************************************************************************
Identifying marker genes in 47 bins with 16 threads:
Finished processing 47 of 47 (100.00%) bins.
Traceback (most recent call last):
File "/home/yunha/miniconda2/envs/metawrap-env/lib/python2.7/multiprocessing/util.py", line 274, in _run_finalizers
finalizer()
File "/home/yunha/miniconda2/envs/metawrap-env/lib/python2.7/multiprocessing/util.py", line 207, in __call__
res = self._callback(*self._args, **self._kwargs)
File "/home/yunha/miniconda2/envs/metawrap-env/lib/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/home/yunha/miniconda2/envs/metawrap-env/lib/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 16] Device or resource busy: 'binsA.tmp/pymp-yl1pXc/.nfs00000000011a2d100000022f'
Saving HMM info to file.
And as a result (I believe) I am getting very low completeness values for the bins and thus, "There are 0 'good' bins found in binsA! (>50% completion and <10% contamination)" log for all combinations of bin sets.
Do you know what this error is?
kraken module needs inputs with specific nomenclature (*_1* and *_2*)
Hello Gherman,
again, congratulations for this amazing tool! I have a very small remark:
when using the kraken module, one has to be aware that the tool will not consider inputs that don't have a specific notation: *_1.fq and *_2.fq (or, instead of fq, fastq, etc).
When I attempted to run the kraken module with inputs files with the given names: seawater1.fastq and seawater2.fastq, it didn't work.
$ metawrap kraken -o seawater -t 2 ../READ_QC/seawater1.fastq ../READ_QC/seawater2.fastq
metawrap kraken -o seawater -t 2 ../READ_QC/seawater1.fastq ../READ_QC/seawater2.fastq
########################################################################################################################
##### RUNNING KRAKEN ON ALL FILES #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- No fasta or fastq files detected! (must be in .fastq .fa .fastq or .fq format) -----
------------------------------------------------------------------------------------------------------------------------
...
But when I simply renamed the files to seawater_1.fastq and seawater_2.fastq, it worked just fine.
$ metawrap kraken -o seawater -t 2 ../READ_QC/seawater_1.fastq ../READ_QC/seawater_2.fastq
metawrap kraken -o seawater -t 2 ../READ_QC/seawater_1.fastq ../READ_QC/seawater_2.fastq
########################################################################################################################
##### RUNNING KRAKEN ON ALL FILES #####
########################################################################################################################
------------------------------------------------------------------------------------------------------------------------
----- Now processing ../READ_QC/seawater_1.fastq and ../READ_QC/seawater_2.fastq -----
------------------------------------------------------------------------------------------------------------------------
Just a small thing that can be considered either on the documentation of maybe on your script...
Still, amazing tool! Extremely user friendly, easy-to-use set of modules. Thank you!
Cheers,
Rodolfo
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.