gtftk peak_anno -i pygtftk/data/mini_real/mini_real.gtf.gz -o toto -c pygtftk/data/mini_real/hg38.genome -p pygtftk/data/mini_real/ENCFF112BHN_H3K4me3_K562_sub.bed -V 3
|-- 11:27:24-INFO-peak_anno : Checking chromosome info file.
|-- 11:27:24-INFO-peak_anno : Instantiating a GTF.
|-- 11:27:25-DEBUG-peak_anno : Calling extract_data_iter.
|-- 11:27:25-DEBUG-peak_anno : Ensembl format detected.
|-- 11:27:25-DEBUG_MEM-peak_anno : GTF created (#l=137670, p=0x7fc53fbd1110, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120808999288, n=1).
|-- 11:27:25-DEBUG-peak_anno : Calling select_by_key (key=seqid, value=chr1,chr2,chr3,chr4...)
|-- 11:27:25-INFO-peak_anno : Found file pygtftk/data/mini_real/mini_real.gtf.gz
|-- 11:27:25-INFO-peak_anno : Instantiating a GTF.
|-- 11:27:25-DEBUG_MEM-peak_anno : GTF created (#l=137670, p=0x7fc54737f530, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120808971736, n=2).
|-- 11:27:25-DEBUG_MEM-peak_anno : GTF deleted (#l=137670, p=0x7fc53fbd1110, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120808999288, n=1).
|-- 11:27:25-DEBUG-peak_anno : Calling 'get_chroms'.
|-- 11:27:25-DEBUG-peak_anno : Calling 'get_feature_list'.
|-- 11:27:25-DEBUG-peak_anno : Calling extract_data (feature).
|-- 11:27:26-DEBUG-peak_anno : Calling select_by_key (key=feature, value=gene)
|-- 11:27:26-INFO-peak_anno : Found file pygtftk/data/mini_real/mini_real.gtf.gz
|-- 11:27:26-INFO-peak_anno : Instantiating a GTF.
|-- 11:27:26-DEBUG_MEM-peak_anno : GTF created (#l=1058, p=0x7fc52d9cac80, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120808693152, n=3).
|-- 11:27:26-DEBUG-peak_anno : Calling 'to_bed' method.
|-- 11:27:26-DEBUG-peak_anno : Calling extract_data (seqid,start,end,score,strand,transcript_id,gene_id,exon_id).
|-- 11:27:26-DEBUG_MEM-peak_anno : GTF deleted (#l=1058, p=0x7fc52d9cac80, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120808693152, n=3).
|-- 11:27:26-INFO-peak_anno : Beginning shuffling for a given set of features...
|-- 11:27:26-DEBUG-peak_anno : BATCHES : 10 batches of 20 shuffles
|-- 11:27:26-DEBUG-peak_anno : Total number of shuffles : 200
|-- 11:27:26-DEBUG-peak_anno : NB_THREADS = 8
|-- 11:27:26-DEBUG-peak_anno : BED files read as lists of intervals in 0.17975282669067383 s
|-- 11:27:26-INFO-peak_anno : --- Minibatch nb. : 1 / 10
|-- 11:27:26-DEBUG-peak_anno : Batch generated and shuffled in 0.023317813873291016 s
|-- 11:27:26-DEBUG-peak_anno : Batch converted to fake beds in : 0.1311180591583252 s
|-- 11:27:27-DEBUG-peak_anno : All intersections computed by custom code in : 0.6467368602752686 s
|-- 11:27:27-INFO-peak_anno : --- Minibatch nb. : 2 / 10
|-- 11:27:27-DEBUG-peak_anno : Batch generated and shuffled in 0.019360065460205078 s
|-- 11:27:27-DEBUG-peak_anno : Batch converted to fake beds in : 0.12729096412658691 s
|-- 11:27:28-DEBUG-peak_anno : All intersections computed by custom code in : 0.6355540752410889 s
|-- 11:27:28-INFO-peak_anno : --- Minibatch nb. : 3 / 10
|-- 11:27:28-DEBUG-peak_anno : Batch generated and shuffled in 0.018916845321655273 s
|-- 11:27:28-DEBUG-peak_anno : Batch converted to fake beds in : 0.13068914413452148 s
|-- 11:27:28-DEBUG-peak_anno : All intersections computed by custom code in : 0.6408717632293701 s
|-- 11:27:28-INFO-peak_anno : --- Minibatch nb. : 4 / 10
|-- 11:27:28-DEBUG-peak_anno : Batch generated and shuffled in 0.0192258358001709 s
|-- 11:27:29-DEBUG-peak_anno : Batch converted to fake beds in : 0.12983202934265137 s
|-- 11:27:29-DEBUG-peak_anno : All intersections computed by custom code in : 0.6501920223236084 s
|-- 11:27:29-INFO-peak_anno : --- Minibatch nb. : 5 / 10
|-- 11:27:29-DEBUG-peak_anno : Batch generated and shuffled in 0.0198819637298584 s
|-- 11:27:29-DEBUG-peak_anno : Batch converted to fake beds in : 0.13089895248413086 s
|-- 11:27:30-DEBUG-peak_anno : All intersections computed by custom code in : 0.7375490665435791 s
|-- 11:27:30-INFO-peak_anno : --- Minibatch nb. : 6 / 10
|-- 11:27:30-DEBUG-peak_anno : Batch generated and shuffled in 0.020968914031982422 s
|-- 11:27:30-DEBUG-peak_anno : Batch converted to fake beds in : 0.1310739517211914 s
|-- 11:27:31-DEBUG-peak_anno : All intersections computed by custom code in : 0.7431747913360596 s
|-- 11:27:31-INFO-peak_anno : --- Minibatch nb. : 7 / 10
|-- 11:27:31-DEBUG-peak_anno : Batch generated and shuffled in 0.01874995231628418 s
|-- 11:27:31-DEBUG-peak_anno : Batch converted to fake beds in : 0.1322779655456543 s
|-- 11:27:32-DEBUG-peak_anno : All intersections computed by custom code in : 0.6475603580474854 s
|-- 11:27:32-INFO-peak_anno : --- Minibatch nb. : 8 / 10
|-- 11:27:32-DEBUG-peak_anno : Batch generated and shuffled in 0.0187680721282959 s
|-- 11:27:32-DEBUG-peak_anno : Batch converted to fake beds in : 0.12628579139709473 s
|-- 11:27:33-DEBUG-peak_anno : All intersections computed by custom code in : 0.7448949813842773 s
|-- 11:27:33-INFO-peak_anno : --- Minibatch nb. : 9 / 10
|-- 11:27:33-DEBUG-peak_anno : Batch generated and shuffled in 0.019285917282104492 s
|-- 11:27:33-DEBUG-peak_anno : Batch converted to fake beds in : 0.13170504570007324 s
|-- 11:27:34-DEBUG-peak_anno : All intersections computed by custom code in : 0.7476060390472412 s
|-- 11:27:34-INFO-peak_anno : --- Minibatch nb. : 10 / 10
|-- 11:27:34-DEBUG-peak_anno : Batch generated and shuffled in 0.018893003463745117 s
|-- 11:27:34-DEBUG-peak_anno : Batch converted to fake beds in : 0.12927627563476562 s
|-- 11:27:35-DEBUG-peak_anno : All intersections computed by custom code in : 0.7500202655792236 s
|-- 11:27:35-DEBUG-peak_anno : All intersections have been generated.
|-- 11:27:35-DEBUG-peak_anno : Statistics on overlaps computed in : 0.12702703475952148 s
Traceback (most recent call last):
File "/Users/puthier/miniconda3/envs/pygtftk/bin/gtftk", line 4, in <module>
__import__('pkg_resources').run_script('pygtftk==0.9.9.dev0+cbb3', 'gtftk')
File "/Users/puthier/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 661, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users/puthier/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1441, in run_script
exec(code, namespace, namespace)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/EGG-INFO/scripts/gtftk", line 104, in <module>
args = main()
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/EGG-INFO/scripts/gtftk", line 89, in main
CmdManager.run(args)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/cmd_manager.py", line 950, in run
fun(**args)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/plugins/peak_anno.py", line 457, in peak_anno
hits[feat_type] = overlap_partial(bedA=peak_file, bedB=gtf_sub_bed)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/stats/intersect/overlap_stats_shuffling.py", line 229, in compute_overlap_stats
ps = nf.check_negbin_adjustment(summed_bp_overlaps, esperance_fitted_summed_bp_overlaps, variance_fitted_summed_bp_overlaps)#.pvalue
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/stats/intersect/negbin_fit.py", line 90, in check_negbin_adjustment
result = 1 - cramers_V(crosstab)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/stats/intersect/negbin_fit.py", line 84, in cramers_V
chi2 = scipy.stats.chi2_contingency(crosstab)[0]
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/scipy/stats/contingency.py", line 253, in chi2_contingency
"frequencies has a zero element at %s." % (zeropos,))
ValueError: The internally computed table of expected frequencies has a zero element at (13, 0).
|-- 11:27:44-DEBUG_MEM-peak_anno : GTF deleted (#l=0, p=0x7fc54737f530, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120808971736, n=2).
puthier (pygtftk)
peak_anno_shuffling ● ? ⍟5 …/pygtftk gtftk peak_anno -i pygtftk/data/mini_real/mini_real.gtf.gz -o toto -c pygtftk/data/mini_real/hg38.genome -p pygtftk/data/mini_real/ENCFF112BHN_H3K4me3_K562_sub.bed -V 3
$RPROMPT_PREFIX$(build_right_prompt)$reset_color$RPROMPT_SUFFIX
puthier (pygtftk)
peak_anno_shuffling ● ? ⍟5 …/pygtftk gunzip -c pygtftk/data/mini_real/mini_real.gtf.gz| less 1 ↵ ⚙ 10067 11:28:07
puthier (pygtftk)
peak_anno_shuffling ● ? ⍟5 …/pygtftk less pygtftk/data/mini_real/hg38.genome PIPE(-13)|0 ↵ ⚙ 10068 11:28:16
puthier (pygtftk)
peak_anno_shuffling ● ? ⍟5 …/pygtftk cp pygtftk/data/mini_real/hg38.genome hg38 ✔ ⚙ 10069 11:28:26
puthier (pygtftk)
peak_anno_shuffling ● ? ⍟5 …/pygtftk gtftk peak_anno -i pygtftk/data/mini_real/mini_real.gtf.gz -o toto -c hg38 -p pygtftk/data/mini_real/ENCFF112BHN_H3K4me3_K562_sub.bed -V 3
|-- 11:28:53-INFO-peak_anno : Checking chromosome info file.
|-- 11:28:53-INFO-peak_anno : Instantiating a GTF.
|-- 11:28:53-DEBUG-peak_anno : Calling extract_data_iter.
|-- 11:28:53-DEBUG-peak_anno : Ensembl format detected.
|-- 11:28:53-DEBUG_MEM-peak_anno : GTF created (#l=137670, p=0x7fb5380d4010, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120752028752, n=1).
|-- 11:28:53-DEBUG-peak_anno : Calling select_by_key (key=seqid, value=chr1,chr2,chr3,chr4...)
|-- 11:28:54-INFO-peak_anno : Found file pygtftk/data/mini_real/mini_real.gtf.gz
|-- 11:28:54-INFO-peak_anno : Instantiating a GTF.
|-- 11:28:54-DEBUG_MEM-peak_anno : GTF created (#l=137670, p=0x7fb5380e3840, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120752000528, n=2).
|-- 11:28:54-DEBUG_MEM-peak_anno : GTF deleted (#l=137670, p=0x7fb5380d4010, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120752028752, n=1).
|-- 11:28:54-DEBUG-peak_anno : Calling 'get_chroms'.
|-- 11:28:54-DEBUG-peak_anno : Calling 'get_feature_list'.
|-- 11:28:54-DEBUG-peak_anno : Calling extract_data (feature).
|-- 11:28:55-DEBUG-peak_anno : Calling select_by_key (key=feature, value=gene)
|-- 11:28:55-INFO-peak_anno : Found file pygtftk/data/mini_real/mini_real.gtf.gz
|-- 11:28:55-INFO-peak_anno : Instantiating a GTF.
|-- 11:28:55-DEBUG_MEM-peak_anno : GTF created (#l=1058, p=0x7fb5380d7eb0, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120751718296, n=3).
|-- 11:28:55-DEBUG-peak_anno : Calling 'to_bed' method.
|-- 11:28:55-DEBUG-peak_anno : Calling extract_data (seqid,start,end,score,strand,transcript_id,gene_id,exon_id).
|-- 11:28:55-DEBUG_MEM-peak_anno : GTF deleted (#l=0, p=0x7fb5380d7eb0, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120751718296, n=3).
|-- 11:28:55-INFO-peak_anno : Beginning shuffling for a given set of features...
|-- 11:28:55-DEBUG-peak_anno : BATCHES : 10 batches of 20 shuffles
|-- 11:28:55-DEBUG-peak_anno : Total number of shuffles : 200
|-- 11:28:55-DEBUG-peak_anno : NB_THREADS = 8
|-- 11:28:55-DEBUG-peak_anno : BED files read as lists of intervals in 0.15019702911376953 s
|-- 11:28:55-INFO-peak_anno : --- Minibatch nb. : 1 / 10
|-- 11:28:55-DEBUG-peak_anno : Batch generated and shuffled in 0.018970966339111328 s
|-- 11:28:55-DEBUG-peak_anno : Batch converted to fake beds in : 0.13421201705932617 s
|-- 11:28:56-DEBUG-peak_anno : All intersections computed by custom code in : 0.6478052139282227 s
|-- 11:28:56-INFO-peak_anno : --- Minibatch nb. : 2 / 10
|-- 11:28:56-DEBUG-peak_anno : Batch generated and shuffled in 0.01854705810546875 s
|-- 11:28:56-DEBUG-peak_anno : Batch converted to fake beds in : 0.1281449794769287 s
|-- 11:28:56-DEBUG-peak_anno : All intersections computed by custom code in : 0.6428921222686768 s
|-- 11:28:56-INFO-peak_anno : --- Minibatch nb. : 3 / 10
|-- 11:28:56-DEBUG-peak_anno : Batch generated and shuffled in 0.01955699920654297 s
|-- 11:28:57-DEBUG-peak_anno : Batch converted to fake beds in : 0.13239288330078125 s
|-- 11:28:57-DEBUG-peak_anno : All intersections computed by custom code in : 0.6378078460693359 s
|-- 11:28:57-INFO-peak_anno : --- Minibatch nb. : 4 / 10
|-- 11:28:57-DEBUG-peak_anno : Batch generated and shuffled in 0.01935100555419922 s
|-- 11:28:57-DEBUG-peak_anno : Batch converted to fake beds in : 0.1293799877166748 s
|-- 11:28:58-DEBUG-peak_anno : All intersections computed by custom code in : 0.6441848278045654 s
|-- 11:28:58-INFO-peak_anno : --- Minibatch nb. : 5 / 10
|-- 11:28:58-DEBUG-peak_anno : Batch generated and shuffled in 0.018750905990600586 s
|-- 11:28:58-DEBUG-peak_anno : Batch converted to fake beds in : 0.1291358470916748 s
|-- 11:28:59-DEBUG-peak_anno : All intersections computed by custom code in : 0.6483819484710693 s
|-- 11:28:59-INFO-peak_anno : --- Minibatch nb. : 6 / 10
|-- 11:28:59-DEBUG-peak_anno : Batch generated and shuffled in 0.019301176071166992 s
|-- 11:28:59-DEBUG-peak_anno : Batch converted to fake beds in : 0.13379311561584473 s
|-- 11:29:00-DEBUG-peak_anno : All intersections computed by custom code in : 0.6421530246734619 s
|-- 11:29:00-INFO-peak_anno : --- Minibatch nb. : 7 / 10
|-- 11:29:00-DEBUG-peak_anno : Batch generated and shuffled in 0.018435001373291016 s
|-- 11:29:00-DEBUG-peak_anno : Batch converted to fake beds in : 0.1339731216430664 s
|-- 11:29:00-DEBUG-peak_anno : All intersections computed by custom code in : 0.7524068355560303 s
|-- 11:29:00-INFO-peak_anno : --- Minibatch nb. : 8 / 10
|-- 11:29:01-DEBUG-peak_anno : Batch generated and shuffled in 0.018976926803588867 s
|-- 11:29:01-DEBUG-peak_anno : Batch converted to fake beds in : 0.13104987144470215 s
|-- 11:29:01-DEBUG-peak_anno : All intersections computed by custom code in : 0.7570860385894775 s
|-- 11:29:01-INFO-peak_anno : --- Minibatch nb. : 9 / 10
|-- 11:29:01-DEBUG-peak_anno : Batch generated and shuffled in 0.01871800422668457 s
|-- 11:29:02-DEBUG-peak_anno : Batch converted to fake beds in : 0.1300792694091797 s
|-- 11:29:02-DEBUG-peak_anno : All intersections computed by custom code in : 0.7510981559753418 s
|-- 11:29:02-INFO-peak_anno : --- Minibatch nb. : 10 / 10
|-- 11:29:02-DEBUG-peak_anno : Batch generated and shuffled in 0.018780946731567383 s
|-- 11:29:02-DEBUG-peak_anno : Batch converted to fake beds in : 0.1317291259765625 s
|-- 11:29:03-DEBUG-peak_anno : All intersections computed by custom code in : 0.7473132610321045 s
|-- 11:29:03-DEBUG-peak_anno : All intersections have been generated.
|-- 11:29:03-DEBUG-peak_anno : Statistics on overlaps computed in : 0.1287541389465332 s
Traceback (most recent call last):
File "/Users/puthier/miniconda3/envs/pygtftk/bin/gtftk", line 4, in <module>
__import__('pkg_resources').run_script('pygtftk==0.9.9.dev0+cbb3', 'gtftk')
File "/Users/puthier/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 661, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users/puthier/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1441, in run_script
exec(code, namespace, namespace)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/EGG-INFO/scripts/gtftk", line 104, in <module>
args = main()
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/EGG-INFO/scripts/gtftk", line 89, in main
CmdManager.run(args)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/cmd_manager.py", line 950, in run
fun(**args)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/plugins/peak_anno.py", line 457, in peak_anno
hits[feat_type] = overlap_partial(bedA=peak_file, bedB=gtf_sub_bed)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/stats/intersect/overlap_stats_shuffling.py", line 229, in compute_overlap_stats
ps = nf.check_negbin_adjustment(summed_bp_overlaps, esperance_fitted_summed_bp_overlaps, variance_fitted_summed_bp_overlaps)#.pvalue
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/stats/intersect/negbin_fit.py", line 90, in check_negbin_adjustment
result = 1 - cramers_V(crosstab)
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/pygtftk-0.9.9.dev0+cbb3-py3.6-macosx-10.7-x86_64.egg/pygtftk/stats/intersect/negbin_fit.py", line 84, in cramers_V
chi2 = scipy.stats.chi2_contingency(crosstab)[0]
File "/Users/puthier/miniconda3/envs/pygtftk/lib/python3.6/site-packages/scipy/stats/contingency.py", line 253, in chi2_contingency
"frequencies has a zero element at %s." % (zeropos,))
ValueError: The internally computed table of expected frequencies has a zero element at (13, 0).
|-- 11:29:13-DEBUG_MEM-peak_anno : GTF deleted (#l=137670, p=0x7fb5380e3840, f=pygtftk/data/mini_real/mini_real.gtf.gz, i=120752000528, n=2).