vaquerizaslab / fanc Goto Github PK

View Code? Open in Web Editor NEW

103.0 103.0 14.0 428.94 MB

FAN-C: Framework for the ANalysis of C-like data

License: GNU General Public License v3.0

Python 99.47% MATLAB 0.12% Clojure 0.01% C 0.04% Makefile 0.33% CSS 0.02% Cython 0.02%

fanc's People

Contributors

Stargazers

Watchers

Forkers

fgroll darlingfuer liz-is markyanghc kunfang93 lie-ne yuzhenpeng yang-mj zhaoxu-gao hzjsxu wikk-chy arpithasrinivas5 naveed-codes wangdong-ls

fanc's Issues

--chromosomes in fanc hic

Dear @kaukrise

I found that when I set the argument --chromosomes 1,2,3 when running fanc hic, it always gave me an error as following:

fanc hic -b 50k -r 0.25 -n -m KR -s input.statistics -d 0 --reset-filters --chromosomes 1,2,3 input_fragment_level.hic input_50K_KR.hic
2020-12-11 20:09:42,281 INFO FAN-C version: 0.9.9
usage: fanc hic [-h] [-b BIN_SIZE] [-l FILTER_LOW_COVERAGE]
[-r FILTER_LOW_COVERAGE_RELATIVE] [-a] [-d FILTER_DIAGONAL]
[--marginals-plot MARGINALS_PLOT] [--reset-filters]
[--downsample DOWNSAMPLE] [--subset SUBSET] [-i] [-k] [-n]
[-m NORM_METHOD] [-w] [-c] [--only-inter] [-s STATS]
[--statistics-plot STATS_PLOT]
[--chromosomes CHROMOSOMES [CHROMOSOMES ...]] [-f]
[-t THREADS] [--deepcopy] [-tmp]
input [input ...]
fanc hic: error: the following arguments are required: input

Sincerely,
Zheng zhuqing

ERROR plotting a line plot

Hello dear FAN-C team,

I tried using fancplot CLI to plot a panel and a line plot using a .hic file and a bigwig file.
Plotting multipanel plots with refseq tracks went fine.

Trying to plot a single line plot for the same coordinates, using a bigwig file from the ENCODE project gave the following output:

fancplot 17:40465000-40665000 -o test.png -p line ENCFF965VYA_GM12878_minus_strand_unique_reads.bw
2021-03-18 14:04:16,665 INFO Using non-interactive backend
[E::hts_hopen] Failed to open file ENCFF965VYA_GM12878_minus_strand_unique_reads.bw
[E::hts_open_format] Failed to open file "ENCFF965VYA_GM12878_minus_strand_unique_reads.bw" : Exec format error
2021-03-18 14:04:17,423 INFO Found 1 regions
/specific/elkon/gonyshanel/python37/bin/fancplot:164: UserWarning: There was an error with plot 0, region 17:40465000-40665000 (Traceback (most recent call last):
File "/specific/elkon/gonyshanel/python37/bin/fancplot", line 144, in init
fig, axes = gf.plot(plot_region)
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/plotter.py", line 211, in plot
p.plot(r)
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/base_plotter.py", line 292, in plot
plot_output = self._plot(region)
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/plotter.py", line 984, in _plot
for i, x, y in self._line_values(region):
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/plotter.py", line 978, in _line_values
x, y = self.values_from_region_iter(region_iter, self.attribute)
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/base_plotter.py", line 523, in values_from_region_iter
return self._STYLES[self.style](self, regions, attribute)
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/base_plotter.py", line 510, in _get_values_per_mid
for region in regions:
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/fanc/plotting/plotter.py", line 926, in _region_iter
for region in data.regions(region, **kwargs):
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/genomic_regions/regions.py", line 1675, in _region_subset
for start, end, score in self.region_intervals(r):
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/genomic_regions/regions.py", line 956, in region_intervals
raw_intervals = self._region_intervals(region, score_field=score_field, *args, **kwargs)
File "/specific/elkon/gonyshanel/python37/lib/python3.7/site-packages/genomic_regions/regions.py", line 1687, in _region_intervals
intervals = self.bw.intervals(region.chromosome, region.start, region.end)
RuntimeError: Invalid interval bounds!
)
tb))
2021-03-18 14:04:17,451 WARNING Invalid interval bounds!

n

Give some meaningful error message in fancplot split when using different resolutions

Hi I tried fancplot split and chose two different binning resolutions.
Right now it looks like this:

2020-02-11 13:42:48,741 INFO Found 1 regions
qt5ct: using qt5ct plugin
/usr/local/bin/fancplot:157: UserWarning: There was an error with plot 0, region chr1:14000000-20000000 (Traceback (most recent call last):
  File "/usr/local/bin/fancplot", line 139, in __init__
    fig, axes = gf.plot(plot_region)
  File "/usr/local/lib/python3.7/dist-packages/fanc/plotting/plotter.py", line 211, in plot
    p.plot(r)
  File "/usr/local/lib/python3.7/dist-packages/fanc/plotting/base_plotter.py", line 785, in plot
    plot_output = self._plot((x_region, y_region))
  File "/usr/local/lib/python3.7/dist-packages/fanc/plotting/hic_plotter.py", line 296, in _plot
    self._refresh(region)
  File "/usr/local/lib/python3.7/dist-packages/fanc/plotting/hic_plotter.py", line 339, in _refresh
    self.current_matrix = self.hic_buffer.get_matrix(*region)
  File "/usr/local/lib/python3.7/dist-packages/fanc/plotting/hic_plotter.py", line 143, in get_matrix
    self._BUFFERING_STRATEGIES[self.buffering_strategy](self, *regions)
  File "/usr/local/lib/python3.7/dist-packages/fanc/plotting/hic_plotter.py", line 186, in _buffer_relative
    norm=self.norm, oe=self.oe, log=self.log)
  File "/usr/local/lib/python3.7/dist-packages/fanc/architecture/comparisons.py", line 494, in matrix
    m_bottom[top_indices] = m_top[top_indices]
  File "/usr/local/lib/python3.7/dist-packages/fanc/matrix.py", line 2605, in __setitem__
    np.ma.MaskedArray.__setitem__(self, key, item)
  File "/usr/lib/python3/dist-packages/numpy/ma/core.py", line 3327, in __setitem__
    _data[indx] = dval
IndexError: index 73 is out of bounds for axis 0 with size 73
)
  tb))
2020-02-11 13:43:01,556 WARNING index 73 is out of bounds for axis 0 with size 73

crash on pyBigWig.open

I was running in auto mode.
There are no bigWig files in the output directory, only sam and sorted sam files.
The temporary directory does not exist anymore.

2020-02-09 05:26:59,352 INFO Cannot find sambamba on this machine, falling back to samtools sort. This is not a problem, but if you want to speed up your SAM/BAM file sorting, install [...]
2020-02-09 06:59:11,503 INFO Getting genome regions (fragments or bins)
2020-02-09 06:59:11,503 INFO Getting regions
[bwHdrRead] There was an error while reading in the header!
[pyBwOpen] bw is NULL!
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/genomic_regions/regions.py", line 183, in load
    f = pyBigWig.open(file_name, 'r')
RuntimeError: Received an error during file opening!

pairs object from 4D Nucleome pairs file

Hi,
I have instanlled fanc 0.81.31. I am very enthusiastic to see coverage plot (--marginals-plot) of the data of my interest (data from 4DNucleome portal). However, I am not able to create pairs object from 4D Nucleome pairs file.

fanc pairs 4DNFIQL5L5I3.pairs ./output/proper_pairs.pairs -g ./restriction_fragments_DpnII_hg38.bed

2020-06-26 12:58:47,981 INFO Getting genome regions (fragments or bins)
2020-06-26 12:58:47,981 INFO Getting regions
2020-06-26 12:58:47,995 INFO Two arguments detected, assuming HiC-Pro or 4D Nucleome input.
Traceback (most recent call last):
  File "/home/tszczep/.local/lib/python3.6/site-packages/fanc/commands/fanc_commands.py", line 983, in pairs
    sb = FourDNucleomePairGenerator(input_file)
  File "/home/tszczep/.local/lib/python3.6/site-packages/fanc/pairs.py", line 653, in __init__
    for i, name in columns_field.split():
ValueError: too many values to unpack (expected 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tszczep/.local/bin/fanc", line 124, in <module>
    Fanc()
  File "/home/tszczep/.local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/home/tszczep/.local/lib/python3.6/site-packages/fanc/commands/fanc_commands.py", line 986, in pairs
    sb = HicProPairGenerator(input_file)
  File "/home/tszczep/.local/lib/python3.6/site-packages/fanc/pairs.py", line 622, in __init__
    chr2_field=4, pos2_field=5, strand2_field=6)
  File "/home/tszczep/.local/lib/python3.6/site-packages/fanc/pairs.py", line 572, in __init__
    int(fields[pos2_field])
ValueError: invalid literal for int() with base 10: '+'

I made a testing file from this 4DN file. Here it is:

## pairs format v1.0.0
#sorted: chr1-chr2-pos1-pos2
#shape: upper triangle
#genome_assembly: unknown
#chromsize: chr1 248956422
#chromsize: chr2 242193529
#chromsize: chr3 198295559
#columns: readID chrom1 pos1 chrom2 pos2 strand1 strand2 pair_type frag1 frag2
K00284:70:HKG22BBXX:5:1127:25022:31453	chr1	10298	chr1	25266563	+	-	LL	0	69472
K00284:70:HKG22BBXX:2:1205:1205:11073	chr1	30899	chr1	228183613	-	+	LL	43	531295
K00284:70:HKG22BBXX:3:2121:8846:29800	chr1	30900	chr1	2078072	-	+	LL	43	4831
K00284:70:HKG22BBXX:1:2205:2351:28885	chr1	30903	chr1	213986772	-	+	LL	43	496325

How can I solve this? Can I modify 4DN pairs file somehow to be handled by fanc?

ValueError: File type not recognised (.loops).

First I run:

fanc loops data.cool \
           fanc_loops.loops \
           -t 32

Then I run:

fanc loops fanc_loops.loops \
           filtered_loops.loops \
           --rh-filter -d 5 -o 5

But I get the following error: ValueError: File type not recognised (fanc_loops.loops).

Does anyone know what the source of this issue is?

UnicodeDecodeError in fanc loops

Hi,
I'm trying to annotate loops from a 10kb-resolution hic file and I get the following:

2020-03-05 10:32:30,848 INFO Calculating intra-chromosomal expected values...
2020-03-05 10:32:30,865 INFO Done.
2020-03-05 10:32:31,091 INFO Initial parameter values: p=2, w=5
2020-03-05 10:32:31,091 INFO Obtaining bias vector...
Expected N/A% (0 of 203545187) | | Elapsed Time: 0:00:00 ETA: --:--:--
Expected 0% (1 of 203545187) | | Elapsed Time: 0:00:00 ETA: 00:00:00
Expected 1% (2035452 of 203545187) | | Elapsed Time: 0:03:18 ETA: 5:28:03
...
Expected 100% (203545187 of 203545187) |#| Elapsed Time: 5:19:24 Time: 5:19:24
2020-03-05 15:53:47,327 INFO Done.
2020-03-05 15:53:47,328 INFO Processing chr1-chr1
2020-03-05 15:57:01,662 INFO using 10 processes
Traceback (most recent call last):
File "/home/ska/jsanper/miniconda3/lib/python3.7/site-packages/gridmap/job.py", line 249, in execute
self.ret = self.function(*self.args, **self.kwlist)
File "/home/ska/jsanper/miniconda3/lib/python3.7/site-packages/fanc/peaks.py", line 1357, in process_matrix_segment_intra
max_w = msgpack.loads(data)
File "/home/ska/jsanper/miniconda3/lib/python3.7/site-packages/msgpack_numpy.py", line 198, in unpackb
return _unpackb(packed, **kwargs)
File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 14: invalid continuation byte

(this error is repeated for many positions and finally...)

TypeError: a bytes-like object is required, not 'UnicodeDecodeError'
Closing remaining open files:loops/HiC_danRer10_48h_ctcf_r10kb.loops...done

I've tried with other hic files getting the same. And these same hic files worked perfectly for other tools like insulation, boundaries, pca, compare, fancplot, etc.
My python environment is 3.7.6.
Could you give please me a clue of what's happening?
Thank you in advance.
Best,

Jose Maria

Questions about TAD calling

Hi,

First, I would like to thank you for developing FANC tools and make it available.
It is helpful and it made simpler to analyse Hi-C data.
However, I'm once again asking for your help.

I intend to perform TAD calling and I would like to know what is the difference between fanc boundaries and tadtool.
It was not clear for me in this statement if fanc boundaries is a 1-TAD caller algorithm itself or 2-calls tadtool

We are therefore currently not bundling a TAD calling tool with FAN-C, and refer the user to one of the many available tools for TAD calling that offer a wide range of features. https://vaquerizaslab.github.io/fanc/fanc-executable/fanc-analyse-hic/domains.html#a-note-on-tad-calling

In case it is option 1- a TAD caller algorithm itself, which one has performed better on your hands?

Thank you for your time

compartment call issue

Hi,
I'm working on Drosophila Hi-C data.
I have called the compartments by using command:

fanc compartments -g genome.fa -e chrX.pdf chrX.hic chrX.ab -v chrX.ev.txt -s 0 -tmp

eigen vector values (chrX.ev.txt file) shows chromatin distribution as "A" and "B" compartment along chrX, but the plot looks strange (don't show negative values).

This is how compartments heatmap look like when I don't use "-s 0" option.

could you please explain how compartment calling works in FanC?

Thanks

Should duplication be removed in HiC-Pro valid pairs

Dear all,

I was wondering that whether the duplication in the valid pairs used in fanc pairs should be removed. That means which file should be used in fanc pairs, validPairs interactions or allValidPairs in the Hic-Pro results? Thank you.

Best wishes,
Zheng zhuqing

A/B compartments calling: Chromosome {} not found in genome. Are you using the correct genome file?

Hi,
I followed the instructions to call A/B compartments. First, created the AB compartment matrix file from a .hic file, then used it to extract .bed coordinates of A/B compartments.

As soon as I supply the genome to refine A/B calls, the function breaks. I use the same genome that was used to create the .hic file. The error raised by compartments.py is:

ValueError: Chromosome {} not found in genome. Are you using the correct genome file?

My genome has chromosome names as chr1 etc. It seems FAN-C uses numeric notation for chromosome names; manually reformatting the genome FASTA file to have numerical chromosome names solved the problem. It is OK if one is aware of chromosome name requirements. I can't recall any place in the documentation. Suggesting adding this.

issue to separate AB domains

Hello, I was trying to seperate the AB domains using command:

fanc compartments -d domains.gc.bed -g genome.fa sample.ab
(using fanc version: 0.9.3)
But the distribution of AB domains looks not normal (image attached). could you please help to resolve this issue.

Thanks

Request: use genomic coordinate info to match eigenvector values to Hi-C bins

Background: I almost always find that the default eigenvectors don't really represent compartments for at least one chromosome, in at least one sample, and so I end up doing some post-processing and hand-picking the proper eigenvector that I want to use. Usually this then ends up saved in a bigwig file.

Because these bigwig files don't necessarily have evenly sized bins (e.g. multiple consecutive Hi-C bins with the same eigenvector value will be merged), and don't necessarily have coverage over the full chromosome (missing values at the start of a chromosome are not filled in), the bigwig file then doesn't have the same number of bins as the Hi-C data it's originally derived from: ValueError: Eigenvector must have same number of entries (27936) as regions in genome (30321)!

I can work around this by saving the output in another format and forcing all bins to be present, of course, but this led me to look at the code and I think that maybe the way that the provided eigenvector and the Hi-C bins are matched up is a bit fragile, since it seems to assume that they are always parallel (of course this will be the case if someone is using the FAN-C calculated eigenvector .bed files without post-processing). If any post-processing has happened, even if the two vectors are the same length, you can't guarantee that the regions are in the same order, since chromosomes could have been (accidentally) re-sorted according to different conventions, for example.

I'm sure there's a more efficient / elegant way to write this, but would replacing this line with something like the following allow matching eigenvector values to Hi-C bins based on the genomic coordinates?

ev = [None] * len(h.regions)

for i, r in enumerate(h.regions):
	bw_score = [bwr.score for bwr in ev_regions.regions(r)]
	if len(bw_score)==1:
		ev[i] = bw_score[0]
	elif len(bw_score)==0:
		ev[i] = np.nan
	else:
		raise ValueError("Hi-C bins must only overlap one eigenvector bin!")

TypeError raised when calling fanc.architecture.compartments.ABCompartmentMatrix.from_hic with oe_per_chromosome=False

Python version: 3.7.10
fanc version: 0.9.0

Downloaded the .hic file from GEO

Reproducing code:

>>> hic_fp=r"GSE82144_Kieffer-Kwon-2017-resting_B_cells_WT_30.hic"
>>> hic = fanc.load(hic_fp+"@50000")
>>> ab = fanc.architecture.compartments.ABCompartmentMatrix.from_hic(hic, oe_per_chromosome=False)

/home/scoulom8/miniconda3/envs/my_fanc_env/lib/python3.7/site-packages/fanc/compatibility/juicer.py:604: RuntimeWarning: divide by zero encountered in true_divide
  return np.array(vectors[resolution]) / sf

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scoulom8/miniconda3/envs/my_fanc_env/lib/python3.7/site-packages/fanc/architecture/compartments.py", line 105, in from_hic
    m = hic.matrix((chromosome, chromosome), oe=True, oe_per_chromosome=oe_per_chromosome)
  File "/home/scoulom8/miniconda3/envs/my_fanc_env/lib/python3.7/site-packages/fanc/matrix.py", line 1024, in matrix
    for source, sink, weight in matrix_entries:
  File "/home/scoulom8/miniconda3/envs/my_fanc_env/lib/python3.7/site-packages/fanc/matrix.py", line 984, in <genexpr>
    entry_iter = ((i, j, weight)
  File "/home/scoulom8/miniconda3/envs/my_fanc_env/lib/python3.7/site-packages/fanc/matrix.py", line 970, in offset_iter
    for edge in edge_iter:
  File "/home/scoulom8/miniconda3/envs/my_fanc_env/lib/python3.7/site-packages/fanc/matrix.py", line 760, in __call__
    edge.expected = ex[abs(sink - source)]
TypeError: 'NoneType' object is not subscriptable

Any methods to accerlarate the running of fanc?

Hi,
I start use fanc recently and found it always take a much longer time than I expected. For example, when I try run fanc compare at whole genome level at 10kb resolution, the running time was estimated as 8 hours; and when I try use from-cooler to convert cooler to fan-c format, the software run for several hours and get almost nothing output...

Still I didn't find any multi-thread option for these process.

Is there any problem with my command line or do you have any suggetion or comments on this?
/home/ch220811/.local/bin/fanc compare -l -I ${MATRIX1} ${MATRIX2} ${OUT}
/home/ch220811/.local/bin/fanc from-cooler ${PREFIX}.cool ${PREFIX}.fanc.hic

Thank you!

Error in fancplot

Hi,
When I used a juicer .hic file as input in fancplot, I got the following errors:

[kurotaki ~/HiC/Juicer_data/GB6367]$fancplot chr8:119,729,676-121,763,375 -p square KO_MDP_rep2.hic@500000
2020-02-26 20:25:00,284 INFO Found 1 regions
failed to get the current screen resources
/home/kurotaki/miniconda3/bin/fancplot:157: UserWarning: There was an error with plot 0, region chr8:119729676-121763375 (Traceback (most recent call last):
  File "/home/kurotaki/miniconda3/bin/fancplot", line 139, in __init__
    fig, axes = gf.plot(plot_region)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/plotting/plotter.py", line 211, in plot
    p.plot(r)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/plotting/base_plotter.py", line 785, in plot
    plot_output = self._plot((x_region, y_region))
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/plotting/hic_plotter.py", line 296, in _plot
    self._refresh(region)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/plotting/hic_plotter.py", line 339, in _refresh
    self.current_matrix = self.hic_buffer.get_matrix(*region)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/plotting/hic_plotter.py", line 143, in get_matrix
    self._BUFFERING_STRATEGIES[self.buffering_strategy](self, *regions)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/plotting/hic_plotter.py", line 186, in _buffer_relative
    norm=self.norm, oe=self.oe, log=self.log)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/matrix.py", line 987, in matrix
    **kwargs)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/matrix.py", line 908, in regions_and_matrix_entries
    row_regions, col_regions, edges_iter = self.regions_and_edges(key, *args, **kwargs)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/matrix.py", line 813, in regions_and_edges
    row_regions = list(row_regions)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/compatibility/juicer.py", line 700, in _region_subset
    subset_ix, subset_start = self._region_start(region)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/compatibility/juicer.py", line 665, in _region_start
    offset_ix = self._chromosome_ix_offset(region.chromosome)
  File "/home/kurotaki/miniconda3/lib/python3.6/site-packages/fanc/compatibility/juicer.py", line 648, in _chromosome_ix_offset
    raise ValueError("Chromosome {} not in matrix.".format(target_chromosome))
ValueError: Chromosome chr8 not in matrix.
)
  tb))
2020-02-26 20:25:02,602 WARNING Chromosome chr8 not in matrix.

How can I overcome this?
Best,
Daisuke

RuntimeError: Task had non-zero exit status. Cancelling execution of all tasks.

Hi,

Many thanks for making this toolset available.
I'm trying to run fanc compartments on my dataset. If I got it right, I should first convert my Hi-C analysis output into fanc input using fanc auto.
Once I do this first step, using .validpairs as fanc auto, I get this error:

$ fanc auto '/media/test/Data/Data/HiC/HiC_fused_150_PE/HiCPro_HiC_fused_150_PE/hic_results/data/K5testEtOH/K5testEtOH.allValidPairs' -g '/media/test/Data/Genomes_ref/bowtie2/Hsapiens/digested_ref/DpnII_resfrag_hg19.bed' '/media/test/Data/Data/HiC/HiC_fused_150_PE/FANC' 
2020-02-17 12:05:43,260 INFO Output folder: /media/test/Data/Data/HiC/HiC_fused_150_PE/FANC/
2020-02-17 12:05:43,260 INFO Input files: /media/test/Data/Data/HiC/HiC_fused_150_PE/HiCPro_HiC_fused_150_PE/hic_results/data/K5testEtOH/K5testEtOH.allValidPairs
2020-02-17 12:05:43,260 INFO Input file types: pairs_txt
2020-02-17 12:05:43,260 INFO Final basename: K5testEtOH (you can change this with the -n option!)
2020-02-17 12:05:43,260 INFO Creating output folders...
2020-02-17 12:05:44,882 INFO Getting genome regions (fragments or bins)
2020-02-17 12:05:44,882 INFO Getting regions
2020-02-17 12:05:44,883 INFO Two arguments detected, assuming HiC-Pro or 4D Nucleome input.
Traceback (most recent call last):
  File "tables/tableextension.pyx", line 1590, in tables.tableextension.Row.__setitem__
TypeError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 124, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/fanc_commands.py", line 966, in pairs
    pairs.add_regions(regions, preserve_attributes=False)
  File "/usr/local/lib/python3.6/dist-packages/fanc/regions.py", line 664, in add_regions
    self.add_region(region, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/genomic_regions/regions.py", line 1219, in add_region
    return self._add_region(new_region, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/fanc/regions.py", line 602, in _add_region
    row['end'] = region.end
  File "tables/tableextension.pyx", line 1595, in tables.tableextension.Row.__setitem__
TypeError: invalid type (<class 'str'>) for column ``end``
Closing remaining open files:/media/test/Data/Data/HiC/HiC_fused_150_PE/FANC/pairs/K5testEtOH_0.pairs...done
Exception ignored in: <object repr() failed>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tables/node.py", line 319, in __del__
  File "/usr/local/lib/python3.6/dist-packages/tables/table.py", line 2961, in _f_close
  File "/usr/local/lib/python3.6/dist-packages/tables/table.py", line 2895, in flush
  File "tables/tableextension.pyx", line 1337, in tables.tableextension.Row._flush_buffered_rows
  File "tables/tableextension.pyx", line 752, in tables.tableextension.Row.table.__get__
  File "/usr/local/lib/python3.6/dist-packages/tables/file.py", line 2150, in _check_open
tables.exceptions.ClosedFileError: the file object is closed
Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 124, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/fanc_commands.py", line 114, in auto
    return fanc.commands.auto.auto(argv, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/auto.py", line 1181, in auto
    runner.run()
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/auto.py", line 141, in run
    "Cancelling execution of all tasks.".format(task.id))
RuntimeError: Task 29185059-d866-430e-b45d-3af483602949 had non-zero exit status. Cancelling execution of all tasks.

I'm using Linux, any idea what is going on?

$ python --version
Python 3.6.8
$ fanc --version
0.8.18

thank you for your time,

email notification

I have trouble with email notification (fanc 0.8.32). I do not get it after a process is finished. This is what I wrote in fanc.conf file.

#
# EMAIL
#
email_from_address: [email protected]
email_smtp_server: smtp.gmail.com
email_smtp_port: 587
email_smtp_username: XXX
email_smtp_password: YYY

What is the best way to test email notifications?

No handling of gzipped genome fasta file

This step happens after sam sorting

  File "/usr/local/lib/python3.7/dist-packages/fanc/regions.py", line 106, in genome_regions
    regions = gr_load(re_or_file)
  File "/usr/local/lib/python3.7/dist-packages/genomic_regions/regions.py", line 190, in load
    raise ValueError("File type not recognised ({}).".format(file_name))
ValueError: File type not recognised (genome.fa.gz)

bin_size for LinePlot

Thanks for providing such a flexible plotting interface with fancplot! When trying to visualize bigWig tracks, I noticed that the bin_size parameter seems to have no effect for LinePlot despite it being a parameter. After looking into the source code it seems that it's essentially ignored. Although it appears that the legacy BigWigPlot class did have routines for changing bin sizes, I was unable to find much related documentation.

Could you please share if there are any plans of re-implementing the capacity to set bin sizes at run-time or if not, how to properly use the deprecated BigWigPlot?

multiple restriction enzymes

Hi,
Thank you for the amazing tool.
I was wondering if the tool is compatible with protocol that uses more than one restriction enzyme.
We use ARIMA kit and the kit uses HinfI and DpnII.
Thank you,
Jun

how to handle multiple replicates in fanc auto option?

I have 3 different replicates of the same sample and want to use them together. if possible in fanc auto option?

I tried to write command as:
fanc auto R1.read1.fastq.gz,R2.read1.fastq.gz,R3.read1.fastq.gz R1.read2.fastq.gz,R2.read2.fastq.gz,R3.read2.fastq.gz

but it didn't work this way.

Thanks!

fanc pca error

Hi many thanks for developing this tool.
It is quite practical and good. However, I'm receiving an error in pca subcommand that I can't fix it.
Can you help me please?

here is the command:
$fanc pca -Z -p /media/pca_hic_fanc_SMC1_reps.pca.png SMC1_n1.allValidPairs.hic@1000000 SMC_n2.allValidPairs.hic@1000000 pca_hic_fanc_RAD21_reps.pca

2020-06-27 23:26:41,147 INFO Calculating scaling factor...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
X
Y
M
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
X
Y
M
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
X
Y
M
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
X
Y
M
Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 124, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/fanc_commands.py", line 2376, in pca
    scale=scale)
  File "/usr/local/lib/python3.6/dist-packages/fanc/architecture/comparisons.py", line 599, in hic_pca
    filters=filters, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/fanc/architecture/comparisons.py", line 43, in _edge_collection
    s = hic.scaling_factor(reference_hic)
  File "/usr/local/lib/python3.6/dist-packages/fanc/matrix.py", line 1317, in scaling_factor
    for v1 in self.edge_data(weight_column, lazy=True):
  File "/usr/local/lib/python3.6/dist-packages/fanc/matrix.py", line 848, in edge_data
    for edge in self.edges(*args, **kwargs):
  File "/usr/local/lib/python3.6/dist-packages/fanc/matrix.py", line 755, in __call__
    *args, **kwargs):
  File "/usr/local/lib/python3.6/dist-packages/fanc/compatibility/juicer.py", line 973, in _edges_subset
    for x, y, weight in self._read_matrix(row_span, col_span):
  File "/usr/local/lib/python3.6/dist-packages/fanc/compatibility/juicer.py", line 872, in _read_matrix
    matrix_file_position = self._matrix_positions()[(str(chromosome1_ix), str(chromosome2_ix))]
KeyError: ('9', '25')

$ fanc --version
0.8.32

Thank you for your time

IndexError on 4DN files when calling loops

Hi, I am having an issue with the fanc loops command.

I am using a multi-resolution file that I downloaded from the 4DN website, I was wandering what does it mean to define different resolutions and how it might lead to the issue in that command because as far as I understand there is only one way to define the resolution no?

The command looks more or less like this.

fanc loops 4DNFI2TK7L2F.hic@5000   microcH1_5000_loopsAnnotate.loops    -t 5

and the error looks like this

Traceback (most recent call last):
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/gridmap/job.py", line 242, in execute
    self.ret = self.function(*self.args, **self.kwlist)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/fanc/peaks.py", line 1365, in process_matrix_segment_intra
    m_expected = expected_f(m_distance)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/numpy/lib/function_base.py", line 2108, in __call__
    return self._vectorize_call(func=func, args=vargs)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/numpy/lib/function_base.py", line 2192, in _vectorize_call
    outputs = ufunc(*inputs)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/fanc/peaks.py", line 1364, in <lambda>
    expected_f = np.vectorize(lambda x: e[x])
IndexError: index 49791 is out of bounds for axis 0 with size 49791
Traceback (most recent call last):
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/bin/fanc", line 127, in <module>
    Fanc()
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/fanc/commands/fanc_commands.py", line 2881, in loops
    peaks = pk.call_peaks(matrix, chromosome_pairs=chromosome_pairs, file_name=o)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/fanc/peaks.py", line 1270, in call_peaks
    self._find_peaks_intra_matrix(m, intra_expected[chromosome1], c[start1:end1],
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/fanc/peaks.py", line 1167, in _find_peaks_intra_matrix
    self._process_jobs(jobs, peak_info, observed_chunk_distribution)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/fanc/peaks.py", line 1056, in _process_jobs
    results = msgpack.loads(compressed_results, strict_map_key=False)
  File "/home/dmas/nxf_conda_env/HiCannotation-e79c6b7076dd13ce2817d3f6422eaed6/lib/python3.8/site-packages/msgpack_numpy.py", line 273, in unpackb
    return _unpackb(packed, **kwargs)
  File "msgpack/_unpacker.pyx", line 178, in msgpack._cmsgpack.unpackb
  File "msgpack/_unpacker.pyx", line 126, in msgpack._cmsgpack.get_data_from_buffer
TypeError: a bytes-like object is required, not 'IndexError'

Originally posted by @davidmasp in #7 (comment)

Can fanc help for generating a chromosome-level genome assembly?

Hi there,

I am currently working on the assembly of a pacbio-sequenced genome data, and I the next step of my work is using my HiC data to generate a chromosome-level genome.

Is there any possibility that I can use fanc to generate a chromosome-level genome assembly based on my HIC data?

My best,
Tianyu

fanc expected crashes in pybedtools (unexpected keyword 'tmpdir')

Hi, I am getting this crash

$ fanc expected -p expected.500kb.pdf -c chr1 output/hic/binned/mySample_500kb.hic expected.mySample.500kb.tab
Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 124, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.7/dist-packages/fanc/commands/fanc_commands.py", line 3880, in expected
    with fanc.load(input_file, mode='a', tmpdir=tmp) as matrix:
  File "/usr/local/lib/python3.7/dist-packages/fanc/tools/load.py", line 87, in load
    return gr_load(file_name, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/genomic_regions/regions.py", line 173, in load
    f = Bed(file_name, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/genomic_regions/regions.py", line 1430, in __init__
    pybedtools.BedTool.__init__(self, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'tmpdir'

genomic regions and pybedtools are current versions:

genomic-regions (0.0.5)                                - Consistently handle genomic regions
  INSTALLED: 0.0.5 (latest)
pybedtools (0.8.1)        - Wrapper around BEDTools for bioinformatics work
  INSTALLED: 0.8.1 (latest)

Does fanc works on Dnase Hic or Microc?

A quick question. Can FAN-C process fastq file from Dnase Hic or MicroC data that do not has restriction enzyme cutting sites?

Thanks

Gang

sorting with sambamba ignores the specified number of threads in auto

Sambamba has -t, --nthreads=NTHREADS option. But it is not being used when run in auto with bam input. Since there are 2 bam files per each run, each sambamba run should get THREADS/2 from the fanc call.

invalid value encountered in true_divide

Hi,

After obtaining .hic file, finc would throw out such warnings as below, when calling AB compartment.

I am not sure what it would influence. And for the AB domains, I found all the scores of domains are negative value, no matter what type they are assigned. How could it be?
My command is : fanc compartments 40Kb.ZW.hic 40Kb.ZW.AB.matrix -d 40Kb.ZW.AB.domain -v 40Kb.ZW.AB.ev.txt -g ZW.fa -w -e 40Kb.ZW.AB.profile.png -p 10 20 30 40 50 60 70 80 90 100

Best~
Jing

fanc compare almost empty matrices

I'm using fanc compare to compare hic matrices at 10kb resolution with KR normalization, either with fold-change and difference. This is the command for example for difference:

$ fanc compare HiC-1.res10kb.KR.hic HiC-2.res10kb.KR.hic HiC-diff.res10kb.KR.hic -c diff -S

I get a file more heavy than the original ones, however when I pick a region to show with fancplot, it is empty. If I do fanc dump, I only get information of a tiny subset of bin pairs, but the difference in those contacts seems to be well calculated. For example, for chr1 I obtain 7602181 pairs for HiC-1, 15870999 pairs for HiC-2, but only 41 pairs for HiC-diff.

Am I doing something wrong?

Thank you,

Jose Maria

example produces "builtin cmap 'viridis'" error

I'm trying to work through the example here with version 0.9.15 installed via pip. I keep getting the following error. Am I doing something wrong? Thanks!

$ fanc auto SRR4271982_chr18_19_1.fastq.gzip SRR4271982_chr18_19_2.fastq.gzip output/ \
>           -g hg19_chr18_19.fa -i hg19_chr18_19/hg19_chr18_19 -n fanc_example -t 4 -r HindIII \
>           --split-ligation-junction -q 30 --run-with test
2021-03-29 15:30:38,351 INFO FAN-C version: 0.9.15
2021-03-29 15:30:38,353 INFO Output folder: output/
2021-03-29 15:30:38,353 INFO Input files: SRR4271982_chr18_19_1.fastq.gzip, SRR4271982_chr18_19_2.fastq.gzip
2021-03-29 15:30:38,354 INFO Input file types: fastq, fastq
2021-03-29 15:30:38,354 INFO Final basename: fanc_example (you can change this with the -n option!)
0 (threads: 4, depends on: ): fanc map -m 25 -s 3 -t 4 --no-iterative --restriction-enzyme HindIII SRR4271982_chr18_19_1.fastq.gzip hg19_chr18_19/hg19_chr18_19 output/sam/SRR4271982_chr18_19_1.bam
1 (threads: 4, depends on: ): fanc map -m 25 -s 3 -t 4 --no-iterative --restriction-enzyme HindIII SRR4271982_chr18_19_2.fastq.gzip hg19_chr18_19/hg19_chr18_19 output/sam/SRR4271982_chr18_19_2.bam
2 (threads: 1, depends on: 0, 1): fanc sort_sam -t 4 output/sam/SRR4271982_chr18_19_1.bam
3 (threads: 1, depends on: 0, 1): fanc sort_sam -t 4 output/sam/SRR4271982_chr18_19_2.bam
4 (threads: 4, depends on: 2, 3): fanc pairs -f -g hg19_chr18_19.fa -t 4 -us -r HindIII -q 30.0 -S output/sam/SRR4271982_chr18_19_1.bam output/sam/SRR4271982_chr18_19_2.bam output/pairs/fanc_example.pairs
5 (threads: 1, depends on: 4): fanc pairs -d 10000 -l -p 2 --statistics-plot output/plots/stats/fanc_example.pairs.stats.pdf output/pairs/fanc_example.pairs
6 (threads: 1, depends on: 5): fanc pairs --ligation-error-plot output/plots/stats/fanc_example.pairs.ligation_error.pdf output/pairs/fanc_example.pairs
7 (threads: 1, depends on: 5): fanc pairs --re-dist-plot output/plots/stats/fanc_example.pairs.re_dist.pdf output/pairs/fanc_example.pairs
8 (threads: 1, depends on: 5): fanc hic -f output/pairs/fanc_example.pairs output/hic/fanc_example.hic
9 (threads: 4, depends on: 8): fanc hic -f -b 5000000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_5mb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_5mb.hic
10 (threads: 4, depends on: 8): fanc hic -f -b 2000000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_2mb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_2mb.hic
11 (threads: 4, depends on: 8): fanc hic -f -b 1000000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_1mb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_1mb.hic
12 (threads: 4, depends on: 8): fanc hic -f -b 500000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_500kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_500kb.hic
13 (threads: 4, depends on: 8): fanc hic -f -b 250000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_250kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_250kb.hic
14 (threads: 4, depends on: 8): fanc hic -f -b 100000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_100kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_100kb.hic
15 (threads: 4, depends on: 8): fanc hic -f -b 50000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_50kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_50kb.hic
16 (threads: 4, depends on: 8): fanc hic -f -b 25000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_25kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_25kb.hic
17 (threads: 4, depends on: 8): fanc hic -f -b 10000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_10kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_10kb.hic
18 (threads: 4, depends on: 8): fanc hic -f -b 5000 -r 0.1 -t 4 --statistics-plot output/plots/stats/fanc_example_5kb.stats.pdf -n --norm-method kr output/hic/fanc_example.hic output/hic/binned/fanc_example_5kb.hic
Singularity> fanc auto SRR4271982_chr18_19_1.fastq.gzip SRR4271982_chr18_19_2.fastq.gzip output/           -g hg19_chr18_19.fa -i hg19_chr18_19/hg19_chr18_19 -n fanc_example -t 4 -r HindIII           --split-ligation-junction -q 30
2021-03-29 15:30:54,988 INFO FAN-C version: 0.9.15
2021-03-29 15:30:54,991 INFO Output folder: output/
2021-03-29 15:30:54,991 INFO Input files: SRR4271982_chr18_19_1.fastq.gzip, SRR4271982_chr18_19_2.fastq.gzip
2021-03-29 15:30:54,991 INFO Input file types: fastq, fastq
2021-03-29 15:30:54,991 INFO Final basename: fanc_example (you can change this with the -n option!)
2021-03-29 15:30:54,991 INFO Creating output folders...
2021-03-29 15:30:56,425 INFO FAN-C version: 0.9.15
2021-03-29 15:30:56,427 INFO Starting mapping for SRR4271982_chr18_19_1.fastq.gzip
2021-03-29 15:30:56,492 INFO Starting to output alignments to intermediate SAM file /tmp/tmpj0wow_5h/intermediate.sam
2021-03-29 15:33:56,809 INFO Converting intermediate SAM file to BAM (output/sam/SRR4271982_chr18_19_1.bam)
2021-03-29 15:34:22,207 INFO Success, removing intermediate.
2021-03-29 15:34:24,041 INFO FAN-C version: 0.9.15
2021-03-29 15:34:24,043 INFO Starting mapping for SRR4271982_chr18_19_2.fastq.gzip
2021-03-29 15:34:24,103 INFO Starting to output alignments to intermediate SAM file /tmp/tmpvly3ik60/intermediate.sam
2021-03-29 15:37:24,126 INFO Converting intermediate SAM file to BAM (output/sam/SRR4271982_chr18_19_2.bam)
2021-03-29 15:37:48,878 INFO Success, removing intermediate.
2021-03-29 15:37:50,728 INFO FAN-C version: 0.9.15
2021-03-29 15:37:50,730 INFO Cannot find sambamba on this machine, falling back to samtools sort. This is not a problem, but if you want to speed up your SAM/BAM file sorting, install sambamba and ensure it is in your PATH!
2021-03-29 15:37:50,739 INFO FAN-C version: 0.9.15
2021-03-29 15:37:50,740 INFO Cannot find sambamba on this machine, falling back to samtools sort. This is not a problem, but if you want to speed up your SAM/BAM file sorting, install sambamba and ensure it is in your PATH!
2021-03-29 15:38:21,800 INFO Replacing input SAM/BAM file output/sam/SRR4271982_chr18_19_2.bam with sorted version...
2021-03-29 15:38:22,976 INFO Replacing input SAM/BAM file output/sam/SRR4271982_chr18_19_1.bam with sorted version...
2021-03-29 15:38:27,006 INFO FAN-C version: 0.9.15
2021-03-29 15:38:27,007 INFO Getting genome regions (fragments or bins)
2021-03-29 15:38:27,007 INFO Getting regions
2021-03-29 15:38:36,876 INFO Three arguments detected, assuming SAM/BAM input.
[E::idx_find_and_load] Could not retrieve index file for 'output/sam/SRR4271982_chr18_19_1.bam'
2021-03-29 15:38:36,878 INFO Using filters appropriate for Bowtie2.
[E::idx_find_and_load] Could not retrieve index file for 'output/sam/SRR4271982_chr18_19_1.bam'
[E::idx_find_and_load] Could not retrieve index file for 'output/sam/SRR4271982_chr18_19_2.bam'
Buffers 100% (630 of 630) |##################################################| Elapsed Time: 0:00:02 Time:  0:00:02
2021-03-29 15:39:55,093 INFO Updating region mappability
Mappability 100% (1612198 of 1612198) |######################################| Elapsed Time: 0:00:10 Time:  0:00:10
2021-03-29 15:40:13,234 INFO All done.
Closing remaining open files:8L87S8...done
2021-03-29 15:40:14,952 INFO FAN-C version: 0.9.15
2021-03-29 15:40:14,954 INFO One argument received, assuming existing Pairs object.
2021-03-29 15:40:14,984 INFO Filtering reads with RE distance > 10000bp
2021-03-29 15:40:14,984 INFO Filtering self-ligated read pairs
2021-03-29 15:40:14,985 INFO Filtering PCR duplicates, threshold <= 2bp
2021-03-29 15:40:25,784 INFO PCR duplicate stats: 186071 (11.5%) of pairs marked as duplicate.  (multiplicity:occurances) 3:15319 2:149415 4:1520 5:206 10:6 12:2 6:43 7:20 8:6 9:4 14:2
2021-03-29 15:40:25,784 INFO Running filters...
Filter 100% (630 of 630) |###################################################| Elapsed Time: 0:00:36 Time:  0:00:36
2021-03-29 15:41:09,994 INFO Total: 1612198. Valid: 666177
2021-03-29 15:41:09,994 INFO Updating region mappability
Mappability 100% (666177 of 666177) |########################################| Elapsed Time: 0:00:08 Time:  0:00:08
2021-03-29 15:41:26,919 INFO Done.
2021-03-29 15:41:27,919 INFO generated new fontManager
2021-03-29 15:41:43,637 INFO Saving statistics...
Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 127, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.8/dist-packages/fanc/commands/fanc_commands.py", line 1071, in pairs
    from fanc.plotting.statistics import summary_statistics_plot
  File "/usr/local/lib/python3.8/dist-packages/fanc/plotting/__init__.py", line 226, in <module>
    plt.register_cmap(name='viridis', cmap=viridis)
  File "/usr/local/lib/python3.8/dist-packages/matplotlib/cm.py", line 149, in register_cmap
    raise ValueError(msg)
ValueError: Trying to re-register the builtin cmap 'viridis'.
Closing remaining open files:output/pairs/fanc_example.pairs...done
Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 127, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.8/dist-packages/fanc/commands/fanc_commands.py", line 115, in auto
    return fanc.commands.auto.auto(argv, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/fanc/commands/auto.py", line 1290, in auto
    runner.run()
  File "/usr/local/lib/python3.8/dist-packages/fanc/commands/auto.py", line 141, in run
    raise RuntimeError("Task {} had non-zero exit status. "
RuntimeError: Task 32ffbe6c-b346-4f4e-9fad-f45d46bd12aa had non-zero exit status. Cancelling execution of all tasks.

fanc compare error

Hi,

I seem to be running into a couple of different types of errors when executing fanc compare on separate datasets (the command also takes many days to run its course and ends up generating the file, but it seems incomplete):

PATH=$PATH:/Genomics/grid/users/xbing/miniconda3/bin
source activate snakes
module load hdf5/1.8.8
fanc compare M_23del.cool M_WT.cool 23del_WT
2020-10-19 13:22:32,009 INFO FAN-C version: 0.9.4
2020-10-19 18:54:53,547 INFO Calculating scaling factor...
Traceback (most recent call last):
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/bin/fanc", line 127, in <module>
    Fanc()
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/commands/fanc_commands.py", line 
3358, in compare
    oe=oe, norm=norm, log=log_matrix)
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/architecture/comparisons.py", lin
e 287, in from_matrices
    for edge in matrix1.edges((chromosome1, chromosome2), **kwargs):
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/matrix.py", line 755, in __call__
    *args, **kwargs):
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/compatibility/cooler.py", line 42
6, in _edges_subset
    yield self._series_to_edge(row, lazy_edge=lazy_edge, *args, **kwargs)
TypeError: _series_to_edge() got an unexpected keyword argument 'mode'
Closing remaining open files:23del_WT...done

And the other one:

fanc compare M_TADdel.cool M_WT.cool TADdel_WT
2020-10-22 10:02:53,594 INFO FAN-C version: 0.9.4
2020-10-22 14:57:29,657 INFO Calculating scaling factor...
Traceback (most recent call last):
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/bin/fanc", line 127, in <module>
    Fanc()
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/commands/fanc_commands.py", line 
3358, in compare
    oe=oe, norm=norm, log=log_matrix)
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/architecture/comparisons.py", lin
e 274, in from_matrices
    sf = matrix2.scaling_factor(matrix1)
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/matrix.py", line 1312, in scaling
_factor
    for v2 in matrix.edge_data(weight_column, lazy=True):
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/matrix.py", line 848, in edge_dat
a
    for edge in self.edges(*args, **kwargs):
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/matrix.py", line 696, in __call__
    bias = self._regions_pairs.bias_vector()
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/compatibility/cooler.py", line 45
6, in bias_vector
    return np.array([r.bias for r in self.regions(lazy=True)])
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/compatibility/cooler.py", line 45
6, in <listcomp>
    return np.array([r.bias for r in self.regions(lazy=True)])
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/fanc/compatibility/cooler.py", line 22
1, in __getattr__
    return getattr(self._series, item)
  File "/Genomics/grid/users/xbing/miniconda3/envs/snakes/lib/python3.6/site-packages/pandas/core/generic.py", line 5139, in
 __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'bias'
Closing remaining open files:TADdel_WT...done

Please help! Thank you!

Loops Exporting to BEDPE

Hi,
I have successfully detect the loops but have issues while exporting them into BEDPE format (got empty file). I have used command as below:
3.7.0/bin/fanc loops chrX.filtered.noSinglets.merged.loops -b chrX.merged.bedpe

Thanks

fanc compartments

Hi,
According to your instructions, I am trying to generate AB enrichment profile heat maps. I would like to change the bin size to a finer binning. However, it did not work. My command entered is below:

$ fanc compartments -e test.png -p 10 20 30 40 50 60 70 80 90 100 test_1mb.hic test.ab

When I did this analysis without "-p" option, I could generate a heat map successfully.
Thank you in advance for your cooperation!
Best,
Daisuke

fanc pairs takes a long time and without any more ouput

Dear all,

I used the valid pairs generated by HiC-Pro as input for fanc pairs, the command is as following, but it runs slowly and three days without any more ouput. The timestamp of the output file just stayed on the time I submitted the commands. Do you have some suggestions on accelerating the progress?
fanc pairs ${pre}_mm10_index.bwt2pairs.validPairs ${pre}_fanc.pairs -g fragment.bed -t 10 -s ${pre}.statistic --filter-pcr-duplicates 1

Best wishes,
Zheng zhuqing

convert cool to FAN-C format

Hi,
I used patient_hic = fanc.load(case) to load cool Hi-C, and then run patient_hic[region_string, region_string].data to get data of interest.
However, I did not get any result.
do you know what is going on?

  I also run `fanc from-cooler `to convert cool to FAN-C format, but I got this error:
2020-11-16 19:39:04,974 INFO FAN-C version: 0.9.7
Traceback (most recent call last):
  File "/nas/longleaf/home/anaconda2/envs/py38/bin/fanc", line 127, in <module>
    Fanc()
  File "/nas/longleaf/home/anaconda2/envs/py38/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/fanc/commands/fanc_commands.py", line 1850, in from_cooler
    cool.deepcopy(fanc.Hic, file_name=output_file, mode='w')
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/fanc/matrix.py", line 1324, in deepcopy
    copy.add_regions(self.regions(lazy=True))
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/fanc/regions.py", line 664, in add_regions
    self.add_region(region, *args, **kwargs)
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/genomic_regions/regions.py", line 1199, in add_region
    return self._add_region(region.copy(), *args, **kwargs)
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/genomic_regions/regions.py", line 505, in copy
    d = {attribute: getattr(self, attribute) for attribute in self.attributes}
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/genomic_regions/regions.py", line 505, in <dictcomp>
    d = {attribute: getattr(self, attribute) for attribute in self.attributes}
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/fanc/compatibility/cooler.py", line 221, in __getattr__
    return getattr(self._series, item)
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/pandas/core/generic.py", line 5139, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'weight'
Closing remaining open files:NeuNpos_10000.fanc...done
Exception ignored in: <function Node.__del__ at 0x7ff6cc899a60>
Traceback (most recent call last):
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/tables/node.py", line 319, in __del__
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/tables/table.py", line 2961, in _f_close
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/tables/table.py", line 2896, in flush
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/tables/table.py", line 626, in autoindex
  File "/nas/longleaf/home/anaconda2/envs/py38/lib/python3.8/site-packages/tables/file.py", line 1590, in _get_node
AttributeError: 'File' object has no attribute '_node_manager'

best,

BEDPE plotting

Hello!

I'm interested in visualizing interactions (BEDPE files) with fancplot or through the Python API. It seems that it's currently possible to overlay them as circles on top of contact maps using HicPeakPlot. But I'm wondering if there are ways to plot them as curved lines between anchors on a separate panel (e.g., using matplotlib.patches.Arc as pyGenomeTracks have done). Optionally, it'd be great to have to ability to vary the opacity/thickness of lines based on some "score" column.

Would this be of interest to implement as a new plot class? Or perhaps there is a simpler way of adding them on top of existing panels through matplotlib directly? I tried looking into the latter option but had a hard time figuring out where to start.

Any pointers would be appreciated

qname_ix missing

Thank you for developing this fantastic library bringing together so many useful functions! I'm looking to use it for only alignment / filtering pairs at the moment and did the following:

fanc map ${samp}_R1.fq.gz $REF/bowtie2_index/mm10 ${samp}_R1.sam -t 48 -r DpnII
fanc map ${samp}_R2.fq.gz $REF/bowtie2_index/mm10 ${samp}_R2.sam -t 48 -r DpnII
samtools sort -@ 47 -n -o ${samp}_R1.bam ${samp}_R1.sam
samtools sort -@ 47 -n -o ${samp}_R2.bam ${samp}_R2.sam
fanc pairs -g $REF/bowtie2_index/mm10.fa \
  -r DpnII -q 30 -t 48 -p 2 -d 500 -i 1000 \
  -f -l -m -us \
  ${samp}_R1.bam \
  ${samp}_R2.bam \
  ${samp}.pair

I then proceeded to use the python API to look at the ouput:

import fanc
p = fanc.load('out.pair')

But the likes of p[0].left.qname_ix and p[0].right.qname_ix all return None. Is this intended? How can I make sure that attribute is assigned in the constructor?

Since my goal is to actually obtain the set difference between two pair files (same set of filters except one also have a uniqueness filter and one does not, to specifically take multi-mapping pairs) the qname or its index is critical.

Though I did find that the parent FragmentReadPair objects do contain a valid 'ix' attribute, but I couldn't figure out what it is from the documentation. Should this also correspond to the read name?

How to deal with lots of contigs?

hic.py:268: UserWarning: Number of chromosomes (1061) is very large. Consider limiting the processed chromosomes using the 'chromosomes' argument or expect significant slowdown during binning!

There are chromosomes in the assembly + 1000 of contigs (up to 1Mb, but mostly shorter)
Isn't binning time linear with the total length of chromosomes/contigs? (Or quadratic if it means contact binning).
Or should I really worry about this warning message? If yes, then how do you recommend running the pipeline?

Attribute error with fanc.hic LowCoverageFilter

Hi Kai,

I wanted to do some downstream analyse for Juicer-generated .hic files using your FANC python API (nice name, btw).

As suggested in your documentation, I was trying to filter my 5kb binned .hic using LowCoverageFilter. This took a while to run, and then python threw an Attribute error:

AttributeError Traceback (most recent call last)
in
1 h22 = fanc.load("./inter_30.hic@5000")
2 low_cov_filt = LowCoverageFilter(h22, rel_cutoff = 0.25)
----> 3 h22.filter(low_cov_filt)
4 h22.run_queued_filters()

AttributeError: 'JuicerHic' object has no attribute 'filter'

Any idea where I might have gone wrong? I'm using fanc 0.9.1 in a conda env with python 3.7.5.

Thanks a lot!

Mikie

Documentation page down

Hi,

Your doc site https://vaquerizaslab.github.io/fanc is down. Would you mind letting me know when it'll be back on? That'd be really awesome. thanks!

Best,

argument order and documentation of fanc fragments

usage: fanc fragments [-h] [-c CHROMOSOMES [CHROMOSOMES ...]]
                      input re_or_bin_size output

If I am not mistaken the order of arguments to fanc fragments does matter.
--chromosomes has nargs='+', so it cannot be passed before the remaining 3 unnamed arguments as the help message suggests. It can only go at the end.

And I had to guess that the list of chromosomes should be space-separated. My first guess was a coma. So CHROMOSOMES isn't actually a list of chromosomes, but a single chromosome and should be renamed.

Also: It is not clear to me which restriction enzymes are supported.

Second eigenvector extraction

I'm using fanc compartments to extract eigenvectors. Sometimes, one needs the second eigenvector. So, I run the following command:

# Resolution
RESOLUTION="1m"
# Eigenvector index
EIGINDEX=2

fanc compartments --domains ${DIROUT}/AB_SAMPLE1_${RESOLUTION}_${EIGINDEX}.bed \
				  --eigenvector ${DIROUT}/AB_SAMPLE1_ev_${RESOLUTION}_${EIGINDEX}.bed \
				  --eigenvector-index $EIGINDEX \
				  --genome $GENOME \
				  ${DIRIN}/SAMPLE1.hic@${RESOLUTION} ${DIROUT}/AB_SAMPLE1_${RESOLUTION}_${EIGINDEX}.ab

Extracting the first eigenvector works fine, at 1Mb or 10kb resolutions. But when I try to extract the second eigenvector, the jobs fail. Errors are different, sometimes memory issue (with 10kb, but all worked fine for the first eigenvector), sometimes various Python errors. Any suggestions what to do?

One of the errors is:

Traceback (most recent call last):
  File "/home/username/miniconda3/envs/fanc/bin/fanc", line 127, in <module>
    Fanc()
  File "/home/username/miniconda3/envs/fanc/bin/fanc", line 93, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/home/username/miniconda3/envs/fanc/lib/python3.7/site-packages/fanc/commands/fanc_commands.py", line 3937, in compartments
    eigenvector=ev_index, force=recalculate)
  File "/home/username/miniconda3/envs/fanc/lib/python3.7/site-packages/fanc/architecture/compartments.py", line 210, in eigenvector
    ab_vector = v[:, eigenvector]
  File "/home/username/miniconda3/envs/fanc/lib/python3.7/site-packages/fanc/matrix.py", line 2740, in __getitem__
    out = np.ma.MaskedArray.__getitem__(self, index)
  File "/home/username/miniconda3/envs/fanc/lib/python3.7/site-packages/numpy/ma/core.py", line 3225, in __getitem__
    dout = self.data[indx]
IndexError: index 1 is out of bounds for axis 1 with size 1

Error when generating pairs file from 4Dnucleome file: AttributeError: 'NoneType' object has no attribute 'Lock'

Hi,

I am trying to generate a pairs file using fanc pairs from a 4Dnucleome formatted file. I am using it like this:
fanc pairs -g /data/slow_disk/share/imotago/LowCoverage/merge_nodups_files/test_files/mm10_noScaffolds.fa -r MboI -s stats_SF1_XX_pairs.txt --statistics-plot stats_plot_SF1_XX.png SF1_XX_formatted.pairs merge_nodups_final_SF1_XX_rep1_2_pairs.pairs

and I am getting the following error:

Traceback (most recent call last):
  File "/home/imotago/.local/bin/fanc", line 124, in <module>
    Fanc()
  File "/home/imotago/.local/bin/fanc", line 90, in _init_
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/home/imotago/.local/lib/python3.6/site-packages/fanc/commands/fanc_commands.py", line 994, in pairs
    pairs.add_read_pairs(sb, threads=threads, batch_size=batch_size)
  File "/home/imotago/.local/lib/python3.6/site-packages/fanc/pairs.py", line 1496, in add_read_pairs
    for fi1, fi2 in self._read_pairs_fragment_info(read_pairs, batch_size=batch_size, threads=threads):
  File "/home/imotago/.local/lib/python3.6/site-packages/fanc/pairs.py", line 1275, in _read_pairs_fragment_info
    monitor = Monitor()
  File "/home/imotago/.local/lib/python3.6/site-packages/fanc/pairs.py", line 137, in _init_
    self.generating_pairs_lock = manager.Lock()
AttributeError: 'NoneType' object has no attribute 'Lock'
Closing remaining open files:merge_nodups_final_SF1_XX_rep1_2_pairs.pairs...doneO6X9UQ...done

My input file looks like this:

## pairs format v1.0 
#shape: upper triangle
#genome_assembly: mm10
#chromsize: chr1 195471971
#chromsize: chr2 182113224
#chromsize: chr3 160039680
#chromsize: chr4 156508116
#chromsize: chr5 151834684
#chromsize: chr6 149736546
#chromsize: chr7 145441459
#chromsize: chr8 129401213
#chromsize: chr9 124595110
#chromsize: chr10 130694993
#chromsize: chr11 122082543
#chromsize: chr12 120129022
#chromsize: chr13 120421639
#chromsize: chr14 124902244
#chromsize: chr15 104043685
#chromsize: chr16 98207768
#chromsize: chr17 94987271
#chromsize: chr18 90702639
#chromsize: chr19 61431566
#chromsize: chrX 171031299
#chromsize: chrY 91744698
#chromsize: chrM 16299
#columns: readID chr1 pos1 chr2 pos2 strand1 strand2
K00302:125:HC335BBXY:3:2105:5142:9877/2K00302:125:HC335BBXY:3:2105:5142:9877/1 chr1 3000015 chr1 3012702 + -
K00302:113:H3L3TBBXY:4:1127:15168:40367/1K00302:113:H3L3TBBXY:4:1127:15168:40367/2 chr1 3000027 chr1 3001190 + -
K00302:113:H3L3TBBXY:4:1209:5457:10563/1K00302:113:H3L3TBBXY:4:1209:5457:10563/2 chr1 3000041 chr1 3022118 + -
K00302:125:HC335BBXY:3:2110:28321:9631/2K00302:125:HC335BBXY:3:2110:28321:9631/1 chr1 3000109 chr1 3559591 + -

Do you know how I can solve this issue?
Thanks in advance!

Loop aggregate (brighter corner + loop strength)

First of all thanks for the amazing resource FANC is.

I am performing some loop aggregates on the Bonev 2017 data I am a bit confused with the results. I assume it is normal that one of the four corners of the heatmap it's brighter, in principle the one that correspond to the "inner" part of the loop closer to the diagonal. However, when I retrieve the matrices such corner is the upper-right corner while in your publications it shows as the bottom-left one (which I think it's much more intuitive). Is there a way to figure out what is happening? Can I just flip it, or there is something wrong in my code that is more profound?

Also, I think I've noticed that the loop strength calculation has been placed in legacy, is there a way to make use of it still from the latest version? Is it problematic in some sense?

Thanks a lot!

Rafa.

ModuleNotFoundError: No module named 'fanc.tools.sambam'

Hi fanc developers,

I'm attempting to install fanc following the instructions as documented in the https://fan-c.readthedocs.io/en/latest/getting_started.html#installation. Installation seems to complete successfully (conda environment on Python 3.6 and 3.7), but invoking fanc --version or fanc -h raises an error "ModuleNotFoundError: No module named 'fanc.tools.sambam'". I'm wondering if there's a roundabout to this problem.

Thank you.

License invokes the GPL but restricts the applicability of the GPL which violates the GPL

Hi,

the license says that for non-commercial use the GPL version 2 applies, but that for commercial use another license must be obtained.

This creates a conflict with the text of the GPL, which grants the rights to use the software for any purpose --- including commercial use. Furthermore, the GPL cannot be extended with additional restrictions:

[...] You may not impose any further restrictions on the recipients' exercise of the rights granted herein. [...]

Does this mean that I may not use fanc under the terms of the GPL at all? This, sadly, prevents inclusion of fanc in free software distributions, such as Debian or GNU Guix.

I would be happy if you could clarify this and perhaps remove the misleading references to the GPL when the license actually is intended to restrict usage contrary to the guarantees granted by the GPL.

Thanks!

Error in fanc auto - .hic as input

Hi,

This thread was mentioned here #9 (comment)
When using juice .hic as input in fanc auto, I get the following error:

$ fanc auto '/media/test/Data/Data/HiC/HiC_fused_150_PE/Juicer/testEtOH/K5testEtOH.allValidPairs.hic' '/media/test/Data/Data/HiC/HiC_fused_150_PE/FANC'
2020-02-17 12:46:45,532 INFO Output folder: /media/test/Data/Data/HiC/HiC_fused_150_PE/FANC/
2020-02-17 12:46:45,532 INFO Input files: /media/test/Data/Data/HiC/HiC_fused_150_PE/Juicer/testEtOH/K5testEtOH.allValidPairs.hic
2020-02-17 12:46:45,532 INFO Input file types: hic
2020-02-17 12:46:45,533 INFO Final basename: K5testEtOH.allValidPairs (you can change this with the -n option!)
2020-02-17 12:46:45,533 INFO Creating output folders...
/usr/local/lib/python3.6/dist-packages/fanc/compatibility/juicer.py:273: UserWarning: No resolution chosen for Juicer Hic! Using 2500000bp
  warnings.warn("No resolution chosen for Juicer Hic! Using {}bp".format(resolution))
usage: fanc hic [-h] [-b BIN_SIZE] [-l FILTER_LOW_COVERAGE]
                [-r FILTER_LOW_COVERAGE_RELATIVE] [-a] [-d FILTER_DIAGONAL]
                [--marginals-plot MARGINALS_PLOT] [--reset-filters]
                [--downsample DOWNSAMPLE] [-i] [-k] [-w] [-c] [--only-inter]
                [-s STATS] [--statistics-plot STATS_PLOT]
                [--chromosomes CHROMOSOMES [CHROMOSOMES ...]] [-f]
                [-t THREADS] [-tmp]
                input [input ...]
fanc hic: error: File (/media/test/Data/Data/HiC/HiC_fused_150_PE/Juicer/testEtOH/K5testEtOH.allValidPairs.hic) type <class 'fanc.compatibility.juicer.JuicerHic'> not supported.Provide Pairs or Hic files!
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/fanc_commands.py", line 1338, in hic
    "Provide Pairs or Hic files!".format(original_input_file, type(o)))
  File "/usr/lib/python3.6/argparse.py", line 2402, in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
  File "/usr/lib/python3.6/argparse.py", line 2389, in exit
    _sys.exit(status)
SystemExit: 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 124, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/fanc_commands.py", line 1341, in hic
    o.close()
AttributeError: 'JuicerHic' object has no attribute 'close'
Traceback (most recent call last):
  File "/usr/local/bin/fanc", line 124, in <module>
    Fanc()
  File "/usr/local/bin/fanc", line 90, in __init__
    command([sys.argv[0]] + sys.argv[option_ix:], log_level=log_level, verbosity=verbosity)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/fanc_commands.py", line 114, in auto
    return fanc.commands.auto.auto(argv, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/auto.py", line 1186, in auto
    runner.run()
  File "/usr/local/lib/python3.6/dist-packages/fanc/commands/auto.py", line 141, in run
    "Cancelling execution of all tasks.".format(task.id))
RuntimeError: Task 3b2018a4-1a22-4505-a546-282b36d7f25d had non-zero exit status. Cancelling execution of all tasks.

thanks,

normalization before fanc pca

Dear @kaukrise

Should the hic files be binning (e.g. by any specific resolution), filtered (e.g. by setting FILTER_LOW_COVERAGE_RELATIVE) and normalized (e.g. KR normalization) when running fanc pca? Thank you.

Best wishes,
Zheng zhuqing

vaquerizaslab / fanc Goto Github PK

fanc's People

Contributors

Stargazers

Watchers

Forkers

fanc's Issues

fanc compartments -g genome.fa -e chrX.pdf chrX.hic chrX.ab -v chrX.ev.txt -s 0 -tmp

Recommend Projects

Recommend Topics

Recommend Org