koszullab / instagraal Goto Github PK

Thank you for developing this super useful tool.

I have installed the instaGRAAL with no-opengl branch because freeglut always showed the display error same as others.

Then I ran the following command:

instagraal hic ph.a60.FALC.fasta

It looks like start running (INFO :: start filtering), but soon encounters an error:

INFO :: start filtering
Traceback (most recent call last):
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/bin/instagraal", line 33, in <module>
    sys.exit(load_entry_point('instagraal==0.1.6', 'console_scripts', 'instagraal')())
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/instagraal-0.1.6-py3.8.egg/instagraal/instagraal.py", line 1164, in main
    p2 = instagraal_class(
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/instagraal-0.1.6-py3.8.egg/instagraal/instagraal.py", line 151, in __init__
    self.simulation = simulation(
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/instagraal-0.1.6-py3.8.egg/instagraal/simu_single.py", line 74, in __init__
    self.select_data_set(name)
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/instagraal-0.1.6-py3.8.egg/instagraal/simu_single.py", line 739, in select_data_set
    self.hic_pyr = pyr.build_and_filter(
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/instagraal-0.1.6-py3.8.egg/instagraal/pyramid_sparse.py", line 103, in build_and_filter
    pyramid_0 = h5py.File(init_pyramid_file)
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/h5py/_hl/files.py", line 424, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "/home/tin_hang_hung/miniconda3/envs/instagraal/lib/python3.8/site-packages/h5py/_hl/files.py", line 190, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 96, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = './pyramids/pyramid_1_no_thresh/pyramid.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

I will be very grateful if you can help with this problem.

Hi,

should be
current_seq = current_seq + extra_seq

PS: The "-j" option only works in "fasta" mode

hugo

HI,
I am planning to use the installGRAAL for my 3C data. I know it is necessary to generate matrix data using hicstuff . But I have some details about the usage of hicstuff you mentioned in the METHOD of paper " Chromosome-level quality scaffolding of brown algal genomes using InstaGRAAL, a
proximity ligation-based scaffolder "
You say you used the parameter --maxin 5 -very�sensitive-local , so could you explain the details about these parameters ? And you Hic data sequence read is 75 bp long , so if sequencing length is 150 bp , are these parameters fittable ? Besides , you used the bowtie2 sepeartely in the paper , not use the hicstuff pipeline ? I am puzzled because I did not find --maxin 5 -very�sensitive-local in the hicstuff pipeline .
second , in your paper , you mentioned you filterd some alignments not fittable for some need ,such as those coverage was less than one standard deviation below the mean of the global coverage distribution ,so are these parameters also default ? or anything else ?
Hope you can give me some advice about the usage you mentioned . Thank you .

Hi,
I ran the software successfully . I use my date ,generated by the command hicstuff pipeline --enzyme MboI --iterative -g mer_20t_scf.fasta -o ./ -t 35 hic_1_clean.fastq.gz hic_2_clean.fastq.gz ,to scaffold my genome .

INFO :: p from estimate parameters = [0.609231115267729, 62.68078982754462, -1.1638487993936344, 2, 54875.35389675029]
INFO :: mean value trans = 0.04067848007063246
INFO :: BEWARE!!! : I will lower mean value trans !!!
INFO :: estimate max dist cis trans = 27023.030339687124
INFO :: cycle = 0
INFO :: generate new fasta file...
INFO :: cycle = 1
INFO :: generate new fasta file...
INFO :: cycle = 2
INFO :: generate new fasta file...
INFO :: cycle = 3
INFO :: generate new fasta file...
INFO :: cycle = 4
WARNING :: /usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/cuda_lib_gl_single.py:1868: RuntimeWarning: invalid value encountered in less
filtered_score[filtered_score < 0] = 0

INFO :: generate new fasta file...
INFO :: cycle = 5
WARNING :: /usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/cuda_lib_gl_single.py:3626: RuntimeWarning: overflow encountered in exp
ratio = np.exp((self.likelihood_nuis - self.likelihood_t) / F_t)

I don't know whether I should worry about the warning like "lower mean value " and other warning as you can see .
In addition, how many cycles I should run to get a good result ? I use the default set , but you recommend in other issue 15 times will be OK . How can I asses the good result ? By genome metrics like N50 ,or something else ?
Thank you !

Hi,
I tried another server with GPU , when I type nvidia-smi ,it shows
-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla V100-SXM2... Off | 00000000:B4:00.0 Off | 0 |
| N/A 33C P0 38W / 300W | 0MiB / 16130MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

and I try to install on it . During my installation , the performance of the server is "6 CPU / 1 GPU / 60.000GB (single node)" . I did what you wrote on the github, and finally it seems right.
But when I type "instagraal -h ", it threw "Segmentation fault (core dumped)" . However, it works fine when I typed "instagraal-polish -h" . I guess it is because of out of memory ? do you have any idea to deal with ?
Here is the core file when it dumped . Maybe you can see the problem from the log file .
core.96317.gz

I've run hicstuff, instagraal, and polish to obtain a scaffold assembly (code below). As expected, the final polished genome has better contiguity measures than the initial reference assembly, which was a long read assembly (Initial ref N90 = 0.71, N50 = 10.43; whereas the final polished genome N90 = 12.52, N50 = 24.83).

However, I am surprised that the sequence length (174.19) and file size (177.1 MB) is equal between the initial reference assembly and final polished genome. This is especially surprising to me considering the assemblies have different contig counts (initial ref = 527 vs. final polished genome = 707). It is also interesting to note that the polishing step did not change our instagraal output assembly (genome.fasta) at all.

1. Is it "normal" behavior for the initial reference assembly to be the same length as the final polished genome?
2. Is it "normal" behavior for the polishing step to not improve the assembly at all?

hicstuff pipeline -t 60 -a minimap2 -e DpnII -o hicstuff_out/ -g initial.fa hic_reads_R1.fastq hic_reads_R2.fastq
instagraal hicstuff_out initial.fa instagraal_out/
instagraal-polish -m polishing -i info_frags.txt -f initial.fa -o final_polished_assembly.fa

Hi there,

I'm trying to run instaGRAAL, but the only NVIDIA-GPU powered compute I have access to is within an HPC cluster environment. It's difficult therefore to install following the recommendations (no root access to a package manager), and without root, I can't use docker either.

However, I can build a singularity image from the docker file like so (using Singularity 3.6.4):

singularity build instagraal.sif docker://koszullab/instagraal

When running, I start a new ssh session (with -Y), start an interactive session on the gpu node I wish to use (w/ Tesla V100), using LSF options to enable X forwarding, and run instagraal like:

singularity run --nv instagraal.gif SsphL_hicstuff/ SsphL.genome.fasta instagraal_out/

But all that happens is a brief flutter of an X11 window opening and immediately closing again, a segmentation fault error, and my cursor being returned to me. I have seen similar behavior in non-interactive run modes and on a windows box through putty (with X11 forwarding enabled and using Xming for display).

I've also tried running like:

DISPLAY=:0 singularity run --nv instagraal.gif SsphL_hicstuff/ SsphL.genome.fasta instagraal_out/

And this just returns:
freeglut (/usr/local/bin/instagraal): failed to open display ':0'

My problem seems similar to issue #13 . But the docker-related part of that issue never seems to have gotten a resolution.

Can you please advise?

Regards,
Chris L

Instagraal is not workinng with numpy 1.23.0 as numpy asscalar function has been removed.
Use numpy 1.22.4 or below.

̍"""
INFO :: Selected_device: Quadro P2200
INFO :: filtering already done...
WARNING :: /home/usr/repo/instaGRAAL/instagraal/pyramid_sparse.py:122: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
pyramid_handle = h5py.File(hdf5_pyramid_file)

INFO :: level already built...
INFO :: level already built
INFO :: level already built
INFO :: level already built
INFO :: level already built
INFO :: level already built
INFO :: level already built
INFO :: level already built
INFO :: level already built
INFO :: pyramid built.
INFO :: init pyramid
WARNING :: /home/usr/repo/instaGRAAL/instagraal/pyramid_sparse.py:1532: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
self.data = h5py.File(self.pyramid_file)

INFO :: pyramid loaded
INFO :: loading data from level = 4
INFO :: import reference genome
INFO :: loading data from level = 3
INFO :: mean frag area = 488.21014404296875
INFO :: N frag duplicated = 0
INFO :: MAX ID CONTIG = 375
INFO :: total mem used by sparse data = 0.97566
Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal')()
File "/home/usr/repo/instaGRAAL/instagraal/instagraal.py", line 2150, in main
p2 = window(
File "/home/usr/repo/instaGRAAL/instagraal/instagraal.py", line 213, in init
self.simulation = simulation(
File "/home/usr/repo/instaGRAAL/instagraal/simu_single.py", line 139, in init
self.sampler = sampler_lib(
File "/home/usr/repo/instaGRAAL/instagraal/cuda_lib_gl_single.py", line 254, in init
self.setup_all_gpu_struct()
File "/home/usr/repo/instaGRAAL/instagraal/cuda_lib_gl_single.py", line 378, in setup_all_gpu_struct
self.gpu_list_len_cont = ga.zeros(self.n_frags, dtype=np.int32)
File "/usr/local/lib/python3.8/dist-packages/pycuda-2020.1-py3.8-linux-x86_64.egg/pycuda/gpuarray.py", line 1209, in zeros
result = GPUArray(shape, dtype, allocator, order=order)
File "/usr/local/lib/python3.8/dist-packages/pycuda-2020.1-py3.8-linux-x86_64.egg/pycuda/gpuarray.py", line 219, in init
s = np.asscalar(s)
File "/home/usr/.local/lib/python3.8/site-packages/numpy/init.py", line 311, in getattr
raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'asscalar'
#-------------------------------------------------------------------
PyCUDA ERROR: The context stack was not empty upon module cleanup.
#-------------------------------------------------------------------
A context was still active when the context stack was being
cleaned up. At this point in our execution, CUDA may already
have been deinitialized, so there is no way we can finish
cleanly. The program will be aborted now.
Use Context.pop() to avoid this problem.
#-------------------------------------------------------------------
Aborted (core dumped)
"""

Hello,
I get the following error:
freeglut (PATH to instagraal bin)
ERROR: Internal error in function fgOpenWindow
Is there a way to get rid of it?

Thank you

Hi,
Sorry to interrupt you again . I successfully install the software , but meet some questions . It throws "freeglut (/usr/local/bin/instagraal): failed to open display '' "when I type instagraal hic_data mer_20t_scf.fasta HIc_scaff . So I did that export DISPLAY=:0 . Then when I try again the scaffolding , it goes wrong saying "freeglut (/usr/local/bin/instagraal): failed to open display ':0' " .As you say on the github , we have to ensure X on the server , when I type "top ", it shows Xvfb is running on server . By the way , no graphics on the remote client . My server is a linux server only with CPUs and GPUs on . So I can't control it with the movie running on it . Do you have any idea to cope with ? I don't know how to deal with , really .
If I have different servers with varying configurations . Will the more CPUs and GPUs will shorten the time of scaffolding ? Do you have any comparison between different servers of computers ?This may give me some clue for determining which one server to use. And will the software fully use the RAM or anything concerning computing ? for that I noticed that you didnot provide any parameters for thread like that .

It looks the cuda kernels being created are using a deprecated function __shfl_down :

When trying to run the code on a computer with recent hardware, which defaults to a compute capability sm_86 we get the error for the kernel compilation:

  kernel.cu(3823): error: identifier "__shfl_down" is undefined

If we force to build for an old arch passing the option -arch sm_53, it builds with the message

kernel.cu(3823): warning: function "__shfl_down(double, unsigned int, int)"                                                                                                                                                                                                                 
/opt/local/easybuild/software/CUDA/10.1.243-GCC-8.3.0/bin/../targets/x86_64-linux/include/sm_30_intrinsics.hpp(295): here was declared deprecated ("__shfl_down() is deprecated in favor of __shfl_down_sync() and may be removed in a future release (Use -Wno-deprecated-declarations to s
uppress this warning).")                                                                                                                                                                                                                                                                    

this makes the code cumbersome to install and use on recent hardware with recent CUDA versions.

maybe the info on this question can help

https://stackoverflow.com/questions/50639194/shfl-down-and-shfl-down-sync-give-different-results

could you consider patching this ?

There is an option "--bomb" to "explode the genome prior to scaffolding". Could you please explain in more detail what it does?

Hi @nadegeguiglielmoni,

I closed the previous issue as I solved my problems with the tool, specifically those related to scaffolding. I've been able to run it in local, and I got something after about 2-3 days (as you predicted) although the algorithm converged much earlier than expected.

After about 8-9 cycles I noted a sharp reduction in the log_likelihood file, but following your advice I let instaGRAAL run for few more cycles just to make sure it was stable. This is what I got after 20 cycles:

Now, I was reading the section about curation. In particular, I would like to run instagraal-polish -m polishing -i info_frags.txt -f contigs.fasta -o curated_assembly.fa; however, for my understanding I would like to know exactly what to input after the -f and the -o flags. I think after the -f I have to indicate the original FASTA genome, whereas after the -o the output of instaGRAAL after the 20 cycles I run. Let me know, thanks.

If it is used pycuda with the latest version as of today (2022.2.2 ) we get this error

Traceback (most recent call last):                                                                                                                                                                                                                                                          
  File "/home/users/a/l/insta_env/bin/instagraal", line 5, in <module>                                                                                                                                                                                                              
    from instagraal.instagraal import main                                                                                                                                                                                                                                                  
  File "/home/users/a/l/insta_env/lib/python3.7/site-packages/instagraal/instagraal.py", line 72, in <module>                                                                                                                                                                       
    from instagraal.simu_single import simulation                                                                                                                                                                                                                                           
  File "/home/users/a/l/insta_env/lib/python3.7/site-packages/instagraal/simu_single.py", line 12, in <module>                                                                                                                                                                      
    from instagraal.cuda_lib_gl_single import sampler as sampler_lib                                                                                                                                                                                                                        
  File "/home/users/a/l/insta_env/lib/python3.7/site-packages/instagraal/cuda_lib_gl_single.py", line 11, in <module>                                                                                                                                                               
    from pycuda import gpuarray as ga                                                                                                                                                                                                                                                       
  File "/home/users/a/l/insta_env/lib/python3.7/site-packages/pycuda-2022.2.2-py3.7-linux-x86_64.egg/pycuda/gpuarray.py", line 2206                                                                                                                                                 
    def logical_and(x1, x2, /, out=None, *, allocator=None):                                                                                                                                                                                                                                
                            ^                                                                                                                                                                                                                                                               
SyntaxError: invalid syntax                                                                                                                                                                                                                                                                 

i tested with a version close to the instagraal 0.1.6 release and it worked fine

The README instructions should be updated to propose to pull pycuda like e.g.:

git clone --recurse-submodules  --depth 1 --branch v2019.1.2 https://github.com/inducer/pycuda.git

At first I ran a real-case sample with genome size ~2.6Gb and then instagraal crashed on a 400Gig machine. So I decreased the size of fragments contacts, which was generated from hicstuff (I used several replicates of Hi-C read files to generated contacts information at first. After the machine crashed, I only used one replicate to generate smaller contact files). But still, the smaller sample needs ~30Gig Memory on the host, and 2Gig on the GPU, with 780M abs_fragments_contacts_weighted.txt.
In addition, the program ran super slowly that had ran over 1 month before killed. Before ends, it had generated new fasta file in cycles, taking approximately a day per cycle. But that's only a test sample, so I terminated it in cycle 28.
Nevertheless, could you provide any information about the computational resource usage of the program. Like what I have demonstrated, the 'small' sample seems to run endless... I doubt the possibility to apply it on my real sample.

Hi,

I try to use instaGRAAL to do Hi-C scaffolding with the output of hicstuff. Here is the error I got:

INFO :: Start filling the pyramid
INFO :: pyramid built.
INFO :: init pyramid
INFO :: pyramid loaded
INFO :: loading data from level = 4
INFO :: import reference genome
Traceback (most recent call last):
  File "/usr/local/bin/instagraal", line 11, in <module>
    load_entry_point('instagraal', 'console_scripts', 'instagraal')()
  File "/src/instagraal/instagraal/instagraal.py", line 1178, in main
    output_folder=output_folder,
  File "/src/instagraal/instagraal/instagraal.py", line 160, in __init__
    output_folder=output_folder,
  File "/src/instagraal/instagraal/simu_single.py", line 106, in __init__
    self.level.build_seq_per_bin(genome_fasta=self.fasta)
  File "/src/instagraal/instagraal/pyramid_sparse.py", line 2215, in build_seq_per_bin
    seq_frag = self.pyramid.dict_sequence_contigs[init_contig][
KeyError: 'tig00000725|arrow|arrow_np1212'

The tig00000725|arrow|arrow_np1212 contig looks like:

>tig00000725|arrow|arrow_np1212 6077
caggttttcatgtaccaatcatggaatcttcgcgtcatcgttgttagagatctttcatctttgacgagaggcttcccgtaatggtatttgtgttcatccacctccaagaaatcataatgtacatcgtcaggcaggtaatctccaagattggtattgggcaaaatctccggatgattagcgacgatatcgctagacaccttgagcggggggcacgattggttcgcttgttcgcccagctgggcaattttttccccagcttgttgttctgctaacctttgatcactgatagtacttcccgaccgctccgct.....

due to changes in Bio.Seq and IUPAC support in latest biopython, modify the requirements list to install biopython==1.75 and not update to latest. Otherwise i was get an error loading IUPAC running instagraal-polish with v 1.78 biopython.

Hi

After a few hiccups, I managed to get the no_opengl branch to work, this should work on Ubuntu 18.04 server:

sudo apt-get install ubuntu-drivers-common
sudo ubuntu-drivers devices
sudo apt-get install nvidia-driver-460 
# reboot
sudo apt-get install libglu1-mesa libxi-dev libxmu-dev libglu1-mesa-dev freeglut3-dev hdf5-tools libboost-all-dev
sudo apt-get install python3-numpy python3-scipy python3-matplotlib python3-h5py python3-opengl python3-docopt python3-biopython
sudo apt-get install nvidia-cuda-toolkit
sudo apt-get install gcc-8 g++-8

git clone --recurse-submodules https://github.com/inducer/pycuda.git
cd pycuda
python3 configure.py --cuda-enable-gl --no-use-shipped-boost
sudo python3 setup.py install

git clone --branch no_opengl https://github.com/koszullab/instagraal.git
cd instagraal
sudo python3 setup.py install
# reboot

So far it's running, but it looks like it will take a loooooooong time (I'm at "0.005421464662893327% proceeded", after 3 hours...). Is the version with opengl support much faster? I.e., is it worth to get it to run?

Best regards

Hello,

I am trying to run instaGRAAL on a linux machine using docker with this command:

sudo docker run --gpus all koszullab/instagraal instagraal $OUT_HICSTUFF $ASSEMBLY $OUT_INSTA

It gave this error:

Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal')()
File "/src/instagraal/instagraal/instagraal.py", line 1178, in main
output_folder=output_folder,
File "/src/instagraal/instagraal/instagraal.py", line 160, in init
output_folder=output_folder,
File "/src/instagraal/instagraal/simu_single.py", line 74, in init
self.select_data_set(name)
File "/src/instagraal/instagraal/simu_single.py", line 743, in select_data_set
thresh_factor=self.thresh_factor,
File "/src/instagraal/instagraal/pyramid_sparse.py", line 58, in build_and_filter
os.mkdir(all_pyramid_folder)
FileNotFoundError: [Errno 2] No such file or directory: '/xxx/pyramids'

I found this other issue #24 which seems similar, but with singularity, and attempted to reproduce the proposed solution in docker using "-v $PWD:/data" (instead of -B $PWD:/data), but obtained the same error.

The HiC matrix is produced from Hicstuff and the assembly is a fasta file (from wtdbg2).

Thanks in advance for any help and sorry if I didn't include sufficient information.
Best regards

Hello, we find some discrepancies between our final polished genome assembly and cytogenetic map.
The main problem is that 3 chromosomes and several other genomic blocks (belonging to other chromosomes) are merged into one very large scaffold.
I tried several combinations of hicstuff parameters (different mapping software: bowtie2, bwa, minimap2; different mapping options: “iterative” or “cutsite”; the “duplicates” option to remove PCR artefacts), and finally run Instagraal using “-l 5” or “-l 5 -c 0” parameters. However, I always find this super scaffold.
Do you have any idea of what I could try to be more “stringent” and avoid that different chromosomes are merged together?
Thank you in advance!

Dear all,
I'am sysadmin and I installed for one of my user the 'instagraal' program successfully but we get an error on the runtime. I already tried to run with CUDA/9.2.28 without success and now with the CUDA/10.0.130 and kernel update, it's also failed (on the runtime)..

Here is my error:
instagraal graal_scaffolding_SRR3105481/ sim_c500_GCF_000002985.6_WBcel235_genomic.mixed.fa graal_14_n100_c1_N5 --debug
INFO :: Selected_device: Tesla V100-PCIE-16GB
Traceback (most recent call last):
File "/opt/cecisw/arch/easybuild/2018b/software/Python/3.6.6-intel-2018b/bin/instagraal", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal')()
File "/opt/sw/src/instagraal/instagraal/instagraal.py", line 2161, in main
output_folder=output_folder,
File "/opt/sw/src/instagraal/instagraal/instagraal.py", line 208, in init
self.cuda_gl_init()
File "/opt/sw/src/instagraal/instagraal/instagraal.py", line 1448, in cuda_gl_init
curr_gpu, flags=cudagl.graphics_map_flags.NONE
pycuda._driver.Error: cuGLCtxCreate failed: unknown error

The pycuda versionis 2019.01 and GCC 7.3.0.

I have already lost a lot of time to build serveral version with success but all of my versions failed on the runtine with the same error 'pycuda._driver.Error: cuGLCtxCreate failed: unknown error'

Any idea ?

Thanks in advance,

Best Regards,
Sébastien.

Hi there,
Thank you for developing the instaGRAAL and your support. I have polished my genome assembly using instagraal-polish. I am a bit confused about understanding the meaning of -1 in the new_info_frags.txt file. Particularly, there is a contig (scaffold) that was aligned to mitochondrial DNA (I guess it is a mitochondrial genome) but the length of this contig is 32 kb while we expect the size of 16 kb for mitochondrial genome:

3C-assembly|contig_80
init_contig id_frag orientation start end
ptg000178l 22986 1 0 16509
ptg000178l -1 1 16509 32593

I wonder if you can provide more details about the structure of info_frags.txt. You mentioned "Orientations are relative to one another, and when "-1" is supplied, it is understood that the reverse complement should be taken", but it is not still clear for me what do you mean. I would be appreciated if you can provide more details or example.

When installing instagraal v 0.1.2 on Ubuntu 18 through pip, the folder kernels is missing from the package. This results in the following error:

Traceback (most recent call last):
  File "/home/cmatthey/.local/bin/instagraal", line 11, in <module>
    sys.exit(main())
  File "/home/cmatthey/.local/lib/python3.6/site-packages/instagraal/instagraal.py", line 2108, in main
    output_folder=output_folder,
  File "/home/cmatthey/.local/lib/python3.6/site-packages/instagraal/instagraal.py", line 174, in __init__
    output_folder=output_folder,
  File "/home/cmatthey/.local/lib/python3.6/site-packages/instagraal/simu_single.py", line 174, in __init__
    self.gl_size_im,
  File "/home/cmatthey/.local/lib/python3.6/site-packages/instagraal/cuda_lib_gl_single.py", line 297, in __init__
    self.loadProgram(kernel_adapt_entry_point)
  File "/home/cmatthey/.local/lib/python3.6/site-packages/instagraal/cuda_lib_gl_single.py", line 1957, in loadProgram
    f = open(filename, "r")
FileNotFoundError: [Errno 2] No such file or directory: '/home/cmatthey/.local/lib/python3.6/site-packages/instagraal/kernels/kernel_sparse_adapt.cu'

Output of
ls -1 $HOME/.local/lib/python3.6/site-packages/instagraal/
cuda_lib_gl_single.py
fragment.py
glutil.py
gpustruct.py
init_nuisance.py
init.py
instagraal.py
leastsqbound.py
log.py
optim_rippe_curve_update.py
parse_info_frags.py
pycache
pyramid_sparse.py
simu_single.py
vector.py
version.py

I followed the instructions' requirements in order to install instaGRAAL; however despite, from getting the nvidia-cuda-toolkit to getting the requirements.txt sorted out and finally the pycuda installed, I haven't experienced any issue at the moment I typed instaGRAAL on my BASH terminal I'm prompted with this message:

Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 5, in
from instagraal.instagraal import main
File "/usr/local/lib/python3.9/dist-packages/instagraal/instagraal.py", line 57, in
import pycuda.gl as cudagl
File "/usr/local/lib/python3.9/dist-packages/pycuda-2021.1-py3.9-linux-x86_64.egg/pycuda/gl/init.py", line 5, in
raise ImportError("PyCUDA was compiled without GL extension support")
ImportError: PyCUDA was compiled without GL extension support

What does it mean? After I installed instaGRAAL it seems there is a problem with the GL extension that I cannot attribute to anything I can think of. Let me know.

After overcoming the #17 issue, the pyramid files were generated successfully but then failed at a nvcc compilation step.

After this:

2020-11-22 22:08:36,640 :: INFO :: loading kernels ...
2020-11-22 22:08:36,642 :: INFO :: size array in shared memory = 1536

(Full log here: instagraal.log)

Error returns:

Traceback (most recent call last):
  File "/usr/local/bin/instagraal", line 11, in <module>
    load_entry_point('instagraal==0.1.6', 'console_scripts', 'instagraal')()
  File "/usr/local/lib/python3.7/dist-packages/instagraal-0.1.6-py3.7.egg/instagraal/instagraal.py", line 1178, in main
    output_folder=output_folder,
  File "/usr/local/lib/python3.7/dist-packages/instagraal-0.1.6-py3.7.egg/instagraal/instagraal.py", line 160, in __init__
    output_folder=output_folder,
  File "/usr/local/lib/python3.7/dist-packages/instagraal-0.1.6-py3.7.egg/instagraal/simu_single.py", line 168, in __init__
    self.pos,
  File "/usr/local/lib/python3.7/dist-packages/instagraal-0.1.6-py3.7.egg/instagraal/cuda_lib_gl_single.py", line 294, in __init__
    self.loadProgram(kernel_adapt_entry_point)
  File "/usr/local/lib/python3.7/dist-packages/instagraal-0.1.6-py3.7.egg/instagraal/cuda_lib_gl_single.py", line 1979, in loadProgram
    options=[
  File "/usr/local/lib/python3.7/dist-packages/pycuda-2020.1-py3.7-linux-x86_64.egg/pycuda/compiler.py", line 358, in __init__
    include_dirs,
  File "/usr/local/lib/python3.7/dist-packages/pycuda-2020.1-py3.7-linux-x86_64.egg/pycuda/compiler.py", line 298, in compile
    return compile_plain(source, options, keep, nvcc, cache_dir, target)
  File "/usr/local/lib/python3.7/dist-packages/pycuda-2020.1-py3.7-linux-x86_64.egg/pycuda/compiler.py", line 155, in compile_plain
    stderr=stderr.decode("utf-8", "replace"),
pycuda.driver.CompileError: nvcc compilation of /tmp/tmp7tvm61k6/kernel.cu failed
[command: nvcc --cubin -arch sm_75 -I/usr/local/lib/python3.7/dist-packages/pycuda-2020.1-py3.7-linux-x86_64.egg/pycuda/cuda kernel.cu]
[stderr:
kernel.cu(432): warning: variable "selec_smem" was set but never used

kernel.cu(437): warning: variable "local_count" was set but never used

kernel.cu(505): warning: variable "pos_fa_down" was declared but never referenced

kernel.cu(521): warning: variable "id_alter_contig" was declared but never referenced

kernel.cu(521): warning: variable "condition_3" was declared but never referenced

kernel.cu(521): warning: variable "condition_4" was declared but never referenced

kernel.cu(740): warning: variable "sub_pos_f_pop" was set but never used

kernel.cu(744): warning: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(746): warning: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(751): warning: variable "start_bp_f_pop" was set but never used

kernel.cu(754): warning: variable "or_f_pop" was set but never used

kernel.cu(1083): warning: variable "contig_f_pop" was set but never used

kernel.cu(1084): warning: variable "pos_f_pop" was set but never used

kernel.cu(1085): warning: variable "sub_pos_f_pop" was set but never used

kernel.cu(1086): warning: variable "l_cont_f_pop" was set but never used

kernel.cu(1087): warning: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(1088): warning: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(1091): warning: variable "start_bp_f_pop" was set but never used

kernel.cu(1092): warning: variable "id_prev_f_pop" was set but never used

kernel.cu(1093): warning: variable "id_next_f_pop" was set but never used

kernel.cu(1375): warning: variable "contig_f_pop" was set but never used

kernel.cu(1376): warning: variable "pos_f_pop" was set but never used

kernel.cu(1377): warning: variable "sub_pos_f_pop" was set but never used

kernel.cu(1378): warning: variable "l_cont_f_pop" was set but never used

kernel.cu(1379): warning: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(1380): warning: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(1383): warning: variable "start_bp_f_pop" was set but never used

kernel.cu(1384): warning: variable "id_prev_f_pop" was set but never used

kernel.cu(1385): warning: variable "id_next_f_pop" was set but never used

kernel.cu(1690): warning: variable "contig_f_pop" was set but never used

kernel.cu(1691): warning: variable "pos_f_pop" was set but never used

kernel.cu(1692): warning: variable "sub_pos_f_pop" was set but never used

kernel.cu(1693): warning: variable "l_cont_f_pop" was set but never used

kernel.cu(1694): warning: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(1695): warning: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(1698): warning: variable "start_bp_f_pop" was set but never used

kernel.cu(1699): warning: variable "id_prev_f_pop" was set but never used

kernel.cu(1700): warning: variable "id_next_f_pop" was set but never used

kernel.cu(1713): warning: variable "id_prev_f_ins" was set but never used

kernel.cu(2141): warning: variable "pop_is_ext" was set but never used

kernel.cu(2421): warning: variable "or_f_cut_a" was set but never used

kernel.cu(2431): warning: variable "or_f_cut_b" was set but never used

kernel.cu(2749): warning: variable "id_prev_f_ins" was set but never used

kernel.cu(2991): warning: variable "or_f_cut" was set but never used

kernel.cu(3369): warning: variable "sub_pos_fA" was set but never used

kernel.cu(3373): warning: variable "len_bp_fA" was set but never used

kernel.cu(3374): warning: variable "sub_len_fA" was set but never used

kernel.cu(3375): warning: variable "start_bp_fA" was set but never used

kernel.cu(3376): warning: variable "id_prev_fA" was set but never used

kernel.cu(3377): warning: variable "id_next_fA" was set but never used

kernel.cu(3378): warning: variable "circ_fA" was set but never used

kernel.cu(3379): warning: variable "or_fA" was set but never used

kernel.cu(3384): warning: variable "sub_pos_fB" was set but never used

kernel.cu(3388): warning: variable "len_bp_fB" was set but never used

kernel.cu(3389): warning: variable "sub_len_fB" was set but never used

kernel.cu(3390): warning: variable "start_bp_fB" was set but never used

kernel.cu(3391): warning: variable "id_prev_fB" was set but never used

kernel.cu(3392): warning: variable "id_next_fB" was set but never used

kernel.cu(3393): warning: variable "circ_fB" was set but never used

kernel.cu(3394): warning: variable "or_fB" was set but never used

kernel.cu(3712): warning: variable "is_activ_fi" was set but never used

kernel.cu(3712): warning: variable "is_rep_fi" was set but never used

kernel.cu(3712): warning: variable "swap" was declared but never referenced

kernel.cu(3714): warning: variable "s" was declared but never referenced

kernel.cu(3716): warning: variable "is_circle" was set but never used

kernel.cu(3775): warning: variable "is_activ_fi" was set but never used

kernel.cu(3775): warning: variable "is_rep_fi" was set but never used

kernel.cu(3775): warning: variable "swap" was declared but never referenced

kernel.cu(3777): warning: variable "s" was declared but never referenced

kernel.cu(3823): error: identifier "__shfl_down" is undefined

kernel.cu(3936): warning: variable "tmp_val" was declared but never referenced

kernel.cu(4010): warning: variable "id" was declared but never referenced

kernel.cu(4056): warning: variable "start" was declared but never referenced

kernel.cu(4398): warning: variable "row" was declared but never referenced

kernel.cu(4398): warning: variable "col" was declared but never referenced

kernel.cu(4399): warning: variable "is_circle" was declared but never referenced

kernel.cu(4636): warning: variable "max_len" was set but never used

kernel.cu(4644): warning: variable "min_id_c_new" was declared but never referenced

kernel.cu(4876): warning: variable "condition_pix" was set but never used

kernel.cu(4877): warning: variable "start" was declared but never referenced

1 error detected in the compilation of "kernel.cu".
]

I compiled the linux-source-4.19 for kernel and CUDA-11.1. I tried reinstalling everything but didn't seem to troubleshoot the problem.

Thank you very much for your help.

Dear all,

We observed the following error message while running instaGRAAL with the --simple flag.
Are we dealing with an installation or a data problem?

Thanks in advance for your help!
hugo

INFO :: mean size kb = 6.4507546267087275
INFO :: min fragment length = 0.073
INFO :: estimation of the parameters of the model
WARNING :: /usr/local/lib/python3.6/dist-packages/instagraal/optim_rippe_curve_update.py:47: RuntimeWarning: invalid value encountered in log

• (d - 2) / ((np.power((lm * x / kuhn), 2) + d))

INFO :: p from estimate parameters = [1.1178771257173667, 18.131529792453716, -1.0325689603584807, 2, 11697.042257632504]
INFO :: mean value trans = 0.03493796078866897
INFO :: BEWARE!!! : I will lower mean value trans !!!
INFO :: estimate max dist cis trans = 50269.59680359397
(array([ 1.11787713e+00, 1.81315298e+01, -1.03256896e+00, 1.16970423e+04]), 1)
Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal==0.1.6', 'console_scripts', 'instagraal')()
File "/usr/local/lib/python3.6/dist-packages/instagraal/instagraal.py", line 1193, in main
p2.simple_start(
AttributeError: 'instagraal_class' object has no attribute 'simple_start'`

Hi,
I have used the docker image through singualrity on a HPC GPU node but I got the following issue:
The command is:
singularity run -B $PWD:/data --nv -C instagraal_latest.sif instagraal /data/hicstuff_out /data/assembly.pilon.fasta /data/instagraal_out

and here is the message:
INFO :: start filtering
INFO :: nfrags = 13398071
INFO :: n init frags = 13398071
INFO :: mean sparsity = 7.225973490676552e-07
INFO :: median sparsity = 3.731880440227542e-07
INFO :: std sparsity = 1.2747863138429238e-06
INFO :: max_sparsity = 0.0006414356175810099
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 2593, in print_pdf
self.figure.draw(renderer)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py", line 41, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/figure.py", line 1864, in draw
renderer, self, artists, self.suppressComposite)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/image.py", line 131, in _draw_list_compositing_images
a.draw(renderer)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py", line 41, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/cbook/deprecation.py", line 411, in wrapper
return func(*inner_args, **inner_kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/axes/_base.py", line 2747, in draw
mimage._draw_list_compositing_images(renderer, self, artists)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/image.py", line 131, in _draw_list_compositing_images
a.draw(renderer)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py", line 41, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/collections.py", line 931, in draw
Collection.draw(self, renderer)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py", line 41, in draw_wrapper
return draw(artist, renderer, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/collections.py", line 404, in draw
mpath.Path(offsets), transOffset, tuple(facecolors[0]))
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 1934, in draw_markers
marker, Op.use_xobject)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 732, in output
self.write(fill([pdfRepr(x) for x in data]))
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 729, in write
self.currentstream.write(data)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 524, in write
self.file.write(compressed)
OSError: [Errno 28] No space left on device

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal')()
File "/src/instagraal/instagraal/instagraal.py", line 1178, in main
output_folder=output_folder,
File "/src/instagraal/instagraal/instagraal.py", line 160, in init
output_folder=output_folder,
File "/src/instagraal/instagraal/simu_single.py", line 74, in init
self.select_data_set(name)
File "/src/instagraal/instagraal/simu_single.py", line 743, in select_data_set
thresh_factor=self.thresh_factor,
File "/src/instagraal/instagraal/pyramid_sparse.py", line 113, in build_and_filter
thresh_factor=thresh_factor,
File "/src/instagraal/instagraal/pyramid_sparse.py", line 856, in remove_problematic_fragments
plt.savefig("sparsity_plot.pdf")
File "/usr/local/lib/python3.6/dist-packages/matplotlib/pyplot.py", line 859, in savefig
res = fig.savefig(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/figure.py", line 2311, in savefig
self.canvas.print_figure(fname, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backend_bases.py", line 2217, in print_figure
**kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backend_bases.py", line 1639, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 2601, in print_pdf
file.close()
File "/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_pdf.py", line 723, in close
self.fh.close()
OSError: [Errno 28] No space left on device

Do you have any idea to solve this issue?
Best

Trying to run instagraal (version 0.1.2) yielded the following issue:

genome="Avaga.Masurca_A1.fasta"
hicstuff_indir="Adineta_vaga.Masurca_A1.Hi-C_reads"
outdir="Adineta_vaga.Masurca_A1.Hi-C_reads_GRAAL"
instagraal $hicstuff_indir $genome $outdir

X Error of failed request: BadValue (integer parameter out of range for operation)
Major opcode of failed request: 154 (GLX)
Minor opcode of failed request: 24 (X_GLXCreateNewContext)
Value in failed request: 0x0
Serial number of failed request: 37
Current serial number in output stream: 38

I do not know where to start with such an error output, so any help would be appreciated !
thanks,
paul

Another issue previously appeared, but I corrected it by installing "freeglut3-dev" :

OpenGL.error.NullFunctionError: Attempt to call an undefined function glutInit, check for bool(glutInit) before calling

sudo apt-get install freeglut3-dev

Of note:

ls $hicstuff_indir 

abs_fragments_contacts_weighted.txt fragments_list.txt hicstuff_20190307174358.log info_contigs.txt plots

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Fri_Feb__8_19:08:17_PST_2019
Cuda compilation tools, release 10.1, V10.1.105

lspci | grep -i nvidia

02:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1080] (rev a1)
02:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)

uname -m && cat /etc/*release | head -n2

x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.10

Hello,

When I try to run instagraal polish post-scaffolding
instagraal-polish -m polishing -i info_frags.txt -o curated.fasta -f instagraal.fasta -j NNNNNNNNNN

I get the following error:

Appending the rest...
Traceback (most recent call last):
File "/usr/local/bin/instagraal-polish", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal-polish')()
File "/pycuda/src/instagraal/instagraal/parse_info_frags.py", line 949, in main
output=output_file,
File "/pycuda/src/instagraal/instagraal/parse_info_frags.py", line 645, in write_fasta
seq_to_add = init_genome[init_contig][start:end]
KeyError: 'ctg000038'

I have tried this with several genomes so it seems to be not dataset dependent. I used the default info_frags.txt file from the instagraal output directory.

Thanks

Hello,
I have been working myself through installing instagraal on Fedora 33 and after several issues with cuda, pycuda and h5py I finally got to a stage where I can generate the pyramid files but now I get a new error that seems to be related to the sequence names in my input reference not being properly loaded into a python dictionary.
I am using a 760 Mb Genome assembled with NECAT and filtered with purge_duplicats. The contigs are named bctg00000000 up to bctg00001099 (not all numbers in between are used).
I did install instagraal via
sudo pip3 install -e git+https://github.com/koszullab/instagraal.git@master#egg=instagraal

When I run instagraal it seems to work fine till the stage where it is loading the reference. Then I get the following error

INFO :: pyramid loaded
INFO :: loading data from level = 4
INFO :: import reference genome
Traceback (most recent call last):
  File "/usr/local/bin/instagraal", line 33, in <module>
    sys.exit(load_entry_point('instagraal', 'console_scripts', 'instagraal')())
  File "/mnt/data/max/20201213_Vinca_instagraal/src/instagraal/instagraal/instagraal.py", line 2149, in main
    p2 = window(
  File "/mnt/data/max/20201213_Vinca_instagraal/src/instagraal/instagraal/instagraal.py", line 212, in __init__
    self.simulation = simulation(
  File "/mnt/data/max/20201213_Vinca_instagraal/src/instagraal/instagraal/simu_single.py", line 108, in __init__
    self.level.build_seq_per_bin(genome_fasta=self.fasta)
  File "/mnt/data/max/20201213_Vinca_instagraal/src/instagraal/instagraal/pyramid_sparse.py", line 2215, in build_seq_per_bin
    seq_frag = self.pyramid.dict_sequence_contigs[init_contig][
KeyError: 'bctg00000000'

Can you tell me what I am missing and how it could be fixed?

Thanks

Dear maintainers,
I encountered a problem with instagraal-polish when using the -j option which does not seem to work. Whatever the value given to the argument nothing is written at the junction of consecutive contigs in the obtained polished fasta file.
I am absolutely not an expert in python so I surely may have misunderstood most of the code but going through the parse_info_frags.py script I noticed the argument "junction" argument in the write_fasta function is defined as a boolean, set by default to FALSE (L604), but when it is called with the "polishing mode" (L877), the given argument seems to be a string (resulting from the parsing L853). In addition within the code of the write_fasta function, the junction argument is considered either as a boolean (L644) or a string (L647 when given to the Seq() function?)?
I think I found a trick to solve the problem for my own purpose by slightly modifying the code. It consist of making the -j argument ineffective by always using the DEFAULT_JUNCTION_SEQUENCE (defined L33) between consecutive contig sequences within a scaffold with the two following slight modifications to the parse_info_frags.py script:

L646-L647 of the current version:
extra_seq = Seq(junction, IUPAC.ambiguous_dna)
current_seq = current_seq + extra_seq
modified as follows:
#extra_seq = Seq(junction, IUPAC.ambiguous_dna)
current_seq = current_seq + DEFAULT_JUNCTION_SEQUENCE

L954 of the current version commented:
# junction=junction,

Is it correct?
Many thanks for you help (and your programs!)
All the best

Hi there,
I would like to use the instaGRAAL to scaffold my data. I have used the docker image through singualrity on a GPU node but I got the following issue:

singularity run --nv instagraal.sif hic_folder hifiasm_ccs.p_ctg.fa output_folder
Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal')()
File "/src/instagraal/instagraal/instagraal.py", line 1178, in main
output_folder=output_folder,
File "/src/instagraal/instagraal/instagraal.py", line 160, in init
output_folder=output_folder,
File "/src/instagraal/instagraal/simu_single.py", line 74, in init
self.select_data_set(name)
File "/src/instagraal/instagraal/simu_single.py", line 743, in select_data_set
thresh_factor=self.thresh_factor,
File "/src/instagraal/instagraal/pyramid_sparse.py", line 58, in build_and_filter
os.mkdir(all_pyramid_folder)
FileNotFoundError: [Errno 2] No such file or directory: 'hic_folder/pyramids'

Do you have any idea to solve this issue?
Best

Hello,
instagraal out_hicstuff_bowtie2 assembly.fa out_instagraal
instagraal generates the following warning

WARNING :: /usr/local/lib/python3.6/dist-packages/instagraal/pyramid_sparse.py:242: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
  pyramid_handle = h5py.File(hdf5_pyramid_file)

it runs anyway and produces an output.
Is it harmless?

when use my data to test the instagraal , it looks fine at first . later , it stopped , throwing error,

INFO :: start filtering
WARNING :: /usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/pyramid_sparse.py:103: H5pyDeprecationWarning: The default file mode will change to 'r' (read-only) in h5py 3.0. To suppress this warning, pass the mode you need to h5py.File(), or set the global default h5.get_config().default_file_mode, or set the environment variable H5PY_DEFAULT_READONLY=1. Available modes are: 'r', 'r+', 'w', 'w-'/'x', 'a'. See the docs for details.
pyramid_0 = h5py.File(init_pyramid_file)

Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal==0.1.6', 'console_scripts', 'instagraal')()
File "/usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/instagraal.py", line 1178, in main
output_folder=output_folder,
File "/usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/instagraal.py", line 160, in init
output_folder=output_folder,
File "/usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/simu_single.py", line 74, in init
self.select_data_set(name)
File "/usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/simu_single.py", line 743, in select_data_set
thresh_factor=self.thresh_factor,
File "/usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/pyramid_sparse.py", line 113, in build_and_filter
thresh_factor=thresh_factor,
File "/usr/local/lib/python3.6/dist-packages/instagraal-0.1.6-py3.6.egg/instagraal/pyramid_sparse.py", line 833, in remove_problematic_fragments
level = pyramid["0"]
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/group.py", line 264, in getitem
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object '0' doesn't exist)"

The article about instaGRAAL states that "instaGRAAL can optionally exploit available genetic linkage data". How can I do this and in what format should the genetic linkage data be supplied to instaGRAAL?

Hello,

I am trying to run the docker version of instagraal on an Ubuntu OS v. 18.10 linux machine.

the output of the command
$nvidia-smi
is the following:
Tue Mar 1 13:22:13 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.40 Driver Version: 430.40 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 1080 Off | 00000000:02:00.0 On | N/A |
| 36% 29C P8 8W / 180W | 38MiB / 8118MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 31500 G /usr/lib/xorg/Xorg 26MiB |
+-----------------------------------------------------------------------------+

#######################################################################
The command
$nvcc --version

gives the following output:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85
(base) ➜ Brachionus_calyciflorus nvidia-smi

#################################################
The command
$ docker run --gpus all koszullab/instagraal instagraal --version

gives the output:
0.1.2
######################################################################
So I tried to run it like this:
$ docker run --gpus all koszullab/instagraal instagraal $outdir $genome $outdir2

The output:
Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 11, in
load_entry_point('instagraal', 'console_scripts', 'instagraal')()
File "/src/instagraal/instagraal/instagraal.py", line 1178, in main
output_folder=output_folder,
File "/src/instagraal/instagraal/instagraal.py", line 160, in init
output_folder=output_folder,
File "/src/instagraal/instagraal/simu_single.py", line 74, in init
self.select_data_set(name)
File "/src/instagraal/instagraal/simu_single.py", line 743, in select_data_set
thresh_factor=self.thresh_factor,
File "/src/instagraal/instagraal/pyramid_sparse.py", line 58, in build_and_filter
os.mkdir(all_pyramid_folder)
FileNotFoundError: [Errno 2] No such file or directory: 'rejoined_polished.fasta.Hi-C_minimap2/pyramids'

Any help would be appreciated.
Thanks

Hi,

I am trying to run instaGRAAL using docker. However, a black window appears for 1 second before the program crashes without an error message. Here is the command that I ran:

sudo docker run  --net=host -e DISPLAY=$DISPLAY -v="/home/bistace/.Xauthority:/root/.Xauthority:rw" -v /datastore:/datastore koszullab/instagraal --debug /datastore/hicstuff /datastore/genome.fasta /datastore/instagraal"

I tried going into the container while running a /bin/bash interpreter to launch the program by myself inside the container but only get a segmentation fault at instaGRAAL startup.

The docker container is running on an Ubuntu 18.04 OS, with 64GB of RAM and a Geforce GTX 1070 with the recommended driver (nvidia-driver-440).

Could you please help me troubleshoot this issue?

Hi instaGRAAL developers, I'd like to seek for help. I'm using the no_opengl branch of instaGRAAL following the previous issue that I encountered. Now, instaGRAAL seems to work though the following error comes out after a short while. Does this have something to do with libboost? I checked that the system has libboost 1.71. Am not sure if this error occurs because of the Python headers appear to be absent. Thanks a lot for your help!

INFO :: kernels compiled
INFO :: setup jumping distribution: start
INFO :: Shape sub mat = (103382, 103382)
INFO :: setup jumping distribution: done
INFO :: recompiling for non-existent cache dir (/home/datb/.cache/codepy/codepy-compiler-cache-v5-py3.6.15.final.0/a94c4c01d73203ac29d06cb2931f5fda).
/home/datb/.cache/codepy/codepy-compiler-cache-v5-py3.6.15.final.0/a94c4c01d73203ac29d06cb2931f5fda/module.cpp:1:10: fatal error: boost/python.hpp: No such file or directory
1 | #include <boost/python.hpp>
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
FAILED compiler invocation:x86_64-conda-linux-gnu-c++ -pthread -Wno-unused-result -Wsign-compare -fwrapv -O2 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/datb/miniconda3/envs/instagraal/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/python_1638555655832/work=/usr/local/src/conda/python-3.6.15 -fdebug-prefix-map=/home/datb/miniconda3/envs/instagraal=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/datb/miniconda3/include -c -DNDEBUG -I/home/datb/miniconda3/envs/instagraal/include/python3.6m -I/home/datb/tools/cuda_toolkit/include /home/datb/.cache/codepy/codepy-compiler-cache-v5-py3.6.15.final.0/a94c4c01d73203ac29d06cb2931f5fda/module.cpp -o /home/datb/.cache/codepy/codepy-compiler-cache-v5-py3.6.15.final.0/a94c4c01d73203ac29d06cb2931f5fda/module.o
x86_64-conda-linux-gnu-c++ -pthread -Wno-unused-result -Wsign-compare -fwrapv -O2 -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/datb/miniconda3/envs/instagraal/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/python_1638555655832/work=/usr/local/src/conda/python-3.6.15 -fdebug-prefix-map=/home/datb/miniconda3/envs/instagraal=/usr/local/src/conda-prefix -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/datb/miniconda3/include -c -DNDEBUG -I/home/datb/miniconda3/envs/instagraal/include/python3.6m -I/home/datb/tools/cuda_toolkit/include /home/datb/.cache/codepy/codepy-compiler-cache-v5-py3.6.15.final.0/a94c4c01d73203ac29d06cb2931f5fda/module.cpp -o /home/datb/.cache/codepy/codepy-compiler-cache-v5-py3.6.15.final.0/a94c4c01d73203ac29d06cb2931f5fda/module.o
Traceback (most recent call last):
File "/home/datb/miniconda3/envs/instagraal/bin/instagraal", line 33, in
sys.exit(load_entry_point('instagraal==0.1.6', 'console_scripts', 'instagraal')())
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/instagraal-0.1.6-py3.6.egg/instagraal/instagraal.py", line 1178, in main
output_folder=output_folder,
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/instagraal-0.1.6-py3.6.egg/instagraal/instagraal.py", line 160, in init
output_folder=output_folder,
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/instagraal-0.1.6-py3.6.egg/instagraal/simu_single.py", line 168, in init
self.pos,
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/instagraal-0.1.6-py3.6.egg/instagraal/cuda_lib_gl_single.py", line 314, in init
self.setup_thrust_modules()
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/instagraal-0.1.6-py3.6.egg/instagraal/cuda_lib_gl_single.py", line 802, in setup_thrust_modules
gcc_toolchain, nvcc_toolchain, debug=True
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/codepy/cuda.py", line 83, in compile
object=True, **local_host_kwargs)
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/codepy/jit.py", line 430, in compile_from_string
toolchain.build_object(ext_file, source_paths, debug=debug)
File "/home/datb/miniconda3/envs/instagraal/lib/python3.6/site-packages/codepy/toolchain.py", line 192, in build_object
raise CompileError("module compilation failed")
codepy.CompileError: module compilation failed

Hi there,

I've been using instaGRAAL for a while and recently happen to upgrade my GPU. I then installed the most recent version of NVIDIA Drivers, see below

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.30.02              Driver Version: 530.30.02    CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Quadro RTX 5000                 On | 00000000:01:00.0 Off |                  N/A |
| N/A   53C    P8                5W / 110W|   1881MiB / 16384MiB |     10%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      5133      G   /usr/lib/xorg/Xorg                          793MiB |
|    0   N/A  N/A      6811      G   /usr/bin/gnome-shell                        120MiB |
|    0   N/A  N/A      7746      G   ...vice,SpareRendererForSitePerProcess      287MiB |
|    0   N/A  N/A      7851      G   ...,WinRetrieveSuggestionsOnlyOnDemand      164MiB |
|    0   N/A  N/A      8132      G   ...ures=SpareRendererForSitePerProcess      318MiB |
|    0   N/A  N/A   3421075      G   ...7741775,17899381394246000989,131072       89MiB |
+---------------------------------------------------------------------------------------+

and CUDA toolkit

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Mon_Oct_24_19:12:58_PDT_2022
Cuda compilation tools, release 12.0, V12.0.76
Build cuda_12.0.r12.0/compiler.31968024_0

Unfortunately, for some reason after pyramids build correctly and the tool starts loading them, I get prompted with the following error

Traceback (most recent call last):
File "/usr/local/bin/instagraal", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/instagraal/instagraal.py", line 2146, in main
p2 = window(
File "/usr/local/lib/python3.10/dist-packages/instagraal/instagraal.py", line 212, in init
self.simulation = simulation(
File "/usr/local/lib/python3.10/dist-packages/instagraal/simu_single.py", line 139, in init
self.sampler = sampler_lib(
File "/usr/local/lib/python3.10/dist-packages/instagraal/cuda_lib_gl_single.py", line 297, in init
self.loadProgram(kernel_adapt_entry_point)
File "/usr/local/lib/python3.10/dist-packages/instagraal/cuda_lib_gl_single.py", line 1981, in loadProgram
self.module = pycuda.compiler.SourceModule(
File "/usr/local/lib/python3.10/dist-packages/pycuda-2022.2.2-py3.10-linux-x86_64.egg/pycuda/compiler.py", line 355, in init
cubin = compile(
File "/usr/local/lib/python3.10/dist-packages/pycuda-2022.2.2-py3.10-linux-x86_64.egg/pycuda/compiler.py", line 304, in compile
return compile_plain(source, options, keep, nvcc, cache_dir, target)
File "/usr/local/lib/python3.10/dist-packages/pycuda-2022.2.2-py3.10-linux-x86_64.egg/pycuda/compiler.py", line 154, in compile_plain
raise CompileError(
pycuda.driver.CompileError: nvcc compilation of /tmp/tmp08zy1h7q/kernel.cu failed
[command: nvcc --cubin -arch sm_75 -I/usr/local/lib/python3.10/dist-packages/pycuda-2022.2.2-py3.10-linux-x86_64.egg/pycuda/cuda kernel.cu]
[stderr:
kernel.cu(39): error: texture is not a template

kernel.cu(438): warning #550-D: variable "selec_smem" was set but never used

Remark: The warnings can be suppressed with "-diag-suppress "

kernel.cu(443): warning #550-D: variable "local_count" was set but never used

kernel.cu(511): warning #177-D: variable "pos_fa_down" was declared but never referenced

kernel.cu(527): warning #177-D: variable "id_alter_contig" was declared but never referenced

kernel.cu(527): warning #177-D: variable "condition_3" was declared but never referenced

kernel.cu(527): warning #177-D: variable "condition_4" was declared but never referenced

kernel.cu(746): warning #550-D: variable "sub_pos_f_pop" was set but never used

kernel.cu(750): warning #550-D: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(752): warning #550-D: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(757): warning #550-D: variable "start_bp_f_pop" was set but never used

kernel.cu(760): warning #550-D: variable "or_f_pop" was set but never used

kernel.cu(1089): warning #550-D: variable "contig_f_pop" was set but never used

kernel.cu(1090): warning #550-D: variable "pos_f_pop" was set but never used

kernel.cu(1091): warning #550-D: variable "sub_pos_f_pop" was set but never used

kernel.cu(1092): warning #550-D: variable "l_cont_f_pop" was set but never used

kernel.cu(1093): warning #550-D: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(1094): warning #550-D: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(1097): warning #550-D: variable "start_bp_f_pop" was set but never used

kernel.cu(1098): warning #550-D: variable "id_prev_f_pop" was set but never used

kernel.cu(1099): warning #550-D: variable "id_next_f_pop" was set but never used

kernel.cu(1381): warning #550-D: variable "contig_f_pop" was set but never used

kernel.cu(1382): warning #550-D: variable "pos_f_pop" was set but never used

kernel.cu(1383): warning #550-D: variable "sub_pos_f_pop" was set but never used

kernel.cu(1384): warning #550-D: variable "l_cont_f_pop" was set but never used

kernel.cu(1385): warning #550-D: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(1386): warning #550-D: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(1389): warning #550-D: variable "start_bp_f_pop" was set but never used

kernel.cu(1390): warning #550-D: variable "id_prev_f_pop" was set but never used

kernel.cu(1391): warning #550-D: variable "id_next_f_pop" was set but never used

kernel.cu(1696): warning #550-D: variable "contig_f_pop" was set but never used

kernel.cu(1697): warning #550-D: variable "pos_f_pop" was set but never used

kernel.cu(1698): warning #550-D: variable "sub_pos_f_pop" was set but never used

kernel.cu(1699): warning #550-D: variable "l_cont_f_pop" was set but never used

kernel.cu(1700): warning #550-D: variable "l_cont_bp_f_pop" was set but never used

kernel.cu(1701): warning #550-D: variable "sub_l_cont_f_pop" was set but never used

kernel.cu(1704): warning #550-D: variable "start_bp_f_pop" was set but never used

kernel.cu(1705): warning #550-D: variable "id_prev_f_pop" was set but never used

kernel.cu(1706): warning #550-D: variable "id_next_f_pop" was set but never used

kernel.cu(1719): warning #550-D: variable "id_prev_f_ins" was set but never used

kernel.cu(2147): warning #550-D: variable "pop_is_ext" was set but never used

kernel.cu(2427): warning #550-D: variable "or_f_cut_a" was set but never used

kernel.cu(2437): warning #550-D: variable "or_f_cut_b" was set but never used

kernel.cu(2755): warning #550-D: variable "id_prev_f_ins" was set but never used

kernel.cu(2997): warning #550-D: variable "or_f_cut" was set but never used

kernel.cu(3375): warning #550-D: variable "sub_pos_fA" was set but never used

kernel.cu(3379): warning #550-D: variable "len_bp_fA" was set but never used

kernel.cu(3380): warning #550-D: variable "sub_len_fA" was set but never used

kernel.cu(3381): warning #550-D: variable "start_bp_fA" was set but never used

kernel.cu(3382): warning #550-D: variable "id_prev_fA" was set but never used

kernel.cu(3383): warning #550-D: variable "id_next_fA" was set but never used

kernel.cu(3384): warning #550-D: variable "circ_fA" was set but never used

kernel.cu(3385): warning #550-D: variable "or_fA" was set but never used

kernel.cu(3390): warning #550-D: variable "sub_pos_fB" was set but never used

kernel.cu(3394): warning #550-D: variable "len_bp_fB" was set but never used

kernel.cu(3395): warning #550-D: variable "sub_len_fB" was set but never used

kernel.cu(3396): warning #550-D: variable "start_bp_fB" was set but never used

kernel.cu(3397): warning #550-D: variable "id_prev_fB" was set but never used

kernel.cu(3398): warning #550-D: variable "id_next_fB" was set but never used

kernel.cu(3399): warning #550-D: variable "circ_fB" was set but never used

kernel.cu(3400): warning #550-D: variable "or_fB" was set but never used

kernel.cu(3742): error: identifier "int2float" is undefined

kernel.cu(3718): warning #550-D: variable "is_activ_fi" was set but never used

kernel.cu(3718): warning #550-D: variable "is_rep_fi" was set but never used

kernel.cu(3718): warning #177-D: variable "swap" was declared but never referenced

kernel.cu(3720): warning #177-D: variable "s" was declared but never referenced

kernel.cu(3722): warning #550-D: variable "is_circle" was set but never used

kernel.cu(3804): error: identifier "int2float" is undefined

kernel.cu(3781): warning #550-D: variable "is_activ_fi" was set but never used

kernel.cu(3781): warning #550-D: variable "is_rep_fi" was set but never used

kernel.cu(3781): warning #177-D: variable "swap" was declared but never referenced

kernel.cu(3783): warning #177-D: variable "s" was declared but never referenced

kernel.cu(3886): error: identifier "int2float" is undefined

kernel.cu(3959): error: identifier "int2float" is undefined

kernel.cu(3942): warning #177-D: variable "tmp_val" was declared but never referenced

kernel.cu(4016): warning #177-D: variable "id" was declared but never referenced

kernel.cu(4062): warning #177-D: variable "start" was declared but never referenced

kernel.cu(4150): error: identifier "int2float" is undefined

kernel.cu(4177): error: identifier "int2float" is undefined

kernel.cu(4290): error: identifier "int2float" is undefined

kernel.cu(4323): error: identifier "int2float" is undefined

kernel.cu(4431): error: identifier "int2float" is undefined

kernel.cu(4404): warning #177-D: variable "row" was declared but never referenced

kernel.cu(4404): warning #177-D: variable "col" was declared but never referenced

kernel.cu(4405): warning #177-D: variable "is_circle" was declared but never referenced

kernel.cu(4642): warning #550-D: variable "max_len" was set but never used

kernel.cu(4650): warning #177-D: variable "min_id_c_new" was declared but never referenced

kernel.cu(4685): error: identifier "int2float" is undefined

kernel.cu(4767): error: identifier "int2float" is undefined

kernel.cu(4810): error: identifier "int2float" is undefined

kernel.cu(4891): error: identifier "int2float" is undefined

kernel.cu(4886): warning #550-D: variable "condition_pix" was set but never used

kernel.cu(4887): warning #177-D: variable "start" was declared but never referenced

14 errors detected in the compilation of "kernel.cu".
]

which at first seemed to be related to kernel.cu but even after following the procedure in issue #18, the problem persists. So, at a first thought I believed something might have changed from CUDA toolkit 11 to 12 so that those lines in the kernel_sparse.cu and in the kernel_sparse_adapt.cu which should handle the changes in the v11 are no longer robust to the latest NVIDIA updates.
Alternatively, it could be an issue with the most recent version of pycuda, the problem is that I tried to go back to last year's version 2021.1 but I'm not allowed to, possibly because of the newer implementation of the NVIDIA Driver and toolkit.

Please, let me know whether there is a solution to this, thanks!

Hello,

When I try to run instaGRAAL locally, I get an error that the first pyramid file does not exist.
Here are my specs:
OS: Ubuntu 18.04
Cuda: Cuda 10.2
Driver: 440.33.11
OpenGL libraries, HDF5 serialization library, and boost libraries are all up to date.
Pycuda was successfully installed.
I've deleted pyramids folder every time I've re-run the command.

The specific error message:

INFO :: Selected_device: GeForce RTX 2080 Ti
Traceback (most recent call last):
  File "/usr/local/bin/instagraal", line 11, in <module>
    load_entry_point('instagraal', 'console_scripts', 'instagraal')()
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/instagraal.py", line 2164, in main
    output_folder=output_folder,
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/instagraal.py", line 223, in __init__
    output_folder=output_folder,
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/simu_single.py", line 76, in __init__
    self.select_data_set(name)
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/simu_single.py", line 747, in select_data_set
    thresh_factor=self.thresh_factor,
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/pyramid_sparse.py", line 64, in build_and_filter
    build(base_folder, init_size_pyramid, factor, min_bin_per_contig)
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/pyramid_sparse.py", line 242, in build
    pyramid_handle = h5py.File(hdf5_pyramid_file)
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 427, in __init__
    swmr=swmr)
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 190, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 96, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = 'spb_hic/pyramids/pyramid_1_no_thresh/pyramid.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
-------------------------------------------------------------------
PyCUDA ERROR: The context stack was not empty upon module cleanup.
-------------------------------------------------------------------
A context was still active when the context stack was being
cleaned up. At this point in our execution, CUDA may already
have been deinitialized, so there is no way we can finish
cleanly. The program will be aborted now.
Use Context.pop() to avoid this problem.
-------------------------------------------------------------------
Aborted ng(core dumped)
``ect_data_set(name)
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/simu_single.py", line 747, in select_data_set
    thresh_factor=self.thresh_factor,
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/pyramid_sparse.py", line 64, in build_and_filter
    build(base_folder, init_size_pyramid, factor, min_bin_per_contig)
  File "/home/blackmonlab/Desktop/instafolder/spb_hic/src/instagraal/instagraal/pyramid_sparse.py", line 242, in build
    pyramid_handle = h5py.File(hdf5_pyramid_file)
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 427, in __init__
    swmr=swmr)
  File "/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py", line 190, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 96, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = 'spb_hic/pyramids/pyramid_1_no_thresh/pyramid.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)
-------------------------------------------------------------------
PyCUDA ERROR: The context stack was not empty upon module cleanup.
-------------------------------------------------------------------
A context was still active when the context stack was being
cleaned up. At this point in our execution, CUDA may already
have been deinitialized, so there is no way we can finish
cleanly. The program will be aborted now.
Use Context.pop() to avoid this problem.
-------------------------------------------------------------------
Aborted ng(core dumped)

My instagraal.log file is only two lines:

2021-04-15 12:51:06,736 :: INFO :: Selected_device: GeForce RTX 2080 Ti
2021-04-15 12:51:09,460 :: INFO :: start filtering

Does anyone have any suggestions for fixing this? I've tried to run the docker version instead, but I get a different error (FileNotFoundError: [Errno 2] No such file or directory: 'spb_hic/pyramids') from that.

Hi,
I'd like to use instaGRAAL on a Jupyter Notebook. The installation is correct except for pycuda: the last command sudo python3 setup.py install doesn't work because I don't have admin rights. Is there any other way of completing the pycuda installation without admin rights?
Thanks,
E.

HI,
I am planning to install the cuda as described in the github on my ubuntu 18.04.4 , but it tells

The driver installation is unable to locate the kernel source,Please make sure that the kernel source packages are installed and set up correctly.
If you know that the kernel source packages are installed and set up correctly, you may pass the location of the kernel source with the '--kernel-source-path' flag.

so how can get it installed ?

Hi,
Can you provide some more accurate version numbers both as github tags and docker tags, please?

I can see

• 0.1 https://github.com/koszullab/instaGRAAL/releases (7 days old)
• 0.1.6 https://github.com/koszullab/instaGRAAL/blob/master/instagraal/version.py (13 mouths old)
• 0.1.2 from your docker lastest (7 days old)

singularity pull docker://koszullab/instagraal
./instagraal_latest.sif instagraal --version

I have to deplay instaGRAAL on our infrastructure but I don't know which version