encounter the memory allocation problem about sviper HOT 3 OPEN

Hi @shuliliu,

thanks for your interest in SViper!
So at a first glance this looks like a memory issue as you already mentioned.
Multi-threading increases the memory consumption but I see that you already only use one thread.

What is your available main memory (RAM)?
What is your short read coverage?

If possible, try running your script on a machine with more main memory and see if it also crashes.
If the error persists it might also be a bug, but than I would need to debug SViper with your data. Is there a chance you can make them available to me?

Best,
Svenja

from sviper.

Hi Svenja ,

Thank you for your reply. I used 4 coverage short reads (for testing) and 400G memory, and didn't work.
Yesterday, I tried the long read bam file generated by pbmm2 instead of minimap2, and it worked. So the issue seems located in the long read bam file.

1. I checked these two bam file in detail and only saw two big differences:
   bam files for pbmm2:
   cigar string: X/= cigars instead of M
   bam files for minimap2:
   cigar string: M instead of X/= cigars.
   MD string: I added a MD string when running minimap2.

2. I also checked the bam file generated from CLR reads and CCS reads using (minimap2). and saw one difference:
   bam files from CLR reads: base quality is missing (Originally missing in fastq file); bam file from CCS reads: with base quality.

I don't know whether above would be the things for the issue. I'll send you the data later if you think it still need to check more.

Best,

Shuli

from sviper.

Hi Shuli,

thanks for looking into it deeper. Yes your memory is sufficient big time, it looks like a bug.

To 1. I'm pretty sure that I handled X/= euqallly to M. And anyway I would suspect the error when handling X/=.
To 2. This seems more promising. I took a quick look into the code and I did not explicitly handle missing base qualities so I need to dig deeper to check if there is a problem.

It has been some time since I have touched the code though, so I would appreciate to get the data because debugging is easier that way. If you know the region where the error happens I am happy to only take a subregion of the data, too.

Best,
Svenja

from sviper.