Large memory request about dysgu HOT 8 CLOSED

Hi Matt,

I have added a fix which I hope should resolve the issue. You can test by building from source via git clone --recursive https://github.com/kcleal/dysgu.git; cd dysgu; bash INSTALL.sh.
As of v1.2.8 there is also better support for specifying regions of interest/exclude regions. For example you can use a bed file with target chromosomes using --search target_chroms.bed

from dysgu.

v1.2.8 should be on pypi now

from dysgu.

Just following up, it did finish this time and without the major memory issue, thanks!

from dysgu.

Hello Matt,

Thanks for reporting this. It looks as though a non-canonical chromosome is causing a problem - the target id (tid) of the problem chromsome appears to be 315. It looks like a large dispersed cluster was formed by reads on that chromosome. I will sort out a fix for this to stop this happening.

from dysgu.

Ah, that would make sense. Is there a way for me to limit the chromosomes that are actually tested? I could use that as a workaround, and to verify that it's the issue as well.

from dysgu.

Hmm, I got an error installing this way (previously, I believe I just used conda and pip). It seemed to make it through all the htslib stuff and finish the dysgu dependencies before throwing the error. Relevant snippet is below:

Using /cluster/home/jholt/githubDL/miniconda3/envs/dysgu/lib/python3.9/site-packages
Finished processing dependencies for dysgu==1.2.8
Traceback (most recent call last):
  File "/cluster/home/jholt/githubDL/miniconda3/envs/dysgu/bin/dysgu", line 33, in <module>
    sys.exit(load_entry_point('dysgu==1.2.8', 'console_scripts', 'dysgu')())
  File "/cluster/home/jholt/githubDL/miniconda3/envs/dysgu/bin/dysgu", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/cluster/home/jholt/githubDL/miniconda3/envs/dysgu/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/cluster/home/jholt/githubDL/miniconda3/envs/dysgu/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/cluster/home/jholt/githubDL/dysgu/dysgu/main.py", line 11, in <module>
    from dysgu import cluster, view, sv2bam
  File "/cluster/home/jholt/githubDL/dysgu/dysgu/view.py", line 11, in <module>
    from dysgu import io_funcs, cluster
ImportError: cannot import name 'io_funcs' from 'dysgu' (/cluster/home/jholt/githubDL/dysgu/dysgu/__init__.py)

I see the io_funcs.cpp and io_funcs.pyx in the dysgu/dysgu folder, so I'm not sure what the issue is. Suggestions on resolving that?

from dysgu.

Import errors can be a pain to debug in my experience. Are you using an env? The paths /cluster/home/jholt/githubDL/miniconda3/envs/dysgu/bin/dysgu and /cluster/home/jholt/githubDL/dysgu/dysgu/__init__.py look a bit different. I will upload the patch to pypi for installing with pip, should be available tomorrow

from dysgu.

Yea, I'm using a conda env and following your install from GitHub instructions. I wouldn't be surprised if there's a mismatch there. I'll just wait until it's on pypi and try it that way.

from dysgu.