Comments (7)
Thanks Balays. We'll take a look at the mmap-file option for a future GTDB-Tk release.
from gtdbtk.
Hello. Unfortunately, there is no way to reduce the system requirements of GTDB-Tk at the moment. We are hoping this software might be hosted as a web service in the future though there is no immediate plans in this regard. Sorry I can't be of direct help.
from gtdbtk.
Thanks for the reply @dparks1134 . We were just discussing your recent paper this morning in our lab seminar and hoped to be able to use the tool. I'd love to contribute to the software as it is open source but not being able to run it locally makes it hard to do so.
from gtdbtk.
The system requirements are pretty high. This is due to the third party software pplacer which we use to place genomes into a tree. This software is excellent and alternatives we have tried are worse in terms of system requirements so I don't expect this situation to get better.
from gtdbtk.
I see. Thanks.
from gtdbtk.
Hello.
I've had a similar problem, as our system with 96 GB RAM was not enough. Indeed the pplacer part killed the program. So I ran the pplacer sub-command only, with a --pretend flag, which only estimates the memory usage. It said that it needs 102 GB, so only just a little more. Then we increased our swap memory and it worked just fine. Actually it used about 96 (physical) + 25 GB (swap) RAM.
Thanks for this excellent tool btw, it's just what I was looking for! :)
from gtdbtk.
Also, at the pplacer's they know the large memory req. is an issues, so there's this option to get around it: --mmap-file, which creates a file that it uses as address space and thus shrinking the need for physical memory. I've integrated this flag into the GTDBTk classify.py code where it actually runs pplacer (line 101), but unfortunately it didn't work for me. But maybe it just needs a little tweaking. Cheers.
from gtdbtk.
Related Issues (20)
- relative evolutionary divergence (RED) values and median RED values HOT 3
- pydantic.dev/2.6/v/missing HOT 1
- de_novo_wf fails to parse gtdb classification file HOT 1
- Specify in documentation that the `--genes` parameter requires proteins as input.
- Small (mock) reference data HOT 3
- Attempting to add duplicate row HOT 4
- Decorate command issue HOT 1
- Using gtdb_to_ncbi_majority_vote.py script to convert gtbd taxonomy to SILVA taxonomy HOT 2
- classify_wf errors out when all inputs are unclassified
- Getting the GTDB version used from GTDB-Tk HOT 2
- def TypedDict(typename, fields=_marker,/, *, total=True, closed=False, **kwargs): SyntaxError: invalid syntax HOT 3
- Total genome counter in log is off (warning message) HOT 1
- Inquiry on the error "libgsl.so.25" HOT 1
- GTDB-tk has given confliction results for two genomes using different command HOT 2
- The GTDB-Tk reference data does not exist or is corrupted. HOT 4
- Is the lasted GTDB database version220 missing files?
- Is the lasted GTDB database version220 missing files? HOT 4
- Do I need a network to run classify_wf HOT 1
- Error from skani HOT 2
- No sequence results for archaea in the align folder of classify_wf HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gtdbtk.