The processes for splitting files are running very slowly with large numbers of input samples about rnasplice HOT 5 CLOSED

Hi @amizeranschi

I've opened a PR with the hot fix, please have a look and test on your samples.

from rnasplice.

Hey @amizeranschi

I think I have a solution for you and it should be fairly easy to implement. The suppa_split_file.R script contains a function which selects columns from the count matrix for each condition, however for some strange reason it reads in the count matrix within the function. Therefore, when we iterate over the conditions and call the function it reads in the count matrix again and again. If I move this line out of the function it should speed up dramatically. I'll have a hot fix for you to try tomorrow.

from rnasplice.

Hi @jma1991

Yes, that R script seems to be the bottleneck. Please let me know when you'll have the commit ready and I'll be happy to test it over the week-end.

from rnasplice.

Hi @jma1991

I'm happy to report that your hot fix solved this issue and that 1700 TPM files were generated in less than an hour, on the infrastructure I've described above.

Are you guys planning to release v.1.0.3 anytime soon? We're going to use these results in a publication and it would be nice to have a proper release version to reference.

from rnasplice.

Glad to hear it is working - I'll release v1.0.3 this week so you should have a proper release to reference. Keep an eye out for the release announcement.

from rnasplice.