Software implementations of `Bloom Filters` about serratus HOT 7 CLOSED

The implementation may be different for different software. The end product will be incorporating the Bloom Filter into our containers to filter the data in a pipe.

• BIGSI is one example. Although it's backwards to what we're hoping to do. In this case there already is a container available so we would only have to modify it to accept data from a stream and/or incorporate it into the serratus-dl containers.

I honestly am not well versed on all the options available and it needs to be researched and then we need to compare the speed/efficiency of each Bloom Filter to ensure it doesn't lag the whole pipeline. Therefore we can break this task down into two objectives

1. Curate a list of available bloom filter software. Include licenses, is it command line, language it's written in, website, academic paper reference if available.

2. Build each software package in a container and test how long it takes to process a benchmark set of RNA-sequencing libraries against a small test filter.

3. If needed, can we alter the program to accept RNA-sequencing reads (in fastq) format from a data-pipe.

See Also: Wikipedia

from serratus.

@emreerhan we could use your sage advice on this issue when you get a chance.

from serratus.

@jefftaylor42 @ababaian

1. What exactly do you mean by software implementation? A library? A standalone program? An AWS functionality?
2. In which programming language should it be?

from serratus.

@jefftaylor42 @ababaian I suggest adding tag Bioinformatics to this issue.

from serratus.

My first thought is the bloom filter library developed by the Birol lab and used in BioBloom tools: https://github.com/bcgsc/biobloom

License: GPL3 licence (free for academic use; not sure if this is an issue)
Language: C++; also has command line interface
doi: 10.1093/bioinformatics/btu558

from serratus.

BIGSI has an incredibly easy to use bloom filter class. https://github.com/Phelimb/BIGSI/blob/master/bigsi/bloom/bloomfilter.py

License: MIT
Language: Python
doi: http://dx.doi.org/10.1038/s41587-018-0010-1

from serratus.

We'll tentatively shut down Bloom Filters until there is a good reason to re-visit them.

from serratus.