positiveblue / libdori Goto Github PK

There are unused/useless imports in the LogLog.hpp file.

Remove them.

I do not know yet why Travis CI is failing to build the project but it should not. I should take a look to it as soon as possible.

Given that I will fix the Travis builds I could try to compile the project with gcc AND clang to make sure all works perfectly (I am doing it now only with clang).

Compiling the library throws some warnings. Basically

delete called on 'XXX' that is abstract but has non-virtual destructor [-Wdelete-non-virtual-dtor]

control reaches end of non-void function [-Wreturn-type]

private field 'XXX' is not used [-Wunused-private-field]

field 'XXX' will be initialized after field '_isGrowing' [-Wreorder]

The code needs to be fixed.

There are three basic cmd tools right now:

• cardinality
• frequency
• sample

Three use an arg parser for arguments, but they do not explain anywhere.

Taking advantage of the wiki pages of GitHub could be interesting for the project.

• There are many people who know nothing about the algorithms. I could point to the right place to learn more about them.

• A section explaining how the code is organized would be very useful for my future yo or other contributors.

The library is now getting bigger so the test are becoming more important.

They have to:

• Cover all the main features of it.
• Build by default and test the installation/compilation with CMake
• Remove dependencies with other files (like D1.txt, etc..)

I want people using my software, and often their first introduction will be through the README in the source code or on the project’s GitHub page.

I should not be lazy, the project needs a great README.

Implement the Recordinality algorithm.

Data Streams can be studied as random permutations. That fact allows a wealth of classical and recent results from combinatorics to be recycled as estimators for various statistics over data streams.

Recordinality estimates the number of distinct elements in a stream by counting the number of K-records occurring in it.

Implement a basic version (extensible) as cardinality estimator.

Robert Sedgewick from Princeton presented a new algorithm for cardinality estimation at AofA '16.

It is inspired in HyperLogLog, but it reduces de memory footprint (even more!).

Would be great (and easy) implement it and use it as default in libDori.

For more information, you can find the slides of the presentation here

Now that the library is growing would be interesting having a web page generated with sphinx describing the API.

The code could be commented a bit more.

HyperLogLog is the most used algorithm in that domain. Paper

Not for anytime soon (I am very busy working on other stuff) but would be great have some basic python bindings for libDori.

• CAPI or cffi would be a fast and straightforward way to write them.