Git Product home page Git Product logo

streamhist's Introduction

Carson Farmer ๐Ÿ‘‹

(he/him/his)

I do research and development with @textileio and @tablelandnetwork. I'm also a former university professor turned distributed web and open source advocate. I enjoy working on advanced algorithms and peer to peer protocols. I'm interested in conflict free replicated data types, linked data, composability, network resilience, maps, and more. You can email me at [email protected] or reach out to @carsonfarmer pretty much everywhere else.

Currently open to advisory roles for non-profits and smaller startups.

streamhist's People

Contributors

belm0 avatar carsonfarmer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

streamhist's Issues

No module named 'histogram'

Hi,
I found a problem when I was using it.

>>> from streamhist import StreamHist
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/streamhist/__init__.py", line 9, in <module>
    from histogram import StreamHist
ModuleNotFoundError: No module named 'histogram'

I tried to install ,But I got No matching distribution found for histogram.
I found this package on pypi(histogram), but I can't download it, returned 404.

How to solve this problem?

Thank you!

sum() is not monotonically increasing

Hey Carson!

Unfortunately, I noticed that the current version of your library has some serious bug in the sum() function. Running your examples leads (soon or later) to the following result:

hist1

I've dug a little bit into it and, most likely, this is the commit that causes it: a937507

I know that @belm0 wrote it to fix the function and quantiles, but it looks like tests do not cover everything needed.

On the other hand, using an earlier version, e.g. 7dc5169, leads to:

u = (bin_i.value + (bin_i1.value - bin_i.value) * z) -> TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

in _compute_quantile(). So it's something that @belm0 probably fixed, but it turns out that there was a price for it ;) I know that you don't actively support the repository, but I think it's important for everyone to be aware of the problem.

git tag for pypi release

would you make a git tag corresponding to the pypi release?

probably:

git tag 0.1.0 28fac98
git push --tags

incorrect quantiles() output

quantiles output can be far out of the histogram input range:

>>> h = streamhist.StreamHist()
>>> h.update((5, 7, 70))
>>> h.quantiles(.5, .9, .95)
[-24.5, 70.0, 70.0]

Alternative implementation

Hello,
Since it seems that this repository is not maintained anymore, and I need such a library, I made another implementation. It is available on pypi and is also hosted on github:
https://github.com/maki-nage/distogram

This may interest other people looking for a python implementation of this algorithm.

quantiles() nondeterministic exception

Unhandled exception seen in tests and real world use:

======================================================================
ERROR: streamhist.test.test_histogram.test_median_mean
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/.../python3.7/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "streamhist/streamhist/test/test_histogram.py", line 97, in test_median_mean
    assert about(h.median(), 0, 0.05)
  File "streamhist/streamhist/histogram.py", line 164, in median
    return self.quantiles(0.5)[0]
  File "streamhist/streamhist/histogram.py", line 407, in quantiles
    qq = _compute_quantile(target_sum, bin_i, bin_i1, prev_sum+1)
  File "streamhist/streamhist/histogram.py", line 462, in _compute_quantile
    u = (bin_i.value + (bin_i1.value - bin_i.value) * z)
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

Weighted Histogram

Can I add weighted data such as tuple of (value, weight) to the histogram?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.