Comments (11)
from streamz.
I am not familiar with pluck, what is it?
from streamz.
part of toolz
lib:
http://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.pluck
you can map
s = Stream()
s_first = s.map(partial(pluck, [0]))
s_second = s.map(partial(pluck, [1]))
s.emit([1,2])
# ...
(I knew of pluck but not use case, wondering if this is the line of thinking)
from streamz.
Cool. I think this would work.
from streamz.
Sorry, I assumed that pluck was already part of the API. It seems like a decently common use case and so possibly merits being added as syntactic sugar.
from streamz.
I don't understand the API. I don't know ahead of time how many streams might have been zipped. I need to take these zipped streams and zip more streams, but zipping zipped streams doesn't work for what I'm trying to do. I want something like:
streams_uzip = streamz.unzip(streams_zip)
streams_zip_out = streamz.zip(*streams_uzip, another_stream)
return streams_zip_out
from streamz.
Why not provide the constituent streams to create streamz_zip_out
? At some point they were all independent. You could also write a function that unpacks the zipped stream so that all the elements are at the same level in the hierarchy. If that was done right after the zipping you could be confident in how many levels you need to go down.
from streamz.
I don't have the constituent streams. It's a dynamic framework. I know that some zipped streams are passed into a function. I don't know how many have been zipped.
Yes, I need to delve into the API to understand how to unpack the zipped stream. That's what an unzip would be. Can you give an example of this function?
def unzip(zipped_stream):
if not is_zipped_stream(zipped_stream):
raise Exception('Not a zipped stream')
.
.
.
return streams_uzip
Basically that's what I need. I don't have any information about how the zipped_stream was formed, just that it's a zip of streams.
A bonus would be some ability to check that this is a zipped stream i.e. how do I achieve is_zipped_stream
.
from streamz.
The problem is that it may be difficult to determine how many levels of tuple hierarchy you need to unpack. Was an inner tuple the product of a zip or were the original data elements a tuple.
from streamz.
Each stream is a tuple or array of numbers. Say I have a stream zip of two streams that in a sink would appear as:
(array1, array2)
I want to zip it to another stream so that in a sink it would be:
(array1, array2, array3)
Without ability to unzip and re-zip it appears something like this:
((array1, array2), array3)
Now it could be more than two streams and I might need to zip up more streams. Sure, if I had all the original streams I could formulate this:
# list of length N stream_list with all original streams
stream_zip = streamz.zip(*stream_list)
# this stream_zip would appear as
(array1, array2, array3, ..., arrayN)
But I don't have all the streams so instead I get this mess:
stream_zip = streamz.zip(some_zipped_stream, another_stream)
# this will end up with a bunch of nested tuples of arrays
(...( ( (array1, array2), array3), array4), array5) . . .)
I want this:
streams_uzip_list = streamz.unzip(some_zipped_stream)
stream_zip = streamz.zip(*streams_uzip_list, another_stream)
# this stream_zip would appear as
(array1, array2, array3, ..., arrayN)
Thanks
from streamz.
With an unzip API I would always insure that there's only one level of tuple hierarchy. I don't want to create nested tuples. If someone were to pass in a nested tuple that would not be my problem. I just need to make sure not to produce nested tuple. So if I get something like this:
((array1, array2), array3)
Unzip/re-zip it so that I would produce:
((array1, array2), array3, array4)
I just don't want to produce this:
(((array1, array2), array3), array4)
from streamz.
Related Issues (20)
- missing positional argument: 'topic' in to_mqtt HOT 5
- How to parametrize stream/pipeline creation? HOT 2
- Passing Username and Password to from_mqtt() HOT 4
- Dynamically add upstreams to zip HOT 2
- Dropping `pkg_resources`
- AttributeError: 'Output' object has no attribute '_ipython_display_' HOT 3
- flatten doesn't work with iterables without defined length HOT 2
- visualizing streams and changing variables during runtime HOT 5
- Is it possible to use event time/syntethic time rather than system time? HOT 2
- Time based lookback window? HOT 1
- Streamz not working in Jupyterlite
- Collect does not allow awaitable sinks
- Quickstart lacks conda/environments/streamz_dev.yml HOT 6
- Hello from LorryStream and Kotori / Streamz is cool HOT 1
- Add pytest fixture to clean up the IO loop HOT 4
- Streamz hvplot resets zoom and pan on each update HOT 3
- streamz's typing system can't work properly in vscode HOT 2
- Streamz with websocket not steaming any data HOT 5
- Parallel streams with buffers HOT 3
- Is streamz not maintained anymore? What happened to cuStreamz? HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from streamz.