longevityframework / stream-adapter Goto Github PK
View Code? Open in Web Editor NEWConversions between various Scala streaming libraries
License: Other
Conversions between various Scala streaming libraries
License: Other
Two adapters, from and into Chunkerator
, would give us reactive streams adapters for all the streaming libraries.
Note that when we do this, we should probably special-case streaming libraries that already have reactive streams adapters, such as Akka and Play. At this point, we will need to make sure the implicit resolution picks up the direct adaptation, and not the one from chain
that uses Chunkerator
in the middle.
Current implementation limitations:
Bimonad
instead of just a Monad
. I'm not sure if this is negotiable or not. It seems like we need to call F.extract
on the iteratee to get it to "run".Future
from the global execution context to kick off the producer asynchronously. A more catsy approach, as described in the last bullet point, should do away with this. I didn't want to add an implicit ExecutionContext
argument here, as that seems like an implementation detail, and a rewrite of this method without it would not be backwards compatible. Maybe there is some way to pull this off with Task
? I tried, couldn't figure it outRight now, I only target the most recent versions of all the various streaming libraries. It would be nice to support a range of versions, from the most recent, down to the latest version that is compatible with the most recent. This would involve a test suite that would test across all these library versions. Some SBT-fu required.
This would really be great to have, but a little too much effort for me at the moment.
I haven't thought too hard about this. Either it's a silly idea or it's a good one and a lot of work.
The current implementation of this method is a little lame, using promises to communicate between the producer and the consumer. There's got to be a better approach. I'd like this method to work without any dependencies besides Play and Scala standard library, so that's a limiting factor.
We should be able to implement this regardless of the effect type.
Current implementation limitations:
Task
effect is hardcoded. We should be able to adapt regardless of the effect type (so long as it's monadic).Future
from the global execution context to kick off the producer asynchronously. A more catsy approach, as described in the last bullet point, should do away with this. I didn't want to add an implicit ExecutionContext
argument here, as that seems like an implementation detail, and a rewrite of this method without it would not be backwards compatible. Maybe there is some way to pull this off with Task
? I tried, couldn't figure it outOnce again, two adapters, from and into Chunkerator
, would give use adapters between Monix and all the other streaming libraries.
There are tests for this for all the FromChunkerator
adapters, but no tests yet for the ToChunkerator
adapters.
Once again, two adapters, from and into Chunkerator
, would give use adapters between RxJava and all the other streaming libraries.
All three of the type arguments to adapt
should be inferrable by the compiler. The first and the third argument are inferrable from the argument, fs2Stream
, and the second argument should be inferrable from the left-hand side. But I haven't figured out how to get the compiler to infer any of them yet. Can you help? It's not as simple as you might think, because the implicit resolution has to navigate between types with two type parameters, such as Enumerator[Eval,Int]
, and types with a single type parameter, such as streamadapter.iterateeio.EvalEnumerator[Int]
.
I've seen a few of these errors come back really hammy. We should be able to provide custom error messages for implicit resolution problems that make more sense. I don't have any examples at the moment, but if I see any, I will comment on it here. (Please do the same, okay?)
How do I change timeout programmatically ? It's hardcoded to 5 seconds for some reason.
And what's the meaning of this timeout ? Is it time within which I should materialize and complete consumption of resulting akka Source ? What if I process it for hours ?
We might get better performance using chunking here. Shouldn't be too hard to implement if you don't mind putting in, say, a private var currentChunk: Seq[A]
between def apply
and def applyInternal
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.