Comments (11)
This is a strictness bug in the Await constructor. It is fixed in the "lazy" branch and the "await" branch pending a merge into trunk.—
Sent from Mailbox for iPhone
On Sat, Feb 1, 2014 at 5:31 PM, Frank S. Thomas [email protected]
wrote:
As I was writing more tests for
process1
processes I noticed an odd behavior ofchunkBy
:
Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList
res2: List[Vector[Int]] = List(Vector(1, 3, 8), Vector(5, 2, 4), Vector(7))
The emitted chunks contain elements that satisfy and that do not satisfy the predicate. Is this behavior intended? Because, I would have expected the following result:
List(Vector(1, 3), Vector(8, 5), Vector(2), Vector(4, 7))
Since there is no test forchunkBy
I do not know what the result should have been. So either the documentation should be extended or the implementation should be fixed.
Btw, here is version ofchunkBy
that produces the result I had expected:
def chunkByFixed[I](f: I => Boolean): Process1[I, Vector[I]] = {
def go(acc: Vector[I], last: Boolean): Process1[I, Vector[I]] =
await1[I].flatMap { i =>
val cur = f(i)
if (cur == last) go(acc :+ i, cur)
else emit(acc) fby go(Vector(i), cur)
} orElse emit(acc)
await1[I].flatMap(i => go(Vector(i), f(i)))}
Reply to this email directly or view it on GitHub:
https://github.com/scalaz/scalaz-stream/issues/86
from fs2.
I checked this in the "lazy" and "await/trampolined" branches and the result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList
was always List(Vector(1, 3, 8), Vector(5, 2, 4), Vector(7))
.
from fs2.
Oh, sorry, I didn't read your message carefully enough. I thought this was
a different problem.
The behavior of chunkBy looks correct. The purpose of this method is to
split the stream on elements that don't match the predicate. The expected
behavior is that it emit a chunk when an an element makes the predicate
false, as indicated by the documentation for the method.
For example, if you had the stream "foo bar baz", chunkBy(_ != " ") would
turn that into ["foo ", "bar ", "baz"].
On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas [email protected]:
I checked this in the "lazy" and "await/trampolined" branches and the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList was always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797
.
from fs2.
Runar,
du you think your laziness fix is tested enough for stack safety etc?
I mean there is separate branch on switching signature of await to
Await(rcv: Throwable/ R => Trampoline[Process[F,A]], clenup : Option[Throwable] => Trampoline[Process[F,Nothing]]) which is in middle of progress.
I am just curious what will happened if we merge your code into master and if the effort with trampolining is worth of.
P.
On 02 Feb 2014, at 01:42, Rúnar [email protected] wrote:
Oh, sorry, I didn't read your message carefully enough. I thought this was
a different problem.The behavior of chunkBy looks correct. The purpose of this method is to
split the stream on elements that don't match the predicate. The expected
behavior is that it emit a chunk when an an element makes the predicate
false, as indicated by the documentation for the method.For example, if you had the stream "foo bar baz", chunkBy(_ != " ") would
turn that into ["foo ", "bar ", "baz"].On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas [email protected]:
I checked this in the "lazy" and "await/trampolined" branches and the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList was always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797
.—
Reply to this email directly or view it on GitHub.
from fs2.
Ok, now chunkBy
and its description makes sense to me. Thanks for the clarification Runar. I'll send a PR with a test for it and an example in the docstring.
Btw, what would be a proper name for the chunkByFixed
that I proposed? Maybe partition
, groupBy
, or chunkWith
? Would it be a useful addition to process1
?
from fs2.
Hi Pavel,
I do not know how stack-safe it is. Do you have any specific suspicions
where stack overflows are possible? I can think of two places I might
suspect trampolining is needed, but I can refute them pretty easily. I know
of one company that is using the lazy
branch right now and they haven't
run into SOEs yet.
The current behavior of chunk |> filter is totally wrong, and has been for
a long time.
Runar
On Sun, Feb 2, 2014 at 2:05 AM, Pavel Chlupacek [email protected]:
Runar,
du you think your laziness fix is tested enough for stack safety etc?
I mean there is separate branch on switching signature of await to
Await(rcv: Throwable/ R => Trampoline[Process[F,A]], clenup :
Option[Throwable] => Trampoline[Process[F,Nothing]]) which is in middle of
progress.I am just curious what will happened if we merge your code into master and
if the effort with trampolining is worth of.P.
On 02 Feb 2014, at 01:42, Rúnar [email protected] wrote:
Oh, sorry, I didn't read your message carefully enough. I thought this
was
a different problem.The behavior of chunkBy looks correct. The purpose of this method is to
split the stream on elements that don't match the predicate. The
expected
behavior is that it emit a chunk when an an element makes the predicate
false, as indicated by the documentation for the method.For example, if you had the stream "foo bar baz", chunkBy(_ != " ")
would
turn that into ["foo ", "bar ", "baz"].On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas <
[email protected]>wrote:I checked this in the "lazy" and "await/trampolined" branches and the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList was
always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797>
.Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33894016
.
from fs2.
Yeah, not sure non that.
Paul was bringing some issues with this before, so maybe he has something more on this.
Anyhow I also solved the left-assotiated flatMap performance issues in trampoline branch (not sure if I can do it w/o trampoline), so maybe I gonna still take this path.
Was more thinking if we can`t get that code of yours merged so we can have something in quickly.
Also one idea is if we should`t just drop case in Await and make his signature lazy
I mean
instead of
case class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: %28%29 => Process[F,O] = %28%29 => halt,
cleanup1: %28%29 => Process[F,O] = %28%29 => halt) extends Process[F,O]
just do
class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: => Process[F,O] = halt,
cleanup1: => Process[F,O] = halt) extends Process[F,O] {
lazy val fb = fallback1
lazy val c = cleanup1
}
object Await {
def apply() …
def unapply() ...
}
That will maybe solve many issues w/o changing the signature heavily and still be non-strict ?
???
P.
On 02 Feb 2014, at 15:15, Rúnar [email protected] wrote:
Hi Pavel,
I do not know how stack-safe it is. Do you have any specific suspicions
where stack overflows are possible? I can think of two places I might
suspect trampolining is needed, but I can refute them pretty easily. I know
of one company that is using thelazy
branch right now and they haven't
run into SOEs yet.The current behavior of chunk |> filter is totally wrong, and has been for
a long time.Runar
On Sun, Feb 2, 2014 at 2:05 AM, Pavel Chlupacek [email protected]:
Runar,
du you think your laziness fix is tested enough for stack safety etc?
I mean there is separate branch on switching signature of await to
Await(rcv: Throwable/ R => Trampoline[Process[F,A]], clenup :
Option[Throwable] => Trampoline[Process[F,Nothing]]) which is in middle of
progress.I am just curious what will happened if we merge your code into master and
if the effort with trampolining is worth of.P.
On 02 Feb 2014, at 01:42, Rúnar [email protected] wrote:
Oh, sorry, I didn't read your message carefully enough. I thought this
was
a different problem.The behavior of chunkBy looks correct. The purpose of this method is to
split the stream on elements that don't match the predicate. The
expected
behavior is that it emit a chunk when an an element makes the predicate
false, as indicated by the documentation for the method.For example, if you had the stream "foo bar baz", chunkBy(_ != " ")
would
turn that into ["foo ", "bar ", "baz"].On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas <
[email protected]>wrote:I checked this in the "lazy" and "await/trampolined" branches and the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList was
always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797>
.Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33894016
.—
Reply to this email directly or view it on GitHub.
from fs2.
That works too, Pavel, although then wherever anyone is using Await, they
will have to say new Await.
On Sun, Feb 2, 2014 at 9:27 AM, Pavel Chlupacek [email protected]:
Yeah, not sure non that.
Paul was bringing some issues with this before, so maybe he has something
Anyhow I also solved the left-assotiated flatMap performance issues in
trampoline branch (not sure if I can do it w/o trampoline), so maybe I
gonna still take this path.
Was more thinking if we can`t get that code of yours merged so we can have
something in quickly.Also one idea is if we should`t just drop case in Await and make his
signature lazyI mean
instead of
case class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: %28%29 => Process[F,O] = %28%29 => halt,
cleanup1: %28%29 => Process[F,O] = %28%29 => halt) extends Process[F,O]just do
class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: => Process[F,O] = halt,
cleanup1: => Process[F,O] = halt) extends Process[F,O] {
lazy val fb = fallback1
lazy val c = cleanup1
}object Await {
def apply() ...
def unapply() ...}
That will maybe solve many issues w/o changing the signature heavily and
still be non-strict ????
P.
On 02 Feb 2014, at 15:15, Rúnar [email protected] wrote:
Hi Pavel,
I do not know how stack-safe it is. Do you have any specific suspicions
where stack overflows are possible? I can think of two places I might
suspect trampolining is needed, but I can refute them pretty easily. I
know
of one company that is using thelazy
branch right now and they
haven't
run into SOEs yet.The current behavior of chunk |> filter is totally wrong, and has been
for
a long time.Runar
On Sun, Feb 2, 2014 at 2:05 AM, Pavel Chlupacek <
[email protected]>wrote:Runar,
du you think your laziness fix is tested enough for stack safety etc?
I mean there is separate branch on switching signature of await to
Await(rcv: Throwable/ R => Trampoline[Process[F,A]], clenup :
Option[Throwable] => Trampoline[Process[F,Nothing]]) which is in
middle of
progress.I am just curious what will happened if we merge your code into master
and
if the effort with trampolining is worth of.P.
On 02 Feb 2014, at 01:42, Rúnar [email protected] wrote:
Oh, sorry, I didn't read your message carefully enough. I thought
this
was
a different problem.The behavior of chunkBy looks correct. The purpose of this method is
to
split the stream on elements that don't match the predicate. The
expected
behavior is that it emit a chunk when an an element makes the
predicate
false, as indicated by the documentation for the method.For example, if you had the stream "foo bar baz", chunkBy(_ != " ")
would
turn that into ["foo ", "bar ", "baz"].On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas <
[email protected]>wrote:I checked this in the "lazy" and "await/trampolined" branches and
the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList was
always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797>
.
Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33894016>
.Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33901550
.
from fs2.
Ok for this we can define Await.apply perhaps?
P.
On 02 Feb 2014, at 16:02, Rúnar [email protected] wrote:
That works too, Pavel, although then wherever anyone is using Await, they
will have to say new Await.On Sun, Feb 2, 2014 at 9:27 AM, Pavel Chlupacek [email protected]:
Yeah, not sure non that.
Paul was bringing some issues with this before, so maybe he has something
Anyhow I also solved the left-assotiated flatMap performance issues in
trampoline branch (not sure if I can do it w/o trampoline), so maybe I
gonna still take this path.
Was more thinking if we can`t get that code of yours merged so we can have
something in quickly.Also one idea is if we should`t just drop case in Await and make his
signature lazyI mean
instead of
case class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: %28%29 => Process[F,O] = %28%29 => halt,
cleanup1: %28%29 => Process[F,O] = %28%29 => halt) extends Process[F,O]just do
class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: => Process[F,O] = halt,
cleanup1: => Process[F,O] = halt) extends Process[F,O] {
lazy val fb = fallback1
lazy val c = cleanup1
}object Await {
def apply() ...
def unapply() ...}
That will maybe solve many issues w/o changing the signature heavily and
still be non-strict ????
P.
On 02 Feb 2014, at 15:15, Rúnar [email protected] wrote:
Hi Pavel,
I do not know how stack-safe it is. Do you have any specific suspicions
where stack overflows are possible? I can think of two places I might
suspect trampolining is needed, but I can refute them pretty easily. I
know
of one company that is using thelazy
branch right now and they
haven't
run into SOEs yet.The current behavior of chunk |> filter is totally wrong, and has been
for
a long time.Runar
On Sun, Feb 2, 2014 at 2:05 AM, Pavel Chlupacek <
[email protected]>wrote:Runar,
du you think your laziness fix is tested enough for stack safety etc?
I mean there is separate branch on switching signature of await to
Await(rcv: Throwable/ R => Trampoline[Process[F,A]], clenup :
Option[Throwable] => Trampoline[Process[F,Nothing]]) which is in
middle of
progress.I am just curious what will happened if we merge your code into master
and
if the effort with trampolining is worth of.P.
On 02 Feb 2014, at 01:42, Rúnar [email protected] wrote:
Oh, sorry, I didn't read your message carefully enough. I thought
this
was
a different problem.The behavior of chunkBy looks correct. The purpose of this method is
to
split the stream on elements that don't match the predicate. The
expected
behavior is that it emit a chunk when an an element makes the
predicate
false, as indicated by the documentation for the method.For example, if you had the stream "foo bar baz", chunkBy(_ != " ")
would
turn that into ["foo ", "bar ", "baz"].On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas <
[email protected]>wrote:I checked this in the "lazy" and "await/trampolined" branches and
the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList was
always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797>
.
Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33894016>
.Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33901550
.—
Reply to this email directly or view it on GitHub.
from fs2.
That works.
On Sun, Feb 2, 2014 at 10:03 AM, Pavel Chlupacek
[email protected]:
Ok for this we can define Await.apply perhaps?
P.
On 02 Feb 2014, at 16:02, Rúnar [email protected] wrote:
That works too, Pavel, although then wherever anyone is using Await,
they
will have to say new Await.On Sun, Feb 2, 2014 at 9:27 AM, Pavel Chlupacek <
[email protected]>wrote:Yeah, not sure non that.
Paul was bringing some issues with this before, so maybe he has
somethingAnyhow I also solved the left-assotiated flatMap performance issues in
trampoline branch (not sure if I can do it w/o trampoline), so maybe I
gonna still take this path.
Was more thinking if we can`t get that code of yours merged so we can
have
something in quickly.Also one idea is if we should`t just drop case in Await and make his
signature lazyI mean
instead of
case class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: %28%29 => Process[F,O] = %28%29 => halt,
cleanup1: %28%29 => Process[F,O] = %28%29 => halt) extends Process[F,O]just do
class Await[F[_],A,+O] private[stream](req: F[A], recv: A => Process[F,O],
fallback1: => Process[F,O] = halt,
cleanup1: => Process[F,O] = halt) extends Process[F,O] {
lazy val fb = fallback1
lazy val c = cleanup1
}object Await {
def apply() ...
def unapply() ...}
That will maybe solve many issues w/o changing the signature heavily
and
still be non-strict ????
P.
On 02 Feb 2014, at 15:15, Rúnar [email protected] wrote:
Hi Pavel,
I do not know how stack-safe it is. Do you have any specific
suspicions
where stack overflows are possible? I can think of two places I
might
suspect trampolining is needed, but I can refute them pretty easily.
I
know
of one company that is using thelazy
branch right now and they
haven't
run into SOEs yet.The current behavior of chunk |> filter is totally wrong, and has
been
for
a long time.Runar
On Sun, Feb 2, 2014 at 2:05 AM, Pavel Chlupacek <
[email protected]>wrote:Runar,
du you think your laziness fix is tested enough for stack safety
etc?I mean there is separate branch on switching signature of await to
Await(rcv: Throwable/ R => Trampoline[Process[F,A]], clenup :
Option[Throwable] => Trampoline[Process[F,Nothing]]) which is in
middle of
progress.I am just curious what will happened if we merge your code into
master
and
if the effort with trampolining is worth of.P.
On 02 Feb 2014, at 01:42, Rúnar [email protected] wrote:
Oh, sorry, I didn't read your message carefully enough. I
thought
this
was
a different problem.The behavior of chunkBy looks correct. The purpose of this
method is
to
split the stream on elements that don't match the predicate. The
expected
behavior is that it emit a chunk when an an element makes the
predicate
false, as indicated by the documentation for the method.For example, if you had the stream "foo bar baz", chunkBy(_ != "
")
would
turn that into ["foo ", "bar ", "baz"].On Sat, Feb 1, 2014 at 5:47 PM, Frank S. Thomas <
[email protected]>wrote:I checked this in the "lazy" and "await/trampolined" branches
and
the
result of Process(1, 3, 8, 5, 2, 4, 7).chunkBy(_ < 4).toList
was
always List(Vector(1,
3, 8), Vector(5, 2, 4), Vector(7)).Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33885797>
.
Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33894016>
.
Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHub<
https://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33901550>
.Reply to this email directly or view it on GitHub.
Reply to this email directly or view it on GitHubhttps://github.com/scalaz/scalaz-stream/issues/86#issuecomment-33902344
.
from fs2.
WRT to naming chunkByFixed
, scalaz.Foldable
has splitWith
:
Vector(1, 3, 8, 5, 2, 4, 7).splitWith(_ < 4)
res3: List[List[Int]] = List(List(1, 3), List(8, 5), List(2), List(4, 7))
from fs2.
Related Issues (20)
- Add `Channel#sendClose` HOT 1
- sbt doc task fails for scala 3 and fs2-io HOT 2
- Update copyright for 2023 HOT 1
- parEvalMap messes up unmasking of cancellation
- Feature request: Dynamic metering of Streams HOT 3
- Bad performance of fs2.io.file.Files[IO].walk() HOT 4
- Fetching permissions through getPosixFileAttributes always fails with None.get
- Fetched file attributes fail equality check HOT 1
- Optimize `Stream#evalTapChunk`
- Root `Path` is incorrectly handled in ScalaJS HOT 4
- Error using reactive streams toStreamBuffered with newer versions of the AWS SDK HOT 9
- Usage of direct buffers on FS2 Socket causes an unpredictable memory use on http4s with embers backend HOT 5
- failed: fs2.io.net.udp.UdpSuite.udp HOT 1
- reactive-streams interop: handled errors treated as unhandled and printed to the console HOT 1
- Improvements around for the PosixPermissions api HOT 3
- Memory leak when a recursive stream has error handling HOT 5
- DatagramSocket read stalls on JVM HOT 2
- Deprecated Files.walk(start: Path, maxDepth: Int, followLinks: Boolean) ignores maxDepth and followLinks HOT 1
- `Stream` created with `fromIterator` looping infinitely when evaluated in 3.10.x HOT 3
- Introduce higher-level "network client" APIs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fs2.