Should streamz handle exceptions? about streamz HOT 7 OPEN

or sending the data + exception to another pipeline.

This is exactly how we handle exceptions in Apache NiFi and the feedback from community members seems to almost always be positive.

from streamz.

Yes, at some point we should handle exceptions. Also stopping signals. I don't have a short term plan for this or any concrete thoughts.

from streamz.

This reminds me a little of the Try-Monads talk from pygotham (not that I think we should necessarily use Try-Mondads).

from streamz.

from streamz.

Relevent? http://reactivex.io/RxJava/2.x/javadoc/io/reactivex/exceptions/Exceptions.html

from streamz.

This might be one area where treating the graph as a walkable DAG may be helpful. One could imagine writing a function which walked the graph, looked for nodes where exceptions could occur (map, filter, etc.) and then decorates update to handle the exceptions, potentially with some callback function, logging, or sending the data + exception to another pipeline.

I think this general pattern, keep the pipeline simple and then decorate things once the DAG has been made is interesting.

from streamz.

We currently pass messages forward and futures backwards with the emit/update methods. My guess is that we would do something similar with exception and stop signals.

from streamz.