Messages from the "main channel" as a receivePort about distributed-process HOT 8 CLOSED

+1 - this would be very useful for distributed-process-platform e.g., GenServer could make good use of this.

from distributed-process.

I can't see how to implement this, however I have implemented an alternative that achieves the desired effect; see 847abf4

Let me know what you think, and maybe we can merge it in.

from distributed-process.

I'm reading through this to understand it at the moment. Forgive me for talking out loud here - I'm relatively green and it seems to help. So this definition is pretty neat:

 waitIncoming = fmap Left (readTChan incoming) `orElse` fmap Right readchans

Am I right in reading this as meaning that the implementation of readTChan must call retry if the channel is empty?
I see the non-blocking read in CQueue composes this with another STM action that just returns Nothing so we can get out if all of the received queue, the main input channel and whatever is given for readchans evaluates to, are all empty or contain no messages that satisfy the list of matches given.

Then in Primatives.receiveWait where we want to block indefinitely (until we get a message) you pass retry as readchans and therefore we restart the transaction again. For the xChans variants in Primatives you're basically passing matchChans instead of retry which uses receiveSTM on the receive port, and this presumably works in a similar vein.

Have I grokked that properly? It seems like a reasonable approach from what I've understood so far.

I can't see at all how to create the alternative expectChan API either. In fact, from what I've seen I think that would be far too invasive a change. You'd surely need to change the node controller's inbound message handling to do that, because from what I can see at the moment, channels are implemented using a different data structure, which is accessed and written to by handleIncomingMessages when the destination of the incoming message is addressable by SendPortIdentifier rather than ProcessIdentifier.

So even if there is some way of getting at that process state from outside the node controller, I can't see how to make that work - how would you know that a message addressed to the ProcessIdentifier needed to be re-routed as it were?

from distributed-process.

Yes, you're understanding it properly. To implement expectChan, the main process inbox would need to be an STM-only data structure, so that we could compose it with the other channel reads. However, doing this would make it very difficult to implement efficiently, because there would be no way to avoid re-reading the whole message queue each time a new message came in.

I talked this over with @edsko and @dcoutts on IRC yesterday, and we think that it's possible to implement the other design alternative:

matchChan :: ReceivePort a -> (a -> Process b) -> Match b

Which is a slightly nicer API, but a bit more difficult to implement. I'm looking into it now.

from distributed-process.

So even if there is some way of getting at that process state from outside the node controller, I can't see how to make that work - how would you know that a message addressed to the ProcessIdentifier needed to be re-routed as it were?

Rerouting would not be necessary -- you would simply inspect the mailbox when somebody tries to read from the ReceivePort, which is just a wrapper around an STM transaction. The difficulty with this approach is that we need to have an STM transaction to extract a message from the mailbox. The mailbox is already an STM channel but at the moment I try hard to keep the STM transactions used in match and co very short. In order to implement expectChan we'd have to change it so that the entire operation to extract a mailbox message would have to run inside STM. This would lead to larger STM transactions when reading from the mailbox, which may be interrupted by writers. This would be bad for performance. There may be a clever way to do it, but it needs more thought.

@simonmar's matchChan solution is very similar to expectChan in that it gives you a way to look at the mailbox as well as a channels, just like expectChan does, but it has the advantage that we can group all matches that look at the mailbox, scan the mailbox as usual, if that fails, more to the next group of matches that look at channels, and so on. In the worst case, if you alternate mailbox and channel matches, this may still have pretty bad performance, but with a bit of care from the person writing the matches it should work.

from distributed-process.

@simon @edsko thanks for the explanations, I can see why a course grained transaction would suck. What irc channel(s) are worth hanging out on btw - is there one for parallel Haskell in particular?

from distributed-process.

Support for matchChan has now been added -- many thanks to @simonmar.

from distributed-process.

Thanks to both of you for the quick turn-around! As @rodlogic mentioned elsewhere, this will be enormously useful in distributed-process-platform. Now all I have to do is read through it and understand how it works... :)

from distributed-process.