document failure/retry modes in distributed use about glow HOT 1 CLOSED

Good point. I feel "at most once" or "at least once" sounds very professional when I hear someone else using these terms, because I always have to pause a bit to think about what it really means. I think glow can be classified to "at most once", for now.

Actually glow does not try to use these terms. Glow runs just like a common program. It does not try to auto-recover from any actual machine shutdown. It just breaks like having an exception.

So this in a sense just give back the retry decision to the user. The user can decide to run it again, or fix some real issue with the code. This would not work for google/facebook scale clusters, but should work fine for 10 ~ 20 nodes.

You are welcome to adjust the readme in a better way to explain this.

from glow.