tjb1982 / linen Goto Github PK

We currently have something like

checkpoint:
  nodes:
  - out: VAR1
    err: VAR2
  source: echo foo

but it might be interesting to have something like

checkpoint:
  nodes:
  - out: VAR1
    in: |
      foo
      bar
      baz
      quux
    source: grep "^b"

where this would be functionally equivalent to

echo "..." | grep "^b"

Right now there's a single key called throw which is used to indicate whether a test should be counted among the failures at the end of the test. There needs to be a way to actually throw the test (i.e., not continue, clean-up, etc.) upon some condition. In conjunction with that there needs to be a separate key that indicates whether the process should be considered a failure or ignored. And I think this key will need to be a function that takes out err exit as parameters.

# means that flax should stop and cleanup if the checkpoint fails
throw: boolean
assert:
    ~(fn:
    - [ out, err, exit ]
    - ~(zero?: [ exit ]

If throw is true and no assert key is set, it should just check the exit code for non-zero, otherwise it should throw based on the pass/fail of the assertion.

Since the source code is known ahead of time we should run a pre-check-modules function or something like that that would report errors due to non-well-formed structures immediately instead of doing it at runtime, when it could potentially be a very long time before you find out that your source code is bad.

Currently there isn't even really a way to destroy a process. I've read here and here that Java 9 has some nice new process api changes that would help with this.

At a minimum, the node manager needs to be able to catch a ThreadInterrupted exception and kill any running processes, ensuring that they are actually killed. I'm sure there are some best practices around this.

e.g.,

invocation: |
  #!/usr/bin/env python2
  import sys
  from __future__ import print_function
  print("FAIL", file=sys.stderr)
  sys.exit(1)