pwr22 / go-zoom Goto Github PK

We don't update lastJobPrinted so it sits with value -1 and we'll only ever do one scan through of pending outputs when job number 0 finishes

Thanks to @jes for spotting this

IMO the Rust Parallel semantics make more sense but GNU Parallel is the popular implementation everyone will be using

I think the answer to this probably boils down to "Are we mostly compatible with GNU Parallel?" and if the answer is yes then adopt those semantics, otherwise not. If supporting GNU Parallel compatibility behind a flag then we can potentially have both semantics available

In the current implementation this will generate an empty list of commands so it's pointless to allow it

In parallel it will do a permutation of the first line with every other line but it does allow it. I consider this more of a bug than a feature since I can't find it documented anywhere

This is to avoid confusion with things like the zoom messenger or presentation library.

The file should be passed as the only argument on the command line

It's like a giant single function now - it can be split up

Just the basics here: any occurrences of {} will be replaced with the job arguments and this will disable the auto-concatenation

We might pull them out into separate packages at some point but for now just keep them all in zoom.

In the event that a command fails then zoom should exit with that itself

Best practice these days.

This will have no meaningful effect right now because the process is going to exit pretty much right after

However, when the runner is itself a separate project then I still don't think it would be a problem because the only goroutine that would ever listen on that channel is gone but it's still worth doing to keep everything clean

Device on what it is and stub out the arguments with a not implemented yet response.

Options

• commands: will work the same except maybe shell functions
• {}: will work the same including the quoting
• {.}: will work the same
• {/}: will work the same
• {//}: will work the same
• {./}: will work the same
• {#}: will work the same
• {%}: will work the same
• {n}: will work the same except not doing input source stuff
• {n.}: will work the same except not doing input source stuff
• {n/}: will work the same except not doing input source stuff
• -I: will work the same
• --er: will work the same but be called --extensionless-placeholder
• --basenamereplace: will work the same but be called --file-placeholder
• --dirnamereplace: will work the same but be called --directory-placeholder
• --basenameextensionreplace: will work the same but be called --extentionless-file-placeholder.
• --seqreplace: will work the same --job-placeholder
• --slotreplace: will work the same but be called --runner-placeholder

WIP

And the Windows equivalent (%COMSPEC%?)

• More details on other similar solutions
• More details on the motivations
• Aiming for a reasonably simple subset of GNU Parallel interface
• Add details on the differences between zoom, GNU Parallel and Rust Parallel

The ID could be the ordinal number of the jobs 1..n. This will be particularly useful if the output is interleaved.

This will make the code more testable and move all the os.Exit() usage to one place (probably main)

This will make the code tidier and setting options easier

Would be good to have a few different scenarios and compare cpu, memory and runtime to GNU parallel, MIT Parallel and rush.

Signals to handle

• SIGINT
• SIGBREAK
• SIGTERM

And any others that are appropriate

Make commands output always come in the order they are given

Details can be found by running go test -race ./...

Better to keep the readme short and sweet

I'm not sure how to do with with pflag or even if it's possible. Right now it shows the flags which is great but there needs to be a lot more documentation of the other arguments that can be passed and what the tool actually does!

This will make testing easier as we'll be able to easily capture the output to check it or to send it to a black hole if we don't care about it

This is to match GNU Parallel, make, gcc and other usual things

This can occur if the commands from the initial batch are sent into the channel before the main goroutine starts. So basically if an error occurs in a command before all the initial commands are started then zoom will not stop all commands and exit early as expected

The standard go flag stuff violates the principle of least surprise and is less feature rich as well. E.g. no "for free" shortened forms

Like in GNU Parallel

Right now if one command fails we try to stop every other command and then wait for them all to finish. This is so things always end gracefully and we always get all output (zoom was designed for some tools that are a bit flakey at flushing their output).

Other failure modes we should support:

• Eventually sending SIGKILL to commands after some timeout.
• Ignoring failures and continuing in a best effort to complete as much work per run as possible (later probably want to add some way of tracking what failed to allow retries).

Since GOMAXPROCS is set to the number of logical CPU cores since Go 1.5 this seems sensible to me

Atm we allocate a slice of length n for n commands to run - two of them actually

Moving to a map should work just fine and reduce memory usage