Add commands to retrieve test history about onos-test HOT 3 CLOSED

@kuujo
I think implementation of this feature should be straightforward but we should decide what is the best way to define history commands. Here some of my ideas for the history commands. What do you think?

// Get the status of all tests that are finished successfully.
onit history --status --phase="Succeeded"

// Get the status of all "Failed "tests
onit history --status --phase="Failed"

// Get the status of all tests in kube-test namespace (i.e. without filtering on phase)
onit history --status

// Get the status of one specific test
onit history --status -n <Test-pod-name>

// Get logs of a specific test
onit history --logs -n <Test-pod-name>

from onos-test.

There is also going to be a differentiation between test history and benchmark history. Benchmarks and logs are where this becomes a really useful feature.

I think we probably want different commands for different types of output. The output for the history of tests (pass/fail) will be different from the output of the history of benchmarks (throughput/latency) will be different from the logs. I think this is a good reason to have separate commands for each of those outputs to ensure command output format is deterministic (e.g. for parsing).

Thinking about how this could work, the log command will have to take the name of a node for which to output the logs. I think before a namespace is torn down, we should store all its logs in a ConfigMap keyed by node ID. In order to get the logs, the client has to know which nodes were running for a test (which keys are in the CM), so we’ll need some command to get that information. And we’ll also want a command to download all the logs from a test run.

Finally, I think we should try to stick with the CLI patterns used by kubectl and formalized in that GopherCon talk:

• Use onit verb noun
• Use flags rather than variadic arguments for optional arguments

So here could be an alternative:

• onit get history
• onit get test history
• onit get benchmark history
• onit get test config
• onit get benchmark config
• onit get test logs --test=x --pod=y
• onit get benchmark logs --benchmark=x --pod=y

from onos-test.

@kuujo
Good points. Agreed. I was thinking about get command but I thought it would be better to make the commands shorter but you are right, we should use onit verb noun pattern.

from onos-test.