comparing what student has done to a target about shellwhat HOT 5 CLOSED

Thanks @machow - cmp will do what I need in the single-file case provided we can put the reference file containing the correct content somewhere that the learner won't see it (don't want to distract them). Unfortunately, it looks like there's no recursive option to cmp as there is to diff, so I'll either:

1. compare a small number of files by hand
2. use diff with appropriate flags

The first is probably the best option anyway, since I don't know how I'd give sensible error reporting in the second case.

So, where should I put the reference files? I presume I provision them in requirements.sh?

from shellwhat.

provisioning in requirements.sh sounds like the right move.

For (1), if it's useful, you can put the SCTs in a loop...

# cmp returns non-zero exit code (error) when files don't match
list_of_files = ['a.txt', 'b.txt']
for fname in list_of_files:
    feedback = "some message"
    cmd = 'cmp {fname} /some_dir/{fname}'.format(fname = fname)
    Ex().test_expr_error(cmd, feedback)

alternatively, tests that don't have Ex before them don't run right away and could be used later...

# defines tests but doesn't run yet
tests = [test_expr_error('cmp {0} /some_dir/{0}'.format(fname)) for fname in list_of_files]
# runs tests
Ex().multi(tests)

from shellwhat.

Thanks @machow - if the tests are in a loop, will the error reports from each be queued up somehow for the user to read, and will the final pass/fail status automatically be the conjunction of all the tests that were run?

from shellwhat.

Outside of logical tests like test_or and test_correct, SCTs stop running / give feedback at the first failure.

https://protowhat.readthedocs.io/en/latest/#module-protowhat.checks.check_logic

(edit: this happens because being able to run one SCT, often requires previous ones to pass)

from shellwhat.

In the refactored version of shellwhat, there are examples of how has_expr_output() can be used. Once you get used to it, it's pretty elegant.

• Link to ref docs
• glossary example

from shellwhat.