Should be in root folder instead.

Soon, there will be two forms of message 'logging' or generation in protowhat:

• Through the history attribute on State (used in check_field() and check_node())
• Through the messages attribute (used by all of sqlwhat's result-checking functions.

For now, it's not too big of a problem, but it could become one in the future. Ideally, there's a unified system of building 'expand messages' like in pythonwhat, that point the student to the right bit of code, and a hint at what is wrong or how things can be fixed.

Currently has form { “line_start”: 0, … }

Need to add optional file property: { “file”: “script.py”, path: “”, “line_start”: 0, … }

This property can either be put on the state (rather than mutating the reporter).

It becomes a bit hard to understand :)

• There is a test_exercise function in sqlwhat and shellwhat, but not in protowhat. Seems like test_exercise should be moved there, with a smart way of including the language-specific SCT context and state

• The reporter depends on a singleton-like failed_test variable, that it uses to build up a payload. It is not required. Thoughts by @machow about this:

I regret never finishing cleaning up Reporter (and taking out failed_test). It would super clean to have reporter just hold options for reporting outcomes, and then rather than carrying around the backend output with it, test_exercise could pass that at the end. See [here](see https://github.com/datacamp/protowhat/blob/fs/test-not/protowhat/Reporter.py#L57-L58). Something like

def fake_test_exercise(output):
    reporter = Reporter()
    try: test_something('bleh')
    except TestFail as e:
        return reporter.build_payload(e)
    return reporter.build_payload(output)

As an SCT author, I want to be able to write:

Ex() >> test_or(test_1(...), test_2(...)) >> test_3(...)

However, test_or doesn't return the state object, so attempts to chain from it in this way result in complaints about NoneType not having certain methods. It's possible to get around this in some cases by reversing the order of the tests (i.e., running test_3 before test_or), but sometimes test_3 only makes sense if one or the other preconditions passed.

Currently, when there's an AST misparse, then ast based SCTs are quietly skipped. This was done since we can't guarantee that for every parser will handle all student submissions handed to it. However, has_equal_ast can be adapted to work without a solution ast.

(In general, it is not clear if checking the submission and solution in an SCT is beneficial--if people will be using the exercise validator)

of form...

{
  'file1.py': "1 + 1",
  'file2.py': "2 + 2"
}

Changes Required

1. add sct check_file('file1.py') that gets file content, and (optionally) tries to parse it to an AST.
2. functions like test_student_typed, has_equal_ast give informative error if, in this case, used before check_file.

from feedback by @gvwilson

@machow commented on Wed Jan 03 2018

Summary of the issue

There are two images for a course

• course image: contains course specific libraries
• shared image: contains backends, SCT libs, etc...

When building a course image, even though SCT extensions depend on SCT libraries, they don't need to install them (since they're already on the shared_image). However, because the shared_image isn't mounted when building a course, tools like pip can't know this, and will install them by default.

This issue also arose in the shell course.

solutions

1. use the --no-deps flag, e.g. pip install --no-deps sqlwhat-ext
2. remove the relevant SCT library as a dependency in the , e.g. sqlwhat-ext package

(1) seems to make more sense from a package development point of view, but (2) probably is less risky, more annoying to develop with.