error in `test_function()` somewhere ? about pythonwhat HOT 1 CLOSED

@hugobowne I found the problem here, when you use test_function('print', 1), pythonwhat will find the first print() function, and see if its evaluated arguments are the same in the student and solution scope. In other words, pythonwhat is executing file.read() in both environments. However, it uses the file object at the end of the execution, which is a closed file. Hence the error message, thrown within the pythonwhat package.

What's important to remember here is that all evaluations in the pythonwhat package (for example evaluating the arguments of a function to compare them) are done in the ending scope, this means the scope of variables that we end up with after executing all the student/solution code. Doing tests on arguments/objects that are changed through the code (like the file object is changed by the file.close() method) can cause errors/strange behaviour. Same problem exists in testwhat package.

It's not easy to improve this and still have a performant testing package (it can be done, but it costs time and effort), hence I'll provide you with some workarounds:

• Use test_function("print", 1, do_eval = False), which will test the argument of print() on its text (== file.read()) instead of its value. Note that things like print( file.read() ) will still work though.
• Use test_expression_output() which will run the complete script in student and solution scope and assert that their value is the same.
• Use test_output_contains(/.../) fill in the dots with a part that you know should be in the text they have to print out.

Hope this helps

from pythonwhat.