Make test_function robust to args vs keywords about pythonwhat HOT 7 CLOSED

@hugobowne This is tricky stuff.
test_function() handles arguments and keywords differently: arguments are specified by position, keywords are specified by name.
So if you use open('moby_dick.txt', 'r') in your solution, test_function() will see that there are two positional arguments, with indices 0 and 1. It then looks for those same arguments in the student code; if the student specified the args in the same way, it's all good, if it did so with open('moby_dick.txt', mode = 'r'), the student used a keyword and an argument, and thus fails.

There's no easy fix for this. In R (testwhat), there's the match.call function, that 'standardizes' a function call, so that you don't have to figure this all out yourself. For Python, something like that does not exist, and it is very hard to build something like this.

Your workaround with test_correct() is elegant, but you're right that test_function() by itself should be robust to different ways of calling a function.

from pythonwhat.

@filipsch

there are other issues i just realised. To recap:

This is the solution code: file = open('moby_dick.txt', mode='r') -- I need to write an SCT that accepts it and also accepts file = open('moby_dick.txt', 'r').

We have a workaround mentioned above. However, this workaround also accepts
file = open('moby_dick.txt'), which is not ideal. I tried playing around with test_student_typed but there were issues there, as raised in in this issue.

Do you have any suggestions for this?

Could we use some cool multi-arg tests once we sort out this issue?

I could see us using a test_or on specific arguments to allow both 'r' and mode='r' as 2nd arg, in this case. The problem is this would not scale well with many args.

As you realise, this is an issue that occurs a lot in all courses.

Do you have any thoughts on this?

from pythonwhat.

@filipsch see comment/question above. any suggestions? sorry if this has been answered in another issue. I'll happily also raise this as another issue if you want me to

from pythonwhat.

There are a lot of question in the message of 9 days ago.

• The issue with test_student_typed() is resolved, so you could try to use that.
• You cannot use test_or() because test_function() still looks for corresponding function calls in the solution code; this would mean that you have to list the two ways of opening the file explicitly in the solution code, which is not what you want.
• The multi-arg custom feedback is working now (although the way of writing is suboptimal), but I'm not sure if it's relevant here.

The big problem is: pythonwhat sees a difference between arguments and keywords, while Python allows you to use both in many cases, and that's a huge pain. I'm working on a test_function_v2 wher e you have to list the arguments expicitly, and where there is robustness against different ways of specifying arguments. As soon as that is there, it'll all be implicit when you simply use test_function_v2.

from pythonwhat.

@hugobowne A first version of functionality that matches arguments and keywords in a robust way, is now available on staging, soon on master. The function is called test_function_v2. If still have to write documentation for it on the wiki.

from pythonwhat.

test_function_v2() has been documented here, function will be on production later today.

from pythonwhat.

It's live. Read about it here

from pythonwhat.