It's not possible to test whether arguments in a function are used, without checking whether they have the correct value (evaluated or un-evaluated)

test_with() works for some context managers, but it doesn't close properly in the SCT for others.

problem im having in this ex: https://campus.datacamp.com/courses/998/2202?ex=3

solution includes print(file.read())

if i switch it out with file.read() i get the error in attached screenshot

https://campus.datacamp.com/courses/1167/2629?ex=12

to demonstrate an issue that wasn't totally related to my course, I needed to create a branch and write an SCT there.

It would be great to have a place where we could write SCTs to demonstrate issues

@hugobowne @vincentvankrunkelsven
Currently, test_function() tests all arguments when you don't specify the args argument.
The more I'm looking at it, however, it seems to be clearer, more straightforward to right and more intuitive to have SCT authors explicitly say which arguments and which keywords they want to check.
What do you think?

For backwords compatibilty, we can go through the SCTs that are live today, make the keywords and args explicit, put those live, and then rewrite test_function().

need this as soon as possible. here is the mini-course index we need it for.
index.txt

https://campus.datacamp.com/courses/1167/2628?ex=17

#Test: call to scipy.io.loadmat() and 'mat' variable
test_correct(
    lambda: test_object("mat"),
    lambda: test_function("scipy.io.loadmat")
)

this issue will block beta testing. please all feel free to suggest workarounds!

in the following exercise, when the actual solution is submitted, it is NOT accepted AND an”Incorrect submission” is thrown (screenshot below):

https://campus.datacamp.com/courses/1167/2627?ex=13

The same occurs in here and a few more:

https://campus.datacamp.com/courses/1167/2627?ex=15

It's weird because there is not a problem in the following exercise and the SCT looks the same:

https://campus.datacamp.com/courses/1167/2627?ex=12

I think this is the SCT in question:

# Test: call to np.genfromtxt() and 'data' variable
test_correct(
    lambda: test_object("data"),
    lambda: test_function("numpy.genfromtxt")
)

All of this is currently on @franciscastro 's branch: https://github.com/datacamp/courses-importing-data-in-python/tree/review-francis

context_vals is used for other things in other test functions. Change the naming.

Maybe generally rethink the naming of a lot of arguments.

https://campus.datacamp.com/courses/1167/2629?ex=16

this issue will block beta testing. please all feel free to suggest workarounds! i could just test_object, if necessary?

https://campus.datacamp.com/courses/1167/2630?ex=2

involves IPython magic command ! ls:

@franciscastro can't seem to write his own error message for test_object_after_expression: setting incorrect_msg doesn't change anything. also, default message has a capital 'A' after the comma, which should be lower case (screenshot). this occurs in other cases also.

hey guys! we're trying to reproduce something like this is pythonwhat": i.e., being able to provide tailored feedback for each argument passed to a function

we can't get it to work yet. have tried the following + other stuff. How would you do it?

I will beta test without it but would be great because makes student experience far better ;-)

tagging @franciscastro

With IPython, you call execute Shell commands by prefacing them with !, e.g. ! ls for directory contents.

I use this in my course and Hamel will use it in his Cleaning Data in Python course.

We will need some test functions for this.

SCT needed for user-defined generator functions such as

# Create a list of strings
lannister = ['cersei', 'jaime', 'tywin', 'tyrion', 'joffrey']

# Define generator function get_lengths
def get_lengths(input_list):
    """Generator function that yields the
    length of the strings in input_list."""

    # Yield the length of a string
    for person in input_list:
        yield len(person)

User-defined functions already good so maybe just yield necessary?

https://campus.datacamp.com/courses/importing-data-into-python/introduction-and-flat-files?ex=13

(issue w/ line 7)

the weird thing is that the exact same SCT works perfectly in this exercise:

https://campus.datacamp.com/courses/importing-data-into-python/introduction-and-flat-files?ex=16

Starts at 1 now; 0 makes more sense.

can we test the definition and use of lambda function?

E.g.

# Define echo_word as a lambda function: echo_word
echo_word = (lambda word1, echo: word1 * echo)

and

# Create a list of strings: spells
spells = ['protego', 'accio', 'expecto patronum', 'legilimens']

# Use map() to apply a lambda function over spells: shout_spells
shout_spells = map(lambda item: item + '!!!', spells)

Here, the solution code is
file = open('moby_dick.txt', 'r') and the SCT is
test_function("open", incorrect_msg="Pass the correct arguments toopen().")

The SCT accepts the solution code but does NOT accept
file = open('moby_dick.txt', mode='r') so there is something funky happening with keyword arguments.

If I change solution code file = open('moby_dick.txt', mode='r') , then the other one isn't accepted.

The only way I can get it to work is making solution code file = open('moby_dick.txt', mode='r') and using the SCT

test_correct(
    lambda: test_object("file", do_eval=False, undefined_msg="You don't have to change any of the predefined code."),
    lambda: test_function(
                "open",
                not_called_msg="You don't have to change any of the predefined code.",
                incorrect_msg="Did you pass the correct arguments to `open()`?"
            )
)

This SCT doesn't seem to work for the keyword arg if solution is positional.

My workaround seems to be working but we should figure out in detail the keyword/positional business in SCTs.

test_function is not working for the sqlalchemy function Table(). To be precise, the solution code is NOT accepted. This is what the SCT responds:

this is the exercise and this is the repo branch

https://campus.datacamp.com/courses/importing-data-into-python/interacting-with-apis-to-import-data-from-the-web?ex=16

thought this may be a product issue but if i remove SCT, it works fine

https://campus.datacamp.com/courses/1167/2629?ex=8

it's weird because this error doesn't show up in any other SQL exercises.

https://campus.datacamp.com/courses/1167/2629?ex=4

SCT is currently this:

# Test: call to create_engine() and 'engine' variable
test_correct(
    lambda: test_object("engine"),
    lambda: test_function("sqlalchemy.create_engine")
)

see screenshot for error:

could it be a ploblem with importing like this?

from sqlalchemy import create_engine

this is a standard import so a workaround would be great.

this issue will block beta testing. please all feel free to suggest workarounds!

Each time a function like test_function() runs a test, it gets back a result. If this result is False, the function will be exited, with not result. This works, but it requires you to explicitly do this after each test. It would be better if asesrtions fail with a certain Error, so that it can be handled with exception handling. (SCTException).

SCT needed for generator expressions such as

result = (num for num in range(31))

I have tried to reproduce this

*** =sct

test_student_typed(/sum\s*\(\s*range\s*\(/, not_typed_msg="You didn't use `range()` inside `sum()`.")
success_msg("Great job!")

from this page and it doesn't work.

See here and here.

I tried by putting quotations marks around the regex and still doesn't work

datacamp/testwhat#32

In https://campus.datacamp.com/courses/1113/2464?ex=3 ,

when we submit the solution code, python throws an error. this did not happen when SCT wasn't present.

see screenshot:

If you use do_eval in test_function(), the ast objects are compared. Would be better if the raw strings are compared.

According to these docs, the tests folder should be inside the package folder. That's also easier for the docker to test; currently the tests have to be fetched separately.

issue with test_function() on pickle.load() -- also had the issue using test_correct
https://campus.datacamp.com/courses/1167/2628?ex=3

this issue will block beta testing. please all feel free to suggest workarounds!

Make sure all files and tests use 4-space indentation

Implement test_with() for context managers

In this exercise, we need to test line 7

data_array = data.values

and test_function doesn't work.

How do we test assigning attributes to variables? :-)

Tagging @franciscastro

hi :-)

I am able to replicate the round example with multi-args here

However, even in MY simplest cases, I cannot seem to get the multi-arg SCTs to work :-(

• This exercise with open (the SCT responds 'first arg wrong!' when 1st correct and 2nd wrong): here is the branch
• This exercise wrt np.loadtxt (solution code now not accepted: 'list index out of range'): here is the branch

Let's say that the solution code has

tables = connection.execute(stmt).fetchall()

How would we test that the correct methods are applied in the correct order? Can test_function do this?

In our sqlalchemy course, this will be necessary in more than 50% of the exercises.

Note: my intern Vincent is currently working on these SCTs and has tried test_student_typed but it isn't quite working. See this issue. any suggestions?

datacamp is throwing an error here wrt parameter passed to def:

https://campus.datacamp.com/courses/1113/2464?ex=4

screenshot attached

can we test raising errors in user-defined functions?

First example would be using try-except. see here for more details:

https://docs.python.org/3/tutorial/errors.html

Now, 'on line x and column y' or something similar is included in automatically generated feedback messages; this should be implicit through highlighting in the editor.

https://campus.datacamp.com/courses/importing-data-into-python/introduction-and-flat-files?ex=3

i thought this may be product issue but I removed SCT and it works fine then

If the student submits code that contains a syntax error, the student will get to see rather cryptic Python interpreter information, that is not necessarily helpful for the student.

Similar to testwhat, we'll want a tool that wraps around syntax error, providing more diagnostics.
This is a tough nut to crack.

We'll need to test comprehensions (list/dict/set); e.g. for

doctor = ['house', 'cuddy', 'chase', 'thirteen', 'wilson']

we will need an SCT for

x = [doc[0] for doc in doctor]

Also conditional comps such as

x = [doc[0] for doc in doctor if doc[1] == 'h']

and nested comps! ;-)

Related to issue posted by @hugobowne
#31

Python Magic Commands, such as ! ls are not supported due to ast, used for parsing.

This is a pain in the ass to support, so will take a long time.

hi! i love that you can test_function within test_function_definition: can we also test objects within the scope?

@franciscastro tried but to no avail yet. If so, could you possibly provide a quick example of usage in the case of this exercise:

https://campus.datacamp.com/courses/1113/2464?ex=3
thanks!

https://campus.datacamp.com/courses/1167/2629?ex=7

this issue will block beta testing. please all feel free to suggest workarounds!

hi!

interesting issue: if student writes nothing after def (): within scope of function, then Python throws an error and SCT doesn't work. See screenshot. any cheeky work-around?

if I import numpy as np, I need to test_function("numpy.array",...) for example, and change every message to say np, instead of numpy.

Is it possible to have `test_function("np.array",...) ?

I checked out your work-around from Intro_python: