ucsd-edx / edx-micro-master-in-data-science Goto Github PK

prespecifies ami, ports, etc...
and then automatically launches a launching script

The cells that have to do with the HW and testing the HW should have the slide directive "skip".

Also, make the title above these sets of sequential cells a small heading (use #### in front of the heading). This way I can easily collapse these cells when I am working on other cells, using the TOC extension. (see my email).

Make assertions/exceptions in tests

either have multiple cells or partial credit in one cell

once an ami is launched the system will use a file called /etc/init/rc

There are two notebooks in the directory 0.MemoryLatency:

01_measuring performance: you don't need to do anything for this one

0_2 SimpleCount which compares the performance of simple counting and counting after sorting. Please use the same homework as we have for the wordcount using spark.

A pretty common problem is that the student is using a module that is too old.
To combat that I added the file 2.PCA/lib/import_modules.py that takes as input a list of modules and the minimal version, imports the modules, checks their version, and compares it to the given.

I recommend that we use that, instead of simple 'import' at the header of our notebooks.

To make the code easily accessible I suggest we have a /lib directory at the level of the notebook directories, in addition to a /lib for each subdirectory.

We should have no large files in the github data directory. Instead the students will download the files from the web using urllib. See how I did it in the two wordcount notebooks.

surround the grading process with os.fork .
Do one student at a time
set a time limit to run grading task
make sure time limit is longer than nbgrader time out

we use s3
cli will be the tool to copy from instance to s3

1: latency graphs
2: word counts with and without sorting