This project reads UW Madison grade distribution and course report PDF files (published by the UW Madison Office of the Registrar) and converts them into CSV or SQL dump files.
You will find published, update-to-date datasets at Kaggle.
The conversion process for a single term is as follows:
-
Open DIR report for the term.
a. Extract table from PDF (using tabula)
b. Read each row, adding new section per row.
c. Collate section info as necessary (i.e. 2 instructors for 1 single section)
d. Collate courses which appear to be cross-listed (based on similarity between sections offered)
-
Open grades report for the term.
a. Extract table from PDF
b. Read each row, add add each section grade data to course data added by the DIR report process
Typically all terms are extracted so this process repeats for each term.
Build it yourself with mvn clean install or grab a release from the releases page.
Usage: <main class> [options]
Options:
-d, -download
Download the PDF reports instead of extracting data
Default: false
-e, -exclude
Comma-separated list of term codes to exclude (ex. -e 1082)
-f, -format
The output format
Default: CSV
Possible Values: [CSV, MYSQL]
-l, -list
Output list of terms to extract
Default: false
-out, -o
Output directory path for exported files (ex. -o ../data)
Default: ./
-t, -terms
Comma-separated list of term codes to run (ex. -t 1082,1072)
Examples:
java -jar madgrades-final-1.0-SNAPSHOT.jar: will fetch every term and output files to the current directoryjava -jar madgrades-final-1.0-SNAPSHOT.jar -t 1082: will fetch just term 1082java -jar madgrades-final-1.0-SNAPSHOT.jar -o ../ -t 1082,1072: will fetch terms 1072 and 1082, output to../
The CSV or SQL dumps are in the format of a collection of relational entities modeled something like this:
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent