Daily COVID-19 data breakdown by zip code for the state of South Carolina.

Prior to July 22, I wasn't aware of the Open Data API, so I was generating data based on PDF exports from SC DHEC.

This runs with Node.js and dependencies are managed by NPM, which comes with Node.

1. Clone this repo and move into the newly-created directory: git clone [email protected]:davidcalhoun/covid-19-data-south-carolina.git && cd covid-19-data-south-carolina
2. Install dependencies by running npm i. This will create a node_modules directory, containing all package dependencies.
3. Take a look at the scripts section of package.json, which are available scripts you can run. For instance, to version bump dependencies automatically, you can run npm run bump.

• To update data/casesMerged.json with the most current data, run $ npm run download-from-api.
• To update and redeploy the map visualization, run $ npm run daily-update

All data exports live here. Prior to July 22, I was converting PDF exports into JSON from the SC DHEC updates - the original PDF is preserved here alongside its TXT and JSON translations.

• data/countyFormat - old exports in "County Format" (prior to April 10).
• data/casesMerged.json - data for all days merged, including quantile distributions for the map visualization. This quantile computation used to be performed clientside, but started to become sluggish as the dataset increased (especially noticeable on mobile devices).
• data/scZipMeta.json - zip code metadata via simplemaps.com. Necessary for per capita calculations.

There are several helper scripts to retrieve and process data. Some scripts are used daily, while others are more experimental or legacy.

1. daily-download-from-api.js - Downloads from the SC DHEC Open Data API and generates casesMerged.json.
2. daily-update.sh - updates and pushes the casesMerged.json file to Github, which automatically redeploys the map visualization
3. utils.js - shared utilities (for file reading, data manipulation, etc)

1. convertToZipFormat.js - (legacy) Translates data sorted by County into Zip code format. Needed for data PDFs prior to April 10
2. daily-download.js - (legacy) Automation to download PDFs from the Tableau dashboard.
3. merge.js - (legacy) Merges all JSON data files and outputs data/casesMerged.json
4. pdfToJSON.js - (legacy) (experimental WIP) - initial attempts to run OCR on PDFs, to avoid needing to use pdftotext.com
5. stats.js - (legacy) helper script to determine which zip codes are newly affected. Not used much now, as most all zip codes are affected.
6. txtToJSON.js - (legacy) converts TXT data to JSON format.