4. Combining data CSVs
   rubric={correctness:10,reasoning:10}

• Use one of the following options to combine data CSVs into a single CSV.

• Pandas

• DASK

• When combining the csv files make sure to add extra column called "model" that identifies the model (tip : you can get this column populated from the file name eg: for file name "SAM0-UNICON_daily_rainfall_NSW.csv", the model name is SAM0-UNICON)

• Compare run times and memory usages of these options on different machines within your team, and summarize your observations in your milestone notebook.

Warning: Some of you might not be able to do it on your laptop. It's fine if you're unable to do it. Just make sure you check memory usage and discuss the reasons why you might not have been able to run this on your laptop.

3. Downloading the data
   rubric={correctness:10}

• Download the data from figshare to your local computer using the figshare API (you can make use of requests library).
• Extract the zip file, again programmatically, similar to how we did it in class.
  You can download the data and unzip it manually. But we learned about APIs, and so we can do it in a reproducible way with the requests library, similar to how we did it in class.

There are 5 files in the figshare repo. The one we want is: data.zip

2. Creating repository and project structure
   rubric={mechanics:10}

• Similar to previous project courses, create a public repository under UBC-MDS org for your project.
• Write brief introduction of the project in the README.
• Create a folder called notebooks in the repository and create a notebook for this milestone in that folder.

1. Team-work contract
   rubric={correctness:10}

Similar to what you did in DSCI 522 and DSCI 524, create a team-work contract. The contract should outline how you are committed to work together so that you are accountable to one another. Again, you may start with your team contract document from previous project courses and adapt it for your new team. It is a fairly personal document and please do not push it into your public repositories. Instead, save it somewhere your team can easily share it, and you can share a link to it, or a copy with us in your submission to Canvas to prove you did this.

5. Load the combined CSV to memory and perform a simple EDA
   rubric={correctness:10,reasoning:10}

• Investigate at least two of the following approaches to reduce memory usage while performing the EDA (e.g., value_counts).

• Changing dtype of your data
• Load just columns what we want
• Loading in chunks
• Dask

• Discuss your observations.

6. Perform a simple EDA in R
   rubric={correctness:15,reasoning:10}

• Pick an approach to transfer the dataframe from python to R.

• Parquet file
• Feather file
• Pandas exchange
• Arrow exchange

• Discuss why you chose this approach over others.