A project to practce Pyspark
- Create a virtual python environment
- Install dependencies from requirements.txt
pip install -r requirements.txt
- If you add any additional dependencies, you can install them with the following command:
pip freeze -l > requirements.txt
- MovieLens - Currently utilizing MovieLens 25M Dataset in this project
- Registry of Open Data on AWS
- Open Library of Books
- Publicly Available Big Data Sets
- Quora
- Spark Docs
- [Old Spark Docs](https://spark.apache.org/docs/3.0.1/api/python/pyspark.sql.ht