An automated ML pipeline to forecast hourly electricity demand for the PJM balancing authority.

The goal of this project is to demonstrate:

• ML Ops
  • ML workflow orchestration.
  • Versioning:
    • Dataset versioning.
    • Model versioning and experiment tracking.
  • ETL Pipeline.
  • Reliability: Pipeline performance visibility and alerting.
  • Isolation between development and deployed environment infrastructure.
  • Model performance comparisons to baseline.
  • Hyperparameter tuning.
• ML
  • Timeseries feature engineering.
  • Timeseries forecasting with XGBoost.
  • Timeseries cross validation.

EIA Open Data's hourly electricity demand data.

• Dev Env: Docker Compose
• ML Workflow Orchestration: Prefect
• Experiment tracking: MLFlow
• Dataset version tracking and 'Data Warehouse': DVC (+ git repo)
• Model: XGBoost

• Géron, Aurélien. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. "O'Reilly Media, Inc.", 2022.
• Rob Mulla's Kaggle tutorial on timeseries forecasting with XGBoost