Ofili Lewis's Projects
This repository contains the lab files and other resources for the free Microsoft course DAT207x: Analyzing and Visualizing Data with Power BI. To learn how to connect, explore, and visualize data with Power BI, sign up for this course on edX.
This project builds a data pipeline to populate the user_behavior_metric table. The user_behavior_metric table is an OLAP table, meant to be used by analysts, dashboarding.
Simple ETL script to migrate data from BigQuery to Postgres database.
Free Data Engineering course!
This project builds an ETL pipeline for a data lake hosted on S3. We will load data from S3, process the data into analytics tables using Spark, and load them back into S3. We will deploy this Spark process on a cluster using AWS.
This project implements a data ingestion and processing pipeline to collect, store and process time-series data. The pipeline consists of a publisher, a message queue (Pub/Sub), a consumer, a data warehouse (BigQuery) and a data extractor. The pipeline is designed to be scalable, efficient and easy to maintain.
This project creates an ETL pipeline to build a data warehouse hosted on Redshift.
Data Cleaning in Python Essential Training
This project builds a data pipeline that ingests Sparkify's music data into an AWS Redshift Data Warehouse. The ETL pipeline will be run on an hourly basis, scheduled using Airflow.
The workflow brings the data into a PostgreSQL database.
jetbrain software全家桶激活码activation code, including intellij idea,pycharm,datagrip, webstorm...
jetbrain software全家桶激活码activation code, including intellij idea,pycharm,datagrip, webstorm...
This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
This script performs automated backups of a MySQL database using the mysqldump command. It allows you to schedule daily backups and provides a notification upon successful completion.
This etl pipeline extracts and integrates NYC Taxi Trip Data with Taxi Zone Lookup Data to create a dataset that can be used for descriptive and predictive analysis. For example, to predict the number of trips per day for a given taxi zone.
Structured Streaming app that can read files from the local system folder as new files are added to the folder as stream data and apply all the operations on the new data and, finally, write the results in an output directory.
The Real-Time Analytics Platform is a robust and scalable solution designed to ingest, process, and visualize high-volume data streams in real-time.