This is a companion project for the post ETL Job Processing with Serverless and AWS Redshift.
Setting up AWS Redshift is out of scope of this sample, but we need one set up to dump data into it from our ETL job. Once you have it set up and configured, keep the cluster endpoint in Redshift handy, as we will need it later to configure the database connection string.
<cluster_name>.xxxxxxxxxxxx.<region>.redshift.amazonaws.com:<port>
postgres://<username>:<password>@<hostname>:<port>/<db_name>
where <hostname>:<port>
is the cluster endpoint
$ pip install -r requirements.txt
$ sls plugin install -n serverless-python-requirements
This will install the plugin and will add the plugins
section in the serverless.yml
file.
sls invoke local -f etlSample
sls invoke -f etlSample