This repository contains Docker and Docker Compose files for building and running Apache Airflow 2.0 as described in our Medium article.

• Docker
• Docker Compose

For simple usage and running with no additional dependencies use running with the official image. If additional configuration is needed use custom image running instructions below.

make run

• Configure Airflow running parameters (.env)
• Configure Dockerfile-dev and docker-compose.dev.yml
• Add dependencies in requirements.txt

make run-dev

• Airflow UI: localhost:8080
  • Username: admin
  • Password: admin

Note: username and password can be configured in /scripts/entrypoint.sh

It's possible to set any configuration value for Airflow from environment variables in .env file where we can set airflow running variables to override the default configuration that is generated as airflow.cfg file after the initial run.

The general rule is the environment variable should be named AIRFLOW__<section>__<key>, for example AIRFLOW__CORE__SQL_ALCHEMY_CONN sets the sql_alchemy_conn config option in the [core] section.

Check out the Airflow documentation for more details

You can also define connections via environment variables by prefixing them with AIRFLOW_CONN_ - for example AIRFLOW_CONN_POSTGRES_MASTER=postgres://user:password@localhost:5432/master for a connection called "postgres_master". The value is parsed as a URI. This will work for hooks etc, but won't show up in the "Ad-hoc Query" section unless an (empty) connection is also created in the DB