Library of docker-compose setups for local development and learning:
./elasticsearch/
- Single node elasticsearch with XPACK enabled. Anonymous user is enabled with http ssl. Certs are created automatically in the manifest.
./pulsar/
- Standalone pulsar with the pulsar management UI
./dagster/
Dagster with:
- cron scheduler
- postgres db
- rabbitMq
- 1 celery dagster worker
- dagit
http://localhost:15672/
username: guest
password: guest
broker: pyamqp://guest:guest@rabbitmq:5672//
http://localhost:3000/
user: dagster
password: dagster
database: dagster
host: localhost
Port: 5432
Turn it on:
cd dagster
docker-compose up
Turn it off:
docker-compose down
Drop your code in the app folder and update the workspace yaml accordingly to point to your pipeline python file.