Git Product home page Git Product logo

ansible_airflow's Introduction

Ansible : Playbook Airflow

The aim of this project is to deploy a simple Airflow instance on Vagrant with a Postgresql backend.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

What things you need to run this Ansible playbook :

  • Vagrant must be installed on your computer
  • Update the Vagrant file based on your computer (CPU, memory), if needed
  • You must have download the ubuntu Xenial64 vagrant box :
$ vagrant box add https://app.vagrantup.com/ubuntu/boxes/xenial64

Usage

A good point with Vagrant is that you can create, update and destroy all architecture easily with some commands.

Be aware that you need to be in the Vagrant directory to be able to run the commands.

Build Environment

Vagrant needs to init the project to run and build it :

$ vagrant up

After build, you can check which virtual machine Vagrant has created :

$ vagrant status

If all run like it is expected, you should see something like this :

$ vagrant status

Current machine states:

airflow-web01                     running (virtualbox)

Deployment

This playbook has some dependencies to other roles that must be downloaded before executing the playbook :

$ ansible-galaxy install -r requirements.yml

This command should download the Pip role from Wikitops Github account to the local role path.

To deploy the Airflow instance, you just have to run the Ansible playbook airflow.yml with this command :

$ ansible-playbook airflow.yml

If everything run as expected, you should access Airflow Web interface : http://10.0.1.81:8080/

Destroy

To destroy the Vagrant resources created, just run this command :

$ vagrant destroy

Author

Member of Wikitops : https://www.wikitops.io/

Licence

This project is licensed under the Apache License, Version 2.0. For the full text of the license, see the LICENSE file.

ansible_airflow's People

Contributors

wikitops avatar

Watchers

 avatar

ansible_airflow's Issues

Ansible Playbook Fails on Task "Install prerequisites packages"

Error while running ansible-playbook airflow.yml. Missing file?

TASK [airflow : Install prerequisites packages] ******************************************
fatal: [10.0.1.81]: FAILED! => {"msg": "Failed to set permissions on the temporary files Ansible needs to create when becoming an unprivileged user (rc: 1, err: chown: invalid user: ‘{'failed': False, u'stat': {u'exists': False}, u'changed': False}’\n}). For information on working around this, see https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user"}

If I look into the yaml files I can see the task is located in roles/airflow/tasks/users.yml

Looking at line 5 I don't see this file anywhere. (add_credentials.py)

path: /opt/airflow/add_credentials.py

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.