Git Product home page Git Product logo

provy's Introduction

WARNING

This project is not maintained anymore.

provy

provy is a provisioning service in python.

Status of the project

Build Status Coverage Status

Examples

For an example script check the test provyfile.py.

Quick start

To run the provyfile script in the command-line you use::

provy -s prod                   # provisions all prod servers
provy -s prod.frontends         # provision all front-end servers in prod
provy -s prod.frontends.server1 # provision only server1

This command will provision all the webservers described under webservers (-s) with the webserver role (-r).

Documentation

For more documentation on how to use it, go to the provy page

provy's People

Contributors

avelino avatar diogobaeder avatar douglas avatar fernandogrd avatar geospark avatar geovanisouza92 avatar heynemann avatar jbzdak avatar michaelheyvaert avatar rafaelcaricio avatar rcmachado avatar renatogp avatar slyons avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

provy's Issues

ensure_line

Create a method that makes sure that a given line is present in a given file.

self.ensure_line('127.0.0.1 localhost', '/etc/hosts')

What with cd

I added parameter cwd to execute since often I wanted to perform operations in specific directory, but fabric already has cd context manager.

I might drop this argument and put comment somewhere in this code about use of fabric's context manager.

Create a BuildbotRole

Create a BuildbotRole to provide a basic Buildbot CI installation (maybe defining master and slave config files to be put in the server as well)

RedisRole

self.provision_role(RedisRole)

The server should be installed with aptitude.

If aptitude does not create it, the role should create a /etc/init.d/redis init script that gets auto-started.

Provy chokes on system that uses sudo lecture

I'll be using provy to manage student's laboratory, so I gather that my requirements may be bizzare, and problem unforseen.

I set up sudoers to lecture users always, and provy chocked on that.

Basically everywhere sudo output is inspected we get lecture string before actual command output.

So for example:
self.execute('test -d %s; echo $?' % file_path, stdout=False, sudo=True) == '0'

will fail always because output is::
<>
0

Any input how I can fix that (other than temporary fix of disabling sudo lecture for provisioning user --- which is more complicated than it seems ;) ).

Allow users to specify console-query arguments

provy should ask the user on the console (or have it passed as arguments) some arguments.

Say we need a production server password. We won't have it in the provyfile.py open so anyone can see. So what we'd do is:

servers = {
    'frontend': {
        'address': '33.33.33.33',
        'user': 'vagrant',
        'roles': [
            FrontEnd
        ],
        'options': {
            'user': 'frontend',
            'db-password': AskFor('db-password', "Please input the database password")
        }
    }
}

What is role of "owner" context variable

Is it OK to place code like this in most of the role functions:

if owner is None:
       owner = self.context.get('owner', None)

such snippets were present on some functions, and I put it in some more, in this case I could log in to remote as user X, and put have every file I change be labeled as user Y.

When I was provisioning my web server it seemed a good idea at the time, but now I'm not sure.

MySQL role bug

Whenever I ask MySQL Role to assign grants to an user at a given location (127.0.0.1, for instance), it says the user does not have the grants and apply them again.

I'm not sure why this is happening, but it should assign grants only once and the next time verify correctly that my user has those grants.

DjangoRole

with self.using(DjangoRole) as role:
    with role.ensure_site('mysite') as site:
        site.path = '/some/folder/with/settings.py'
        site.threads = 4
        # settings that override the website defaults.
        site.settings = {

        }

The website should be added to /etc/init.d/website and should be started using gunicorn_django.

PostgreRole

Someone that is more familiar with postgre should specify this one.

ApacheRole

Create a role similar to nginx role for apache.

Allow per server options

Allow users to pass options that will be in the context when a given server is being provisioned:

servers = {
"server1": {
options: {

    }
}

}

These have precedence over defaults.

Bundles

What I envision as recipe is like a bundled parametric provyfile.

What I mean is to have higher level bundles that configure a server.

A good example:

DjangoBundle.configure_site(
name="my-web-app",
git_repo="my.git",
workers=10
)

And then my django application is configured. We would find the settings.py file or specify one, like settings_provy.py. Any convention works for me, as long as I can get a django app up and running in minutes, instead of hours.

This ticket is NOT about creating a django bundle, though. It's about creating the infrastructure for bundles. Not sure what that is, so it might be a good idea to just choose one example (django sounds good since it's useful to so many people).

VirtualEnv Role and support in other roles

Add VirtualEnvRole and support in pip, django, tornado and supervisor roles.

Using it should be as easy as:

with self.using(VirtualEnvRole) as venv:
    venv.name = 'something'
    with self.using(PipRole) as pip:
        pip.ensure_package_installed('provy') # installs to virtual env something

NPM Package Role

Create a role that installs packages from npm, like aptitude or pip's.

Advanced Server configuration

I'm interested in using Provy to manage and deploy a number of servers where I work, but there are a number of configuration options that are missing that prevent from doing so.

Most notably:

  • Server SSH port number
  • Identity key file

There may be some other features that Tav writes about in this article that may be of some value.

update_file throws encoding errors when using UTF-8 encoded text

Codec is thrown from write_to_temp_file.

I have really no idea how to fix it --- since updated files can be encoded in arbitral encoding, so making this code error proof can be hard. In most of my code I circumvented this issue by passing StringIO to fabric function that would put data on remote server (which will work with unicode).

I could fix it by omitting write_to_temp_file and use StringIO instead, or just encode everything in write_to_temp_file as UTF-8 (which will work for ascii files).

Django Provisioning

As of now the django install kinda works. I think we should make a couple adjustments:

a) Improve the way you can override settings for each environment (currently it's a little buggy).
b) Improve the way we can pass environment variables to be set when running gunicorn
c) Drastically diminish the number of available configuration options.

The message here is it should be simple to host a django website.

Java Role

Create a role that installs java on the server.

SupervisorRole not respecting config owner

In the end of the provisioning, SupervisorRole doesn't set the config file to the proper owner set in the config() method (as "user" argument). The current behavior is that it's using the current provy user/owner.

Allow users to put folders

An user should be able to put a folder.

When that happens, provy must:
a) Compare local to remote, file by file;
b) If file exists locally but not remotely, copy it;
c) If file does not exist locally, but exists remotely, do nothing (on risk of losing stuff);
d) If file exists on both, but md5 differ, update it;

This way users can still use templates when putting folders.

MongoDB Role

with self.using(MongoDbRole) as role:
    role.database_path = '/tmp/db'
    role.number_of_services = 4
    role.starting_port = 20000

    role.replica_set = 'some-replica-set'
    role.master_port = 20000

If any new configurations come up in the future we'll add.

Create a JenkinsRole

Create a JenkinsRole to provide a basic Jenkins CI installation (possibly some CI jobs as well, but this would be probably far harder than the basic installation, as a Jenkins job can require lots of configuration)

Add provy to travis-ci

Add our project to travis-ci ( http://travis-ci.org/ ), so that we have it continuously build on pushes.

This has to be done by /heynemann because only the project owner can enable it in travis.

MySQL Role

Create a role for provisioning a MySQL Server:

with self.using(MySQLServerRole) as role:
role.ensure_user_present('my-user', identified_by='my-pass')
role.ensure_database_present('some_data_base', owner='my-user')

PHPRole

Create a role that configures php.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.