Git Product home page Git Product logo

pulsar's Introduction

Pulsar Gem Version CircleCI Coverage Status

The easy Capistrano deploy and configuration manager.

Pulsar allows you to run Capistrano tasks via a separate repository where all your deploy configurations are stored. Once you have your own repository, you can gradually add configurations and recipes so that you never have to duplicate code again.

The way Pulsar works means that you can deploy without actually having the application on your local machine (and neither have all your application dependencies installed). This lets you integrate Pulsar with nearly any deploy strategy you can think of.

Some of the benefits of using Pulsar:

  • No Capistrano configurations in the application code
  • No need to have the application locally to deploy
  • Every recipe can be shared between all applications
  • Can easily be integrated with other tools
  • Write the least possible code to deploy

DISCLAIMER: Understanding Capistrano is strongly suggested before using Pulsar.

Capistrano support

This version of Pulsar (version >= 1.0.0) only supports Capistrano v3. If you're looking for Capistrano v2 support you can use Pulsar version 0.3.5 but, take care, that version is not maintained anymore.

Installation

The most useful way of installing Pulsar is as a system gem:

$ gem install pulsar

This will install the pulsar command which will be used to for everything, from running Capistrano to listing your configurations.


The first thing you'll need to do is to create your own configuration repo:

$ pulsar install ~/Desktop/pulsar-conf

This will create a basic starting point for building your configuration repository. As soon as you're done configuring you should consider transforming this folder to a git repository.

You can have a look at how your repository should look like by browsing the Pulsar Conf Demo.

NOTE: Pulsar only supports git.

Configuration

This is an example repository configuration layout:

pulsar-conf/
  |── Gemfile
  ├── Gemfile.lock
  ├── apps
  │   ├── Capfile
  │   ├── deploy.rb
  │   └── my_application
  │       ├── Capfile
  │       ├── deploy.rb
  │       ├── production.rb
  │       └── staging.rb
  └── recipes
      ├── generic
      │   ├── maintenance_mode.rake
      │   ├── notify.rake
      │   └── utils.rake
      ├── rails
      │   ├── passenger.rake
      │   ├── repair_permissions.rake
      │   ├── symlink_configs.rake
      │   ├── unicorn.rake
      │   └── whenever.rake
      └── spree_1
          └── symlink_assets.rake

Pulsar uses these files to build Capistrano configurations on the fly, depending on how you invoke the pulsar command. Since Pulsar it's basically a Capistrano wrapper, the content of these files is plain old Capistrano syntax.

apps directory

This directory contains your application configurations. You'll have one directory per application.

  • Capfile is the generic Capfile shared by all applications
  • deploy.rb has configurations that are shared by all applications
  • my_application/Capfile is the Capfile that will be used for this particular application
  • my_application/deploy.rb includes configuration shared by every stage of the application
  • my_application/staging.rb and my_application/production.rb files include stage configurations

recipes directory

This directory contains your recipes. You can create any number of directories to organize your recipes. In Capistrano v3 fashion, all the files are plain old rake tasks (make sure to name them with the .rake extension).

The recipes contained in this folder are always included in each stage for each application.


Another way to include your recipes is by using the Gemfile. Many gems already include custom recipes for Capistrano so you just need to require those. An example with Whenever:

#
# Inside Gemfile
#
gem 'whenever'

#
# Inside some Capfile (either generic or application specific)
#
require 'whenever/capistrano'

#
# Inside some .rb configuration file (either generic or application specific)
#
set :whenever_command, "bundle exec whenever"

Loading the repository

Once the repository is ready, you'll need to tell Pulsar where it is. The repository location can be specified either as a full git path or a GitHub repository path (i.e. gh-user/pulsar-conf).

Since Pulsar requires the repository for everything, there are multiple ways to store this information so that you don't have to type it every time. You can also use local repository, which is useful while developing your deployment.

You have three possibilities:

  • -c command line option
  • PULSAR_CONF_REPO environment variable
  • ~/.pulsar/config configuration file

The fastest way is probably the .pulsar/config file inside your home directory:

#
# Inside ~/.pulsar/config
#
PULSAR_CONF_REPO="gh-user/pulsar-conf"

#
# Also supported
#
# PULSAR_CONF_REPO="git://github.com/gh-user/pulsar-conf.git"

Pulsar will read this file and set the environment variables properly.


If you don't want to add another file to your home directory you can export the variables yourself:

#
# Inside ~/.bash_profile or ~/.zshrc
#
export PULSAR_CONF_REPO="gh-user/pulsar-conf"

Usage

After the repository is ready, running Pulsar is straightforward. You can either list your applications or build a configuration and run Capistrano on it.

Deploy

Running the deploy command really means running Capistrano on a certain configuration.

To deploy my_application to production:

$ pulsar deploy my_application production

The above command will fetch the Pulsar configuration repository, run bundle install, combine the generic Capfile and deploy.rb files with the my_application specific ones and add specific production.rb stage configuration. At last it will run cap deploy on it.

Listing applications

Pulsar can fetch your configuration repository and list the application and stages you can run deploys on:

$ pulsar list

This will fetch the Pulsar configuration repository and list everything that's inside, like this:

my_application: production, staging
awesome_startup: dev, production, staging

Execute arbitrary Capistrano tasks

You can launch any Capistrano task via task command. You can also pass arguments in Rake style (i.e. via square brackets after task name)

$ pulsar task my_application staging mycustom:task[argumen1,argument2]

or via environment variables.

$ TASK_ARG1=arg1 TASK_ARG2=arg2 pulsar task my_application staging mycustom:task

Integrations

Pulsar is easy to integrate, you just need access to the configurations repository and the ability to run a command.

Right now there are no integrations for Pulsar v1 but there are some built for the old v0.3 version that can be used as an example.

Chat Bots

Pulsar REST API service

Pulsar REST API is a service to provide a REST API for executing pulsar jobs.

Here is a real-life example of how you can integrate and simplify your Pulsar workflow.

About

Nebulab

Pulsar is funded and maintained by the Nebulab team.

We firmly believe in the power of open-source. Contact us if you like our work and you need help with your project design or development.

pulsar's People

Contributors

bravosimone avatar davidedistefano avatar fievelk avatar gitter-badger avatar kennyadsl avatar lukaszkorecki avatar minasmazar avatar mtylty avatar njam avatar octoxan avatar vogdb avatar zigomir avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pulsar's Issues

Why are environments limited to only dev/stag/prod?

https://github.com/nebulab/pulsar/blob/master/lib/pulsar/commands/main.rb#L20 explicitly lists the allowed values for ENVIRONMENT. We're in the situation where we deploy the same application many times over, and currently we use capistrano's stages for this. So we have stages called: demo-staging, demo-production, spacely-staging, spacely-production, etc.

Instead of limiting to three specific stage names, wouldn't it be more logical to limit to whatever stages are defined for that application in the config repo?

The error message given is also really confusing, as a user you simply get:

$ pulsar -v roqua demo-staging --tasks
ERROR: parameter 'ENVIRONMENT': ArgumentError

Add --all option

Add a --all option that makes it easy to deploy (or run a command) on every configured app.

Something like:

pulsar cap -c nebulab/conf --all deploy

We should also consider if this is useful/feasible:

pulsar cap -c nebulab/conf --all --only-sinatra deploy

pulsar cap -c nebulab/conf --all --only-rails deploy

pulsar cap -c nebulab/conf --all --only-rails-3 deploy

pulsar cap -c nebulab/conf --all --only-rails-3-1 deploy

Multiple deploys

Pulsar should support multiple concurrent deploys.

Right now I don't think it does, the /tmp directories where the conf_repo is downloaded would clash when multiple deploys are triggered.

Uploading application configuration files

This is more of a questions/request for best practice.

I thought I will put my application's configuration file config.ini into the apps folder of the pulsar configuration. This file contains database configuration, ip addresses etc.

I stored the files like this:

├── apps
│   ├── fuboo
│   │   ├── configs
│   │   │   ├── production
│   │   │   │   └── config.ini
│   │   │   └── staging
│   │   │       └── config.ini
│   │   ├── defaults.rb
│   │   ├── production.rb
│   │   └── staging.rb
│   └── base.rb

To upload them I use something like this in my capistrano recipe:

top.upload("apps/#{application}/configs/#{stage}", "/path/on/server")

What do you think about this, does it make sense? Do you do something similar?

What I'm not too fond of are the usages of #{application} and #{stage} in the local path. These match the actual path rather randomly, i.e. the application name could theoretically also be different from the pulsar-application path.
What do you think about this? Is this considered convention that they always match the pulsar-paths?
Or would it make sense to provide these values from within pulsar to the recipes (something like pulsar-application and pulsar-environment)?

System-wide configuration

I'm installing pulsar on various linux/osx systems with puppet now.
Preferably these installations would have a configuration (of the repository) set.

Currently the only way I see to do this is to store a .pulsar file in the user's home-directory. For systems with various users this is not easily accomplished - I would need to store this file in each user's home-directory.

How about allowing to configure pulsar for a whole system, i.e. specifying the repo in a file like /etc/pulsar/pulsar.conf?
Would it be okay to hardcode /etc? On the other hand it would be good if it could observe a prefix (${prefix}/etc).

Again I'm really not so knowledgable about the ruby world, so I don't know how this could be done best. Can one pass options to gem install which could be used to define the prefix?
Or should we observe ruby's installation path by reading RbConfig::CONFIG["prefix"] - on my homebrew installation this would be /usr/local/homebrew/Cellar/ruby/2.0.0-p247.

What do you think?

Adding @tomaszdurka

Default pulsar configuration breaks tasks using rake

Default pulsar configuration (apps/base.rb) loads 'rake' recipe that defines :rake namespace.

On the other hand, capistrano uses rake as a command to run rake tasks. For example take a look at capistrano assets task:

task :precompile, :roles => lambda { assets_role }, :except => { :no_release => true } do
    run <<-CMD.compact
      cd -- #{latest_release} && 
      RAILS_ENV=#{rails_env.to_s.shellescape} #{asset_env} #{rake} assets:precompile
    CMD

We can see it calls #{rake}.

But when this task is ran via pulsar it is transformed into:

executing "cd -- /home/vk/www/quails/releases/20130729092556 && RAILS_ENV=production RAILS_GROUPS=assets #<Capistrano::Configuration::Namespaces::Namespace:0xa2c077c> assets:precompile"

As you see rake is being replaced by namespace object, and not the rake command.

This happens even if I set set :rake, "bundle exec rake" explicitly.

The only workaround I found is not to load pulsar's rake recipe.

P.S. many thanks for this great tool.

Capistrano 3 support

We should test how pulsar works with capistrano version 3. From what I understand probably the only thing that it's not working is the Capistrano::Logger levels.

I'll test some 💩 and come back to this issue.

Add a capistrano helper if pulsar is called from inside the application

There should be a Capistrano helper (like load_recipes) that lets you know when capistrano is called from pulsar from within a Rack application.

This way one could use, from inside the recipes, something like:

if inside_app?
  do_recipe_stuff
else
  puts "only supported from inside app!"
end

Pulsar does not create /tmp/pulsar directory

When I try to run pulsar i get following error:

cp: cannot create directory `/tmp/pulsar/conf-repo-2013-08-13-091924-s4537': No such file or directory
/home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/helpers/shell.rb:22:in `run_cmd': Command cp -rp /home/vagrant/Deployment /tmp/pulsar/conf-repo-2013-08-13-091924-s4537 Failed (RuntimeError)
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/helpers/clamp.rb:64:in `fetch_directory_repo'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/helpers/clamp.rb:53:in `fetch_repo'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:27:in `block (2 levels) in execute'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:218:in `block in with_clean_env'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:205:in `with_original_env'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:211:in `with_clean_env'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:25:in `block in execute'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:24:in `each'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:24:in `execute'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/clamp-0.6.1/lib/clamp/command.rb:67:in `run'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/clamp-0.6.1/lib/clamp/command.rb:125:in `run'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/bin/pulsar:5:in `<top (required)>'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/bin/pulsar:23:in `load'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/bin/pulsar:23:in `<main>'

If I create folder mkdir /tmp/pulsar that pulsar starts to work. Could pulsar automatically create this tmp folder?

Missing tests for load_recipes helper method

There are no tests in place for the load_recipes method.

This is due to the fact that the method is used in another process, that is when the actual capistrano command is run. That makes it difficult to test the load_recipes method and the actual capistrano run.

Is there an easy way to test a process run from a ruby command?

Ugly output for --help text

Running pulsar --help will output some really ugly text which has no real limit to the character count.

This issue should be fixed by adding a character limit of 78.

On command line output colors

Since I've seen so many of Pulsar's users using Pulsar with system provisioning software like Chef/Puppet, I was wandering if any of you is experiencing problems with command line output coloring.

Since I've started thinking about adding some more verbose output to Pulsar, I was wandering about arguments pro/against coloring of cmd tools.

Possible to run a task before it attempts to check if it can SSH?

I'm automating the management of keys on a server and I'm trying to auth the key before running the rest of the tasks, but Pulsar fails every time it goes to start if it doesn't have access to the server.

I've tried adding it to before "deploy:starting", "auth:key" but it still doesn't seem to run it before checking.

Any ideas?

Doesn't work when user has no permissions on PULSAR_HOME

When the PULSAR_HOME is not accessible for some reason Pulsar breaks down miserably 😅

This is caused by Dotenv's load method that raises and exception when it cannot read the env file.

This is bad because it should ignore the env file altogether in case somebody wants to use only the command line options to specify the configuration.

Configurable default task

Is it feasible to make the default task customizable?

E.g.

$ pulsar --help
Usage:
    pulsar [OPTIONS] APPLICATION STAGE [TASKS] ...

Parameters:
    APPLICATION                   the application which you would like to deploy. Pass a comma separated list to deploy multiple applications at once
    STAGE                         the stage on which you would like to deploy
    [TASKS] ...                   the arguments and/or options that will be passed to capistrano (default: "deploy")

If I invoke pulsar without TASK, capistrano starts with deploy

Is there anyway I could configure that or does this entail major code changes?

THX

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.