nebulab / pulsar Goto Github PK

Add a --all option that makes it easy to deploy (or run a command) on every configured app.

Something like:

pulsar cap -c nebulab/conf --all deploy

We should also consider if this is useful/feasible:

pulsar cap -c nebulab/conf --all --only-sinatra deploy

pulsar cap -c nebulab/conf --all --only-rails deploy

pulsar cap -c nebulab/conf --all --only-rails-3 deploy

pulsar cap -c nebulab/conf --all --only-rails-3-1 deploy

Opening an issue because maybe others will have same question.

I didn't go through code to find out, but I am really interested if there is a way to add a file to .gitignore and test deployment with pulsar?

Default pulsar configuration (apps/base.rb) loads 'rake' recipe that defines :rake namespace.

On the other hand, capistrano uses rake as a command to run rake tasks. For example take a look at capistrano assets task:

task :precompile, :roles => lambda { assets_role }, :except => { :no_release => true } do
    run <<-CMD.compact
      cd -- #{latest_release} && 
      RAILS_ENV=#{rails_env.to_s.shellescape} #{asset_env} #{rake} assets:precompile
    CMD

We can see it calls #{rake}.

But when this task is ran via pulsar it is transformed into:

executing "cd -- /home/vk/www/quails/releases/20130729092556 && RAILS_ENV=production RAILS_GROUPS=assets #<Capistrano::Configuration::Namespaces::Namespace:0xa2c077c> assets:precompile"

As you see rake is being replaced by namespace object, and not the rake command.

This happens even if I set set :rake, "bundle exec rake" explicitly.

The only workaround I found is not to load pulsar's rake recipe.

P.S. many thanks for this great tool.

https://github.com/nebulab/pulsar/blob/master/lib/pulsar/commands/main.rb#L20 explicitly lists the allowed values for ENVIRONMENT. We're in the situation where we deploy the same application many times over, and currently we use capistrano's stages for this. So we have stages called: demo-staging, demo-production, spacely-staging, spacely-production, etc.

Instead of limiting to three specific stage names, wouldn't it be more logical to limit to whatever stages are defined for that application in the config repo?

The error message given is also really confusing, as a user you simply get:

$ pulsar -v roqua demo-staging --tasks
ERROR: parameter 'ENVIRONMENT': ArgumentError

Is it feasible to make the default task customizable?

E.g.

$ pulsar --help
Usage:
    pulsar [OPTIONS] APPLICATION STAGE [TASKS] ...

Parameters:
    APPLICATION                   the application which you would like to deploy. Pass a comma separated list to deploy multiple applications at once
    STAGE                         the stage on which you would like to deploy
    [TASKS] ...                   the arguments and/or options that will be passed to capistrano (default: "deploy")

If I invoke pulsar without TASK, capistrano starts with deploy

Is there anyway I could configure that or does this entail major code changes?

THX

We should test how pulsar works with capistrano version 3. From what I understand probably the only thing that it's not working is the Capistrano::Logger levels.

I'll test some 💩 and come back to this issue.

I'm automating the management of keys on a server and I'm trying to auth the key before running the rest of the tasks, but Pulsar fails every time it goes to start if it doesn't have access to the server.

I've tried adding it to before "deploy:starting", "auth:key" but it still doesn't seem to run it before checking.

Any ideas?

There should be a Capistrano helper (like load_recipes) that lets you know when capistrano is called from pulsar from within a Rack application.

This way one could use, from inside the recipes, something like:

if inside_app?
  do_recipe_stuff
else
  puts "only supported from inside app!"
end

We should investigate if parallel execution of Capistrano deployments is needed and worth doing.

When the PULSAR_HOME is not accessible for some reason Pulsar breaks down miserably 😅

This is caused by Dotenv's load method that raises and exception when it cannot read the env file.

This is bad because it should ignore the env file altogether in case somebody wants to use only the command line options to specify the configuration.

Right now we only output some error text and return a 0 exit status. This is wrong and we should return a non-zero status code on exit when something goes bad.

There are no tests in place for the load_recipes method.

This is due to the fact that the method is used in another process, that is when the actual capistrano command is run. That makes it difficult to test the load_recipes method and the actual capistrano run.

Is there an easy way to test a process run from a ruby command?

When I try to run pulsar i get following error:

cp: cannot create directory `/tmp/pulsar/conf-repo-2013-08-13-091924-s4537': No such file or directory
/home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/helpers/shell.rb:22:in `run_cmd': Command cp -rp /home/vagrant/Deployment /tmp/pulsar/conf-repo-2013-08-13-091924-s4537 Failed (RuntimeError)
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/helpers/clamp.rb:64:in `fetch_directory_repo'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/helpers/clamp.rb:53:in `fetch_repo'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:27:in `block (2 levels) in execute'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:218:in `block in with_clean_env'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:205:in `with_original_env'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/bundler-1.3.5/lib/bundler.rb:211:in `with_clean_env'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:25:in `block in execute'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:24:in `each'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/lib/pulsar/commands/main.rb:24:in `execute'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/clamp-0.6.1/lib/clamp/command.rb:67:in `run'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/clamp-0.6.1/lib/clamp/command.rb:125:in `run'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/pulsar-0.3.2/bin/pulsar:5:in `<top (required)>'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/bin/pulsar:23:in `load'
    from /home/vagrant/.rbenv/versions/2.0.0-p247/bin/pulsar:23:in `<main>'

If I create folder mkdir /tmp/pulsar that pulsar starts to work. Could pulsar automatically create this tmp folder?

This is more of a questions/request for best practice.

I thought I will put my application's configuration file config.ini into the apps folder of the pulsar configuration. This file contains database configuration, ip addresses etc.

I stored the files like this:

├── apps
│   ├── fuboo
│   │   ├── configs
│   │   │   ├── production
│   │   │   │   └── config.ini
│   │   │   └── staging
│   │   │       └── config.ini
│   │   ├── defaults.rb
│   │   ├── production.rb
│   │   └── staging.rb
│   └── base.rb

To upload them I use something like this in my capistrano recipe:

top.upload("apps/#{application}/configs/#{stage}", "/path/on/server")

What do you think about this, does it make sense? Do you do something similar?

What I'm not too fond of are the usages of #{application} and #{stage} in the local path. These match the actual path rather randomly, i.e. the application name could theoretically also be different from the pulsar-application path.
What do you think about this? Is this considered convention that they always match the pulsar-paths?
Or would it make sense to provide these values from within pulsar to the recipes (something like pulsar-application and pulsar-environment)?

I'm installing pulsar on various linux/osx systems with puppet now.
Preferably these installations would have a configuration (of the repository) set.

Currently the only way I see to do this is to store a .pulsar file in the user's home-directory. For systems with various users this is not easily accomplished - I would need to store this file in each user's home-directory.

How about allowing to configure pulsar for a whole system, i.e. specifying the repo in a file like /etc/pulsar/pulsar.conf?
Would it be okay to hardcode /etc? On the other hand it would be good if it could observe a prefix (${prefix}/etc).

Again I'm really not so knowledgable about the ruby world, so I don't know how this could be done best. Can one pass options to gem install which could be used to define the prefix?
Or should we observe ruby's installation path by reading RbConfig::CONFIG["prefix"] - on my homebrew installation this would be /usr/local/homebrew/Cellar/ruby/2.0.0-p247.

What do you think?

Adding @tomaszdurka

Something like?

gem 'capistrano', '~>2'

Since I've seen so many of Pulsar's users using Pulsar with system provisioning software like Chef/Puppet, I was wandering if any of you is experiencing problems with command line output coloring.

Since I've started thinking about adding some more verbose output to Pulsar, I was wandering about arguments pro/against coloring of cmd tools.

Pulsar should support multiple concurrent deploys.

Right now I don't think it does, the /tmp directories where the conf_repo is downloaded would clash when multiple deploys are triggered.

If the deploy fails it's raised an exception so the capfile and repo dir is not removed.

This is NOT wanted behavior. The exception should be managed properly.

Running pulsar --help will output some really ugly text which has no real limit to the character count.

This issue should be fixed by adding a character limit of 78.

like cap local deploy --dry-run --trace

When running pulsar within a Rack application, there is no fallback that uses .pulsar file inside the $HOME directory if there is no .pulsar file inside the app.

Right now Pulsar v1 only runs cap deploy, we need to add support for other capistrano tasks.