moll / capistrano-rsync Goto Github PK

Deploy with Rsync from any local (or remote) repository when using Capistrano. Capistrano v3 ready!

License: Other

Ruby 100.00%

capistrano-rsync's Introduction

Capistrano::Rsync for Capistrano v3

![Gem version][gem-badge] [gem-badge]: https://badge.fury.io/rb/capistrano-rsync.png

Deploy with Rsync to your server from any local (or remote) repository when using Capistrano. Saves you from having to install Git on your production machine and allows you to customize which files you want to deploy. Also allows you to easily precompile things on your local machine before deploying.

Tour

Works with the new Capistrano v3 (source code) versions >= 3.0.0pre14 and < 4.
Suitable for deploying any apps, be it Ruby, Rails, Node.js or others.
Exclude files from being deployed with Rsync's --exclude options.
Precompile files or assets easily before deploying, like JavaScript or CSS.
Caches your previously deployed code to speed up deployments ~1337%.
Currently works only with Git (as does Capistrano v3), so please shout out your interest in other SCMs.

Using Mina instead of Capistrano? I've built Mina::Rsync as well.

Using

Install with:

gem install capistrano-rsync

Require it at the top of your Capfile (or config/deploy.rb):

require "capistrano/rsync"

Set some rsync_options to your liking:

set :rsync_options, %w[--recursive --delete --delete-excluded --exclude .git*]

And after setting regular Capistrano options, deploy as usual!

cap deploy

Implementation

Clones and updates your repository to rsync_stage (defaults to tmp/deploy) on your local machine.
Checks out the branch set in the branch variable (defaults to master).
If rsync_cache set (defaults to shared/deploy), rsyncs to that directory on the server.
If rsync_cache set, copies the content of that directory to a new release directory.
If rsync_cache is nil, rsyncs straight to a new release directory.

After that, Capistrano takes over and runs its usual tasks and symlinking.

Exclude files from being deployed

If you don't want to deploy everything you've committed to your repository, pass some --exclude options to Rsync:

set :rsync_options, %w[
  --recursive --delete --delete-excluded
  --exclude .git*
  --exclude /config/database.yml
  --exclude /test/***
]

Precompile assets before deploy

Capistrano::Rsync runs rsync:stage before rsyncing. Hook to that like this:

task :precompile do
  Dir.chdir fetch(:rsync_stage) do
    system "rake", "assets:precompile"
  end
end

after "rsync:stage", "precompile"

Deploy release without symlinking the current directory

cap rsync:release

Configuration

Set Capistrano variables with set name, value.

Name	Default	Description
repo_url	`.`	The path or URL to a Git repository to clone from.
branch	`master`	The Git branch to checkout.
rsync_stage	`tmp/deploy`	Path where to clone your repository for staging, checkouting and rsyncing. Can be both relative or absolute.
rsync_cache	`shared/deploy`	Path where to cache your repository on the server to avoid rsyncing from scratch each time. Can be both relative or absolute. Set to `nil` if you want to disable the cache.
rsync_options	`[]`	Array of options to pass to `rsync`.

License

Capistrano::Rsync is released under a Lesser GNU Affero General Public License, which in summary means:

You can use this program for no cost.
You can use this program for both personal and commercial reasons.
You do not have to share your own program's code which uses this program.
You have to share modifications (e.g bug-fixes) you've made to this program.

For more convoluted language, see the LICENSE file.

About

Andri Möll made this happen.
Monday Calendar was the reason I needed this.

If you find Capistrano::Rsync needs improving, please don't hesitate to type to me now at [email protected] or create an issue online.

capistrano-rsync's People

Contributors

Stargazers

Watchers

capistrano-rsync's Issues

bypass git clone?

I have already cloned my repo and ran composer install, is their a way to by pass the clone stage?

Then just sync the my local folder to the destination?

capistano 3.2.x error - #{scm}:set_current_revision ?

It appears something has changed in latest capistrano and now it requires #{scm}:set_current_revision

I have forked the code and added the following definition

    Rake::Task.define_task("#{scm}:set_current_revision") do
      invoke "rsync:current_revision" 
    end

but not sure how to implement

so far I have this, but the system is not deploying the proper files (Ubuntu 14.04)

  task :current_revision => %w[set_current_revision]

Any thoughts?

Rsync unstaged files

Is there a way to rsync all of the files from the repository? In fact, I don't even want it to check the git remote at all, I just want to copy all of the files to the release.

Compatibility with capistrano >=3.7

This gem is no longer working for me after upgrading to capistrano 3.11. I think this may be a result of changes introduced by version 3.7.

Outlined here: https://github.com/capistrano/capistrano/blob/master/UPGRADING-3.7.md

Is my hunch correct? if yes, any recommendations on how I can get it to work?

Doesn't work with Capistrano 3.0.1

Capistrano 3.0.1

Here's deploy.rb:

require "capistrano/rsync"

set :application, 'auth-backend'
set :repo_url, 'git@red:omg-auth-backend.git'
set :branch, 'master'

set :deploy_to, '/home/cpc-system/apps/auth-backend'
set :scm, :git

set :keep_releases, 5

namespace :deploy do
  desc 'Restart application'
  task :restart do
    on roles(:app), in: :sequence, wait: 5 do
      execute :touch, release_path.join('tmp/restart.txt')
    end
  end

  after :finishing, 'deploy:cleanup'
end

production.rb is the following:

set :stage, :production
server 'cpc-server', user: 'cpc-system', roles: %w{web app db}
set :rsync_options, %w[--recursive --delete --delete-excluded --exclude .git*]

On a cold deployment outputs the following:

$ be cap production deploy --trace
** Invoke production (first_time)
** Execute production
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke deploy (first_time)
** Execute deploy
** Invoke deploy:starting (first_time)
** Execute deploy:starting
** Invoke deploy:check (first_time)
** Invoke rsync:hook_scm (first_time)
** Execute rsync:hook_scm
** Execute deploy:check
** Invoke git:check (first_time)
** Invoke git:wrapper (first_time)
** Execute git:wrapper
 INFO [162b1537] Running /usr/bin/env mkdir -p /tmp/auth-backend/ on cpc-server
DEBUG [162b1537] Command: /usr/bin/env mkdir -p /tmp/auth-backend/
 INFO [162b1537] Finished in 1.023 seconds with exit status 0 (successful).
DEBUG Uploading /tmp/auth-backend/git-ssh.sh 0.0%
 INFO Uploading /tmp/auth-backend/git-ssh.sh 100.0%
 INFO [0648d87e] Running /usr/bin/env chmod +x /tmp/auth-backend/git-ssh.sh on cpc-server
DEBUG [0648d87e] Command: /usr/bin/env chmod +x /tmp/auth-backend/git-ssh.sh
 INFO [0648d87e] Finished in 0.149 seconds with exit status 0 (successful).
** Execute git:check
DEBUG [bc78532f] Running /usr/bin/env git ls-remote git@red:omg-auth-backend.git on cpc-server
DEBUG [bc78532f] Command: ( GIT_ASKPASS=/bin/echo GIT_SSH=/tmp/auth-backend/git-ssh.sh /usr/bin/env git ls-remote git@red:omg-auth-backend.git )
DEBUG [bc78532f]    Permission denied (publickey,password).
DEBUG [bc78532f]    fatal: The remote end hung up unexpectedly
DEBUG [bc78532f] Finished in 0.368 seconds with exit status 128 (failed).

It seems like it's trying to do git stuff on a server, which it is not supposed to be doing.

rsync being run twice

Using capistrano 3.2.1 and capistrano-rsync 1.0.2, I managed to get a very basic example working, with a single Capfile like this.

However, if I try to make the directory structure used by Capistrano, I see the rsync command is run twice.

Here are my files:

./Capfile

require 'capistrano/setup'
require 'capistrano/deploy'
require "capistrano/rsync"

Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }

./config/deploy.rb

set :application, "My Application"
set :repo_url, "[email protected]:XXX/XXX.git"
set :branch, "master"
set :deploy_to, "/var/www/test"
set :rsync_options, %w[--recursive --delete --delete-excluded  --exclude .git*]
set :scm, :rsync

namespace :rsync do
  task :set_current_revision do
    # nothing
  end
end

./config/deploy/dev.rb

server 'myserver', user: 'deploy', roles: %w{web}

(I added myserver to my hosts file for this example).

If I now run cap dev deploy, I get this output:

#!text
INFO[ee7cbc5d] Running /usr/bin/env mkdir -pv /var/www/test/shared /var/www/test/releases on myserver
DEBUG[ee7cbc5d] Command: /usr/bin/env mkdir -pv /var/www/test/shared /var/www/test/releases
INFO[ee7cbc5d] Finished in 0.152 seconds with exit status 0 (successful).
DEBUG[d2719cce] Running /usr/bin/env [ -f /var/www/test/current/REVISION ] on myserver
DEBUG[d2719cce] Command: [ -f /var/www/test/current/REVISION ]
DEBUG[d2719cce] Finished in 0.007 seconds with exit status 0 (successful).
DEBUG[73100218] Running /usr/bin/env cat /var/www/test/current/REVISION 2>/dev/null on myserver
DEBUG[73100218] Command: /usr/bin/env cat /var/www/test/current/REVISION 2>/dev/null
DEBUG[73100218]
DEBUG[73100218] Finished in 0.009 seconds with exit status 0 (successful).
HEAD is now at f3652d4 Merged in DEV-3008-support-release-version (pull request #3)
HEAD is now at f3652d4 Merged in DEV-3008-support-release-version (pull request #3)
skipping directory .
skipping directory .
INFO[c14f82bc] Running /usr/bin/env rsync --archive --acls --xattrs "/var/www/test/shared/deploy/" "/var/www/test/releases/20140609162536/" on myserver
DEBUG[c14f82bc] Command: rsync --archive --acls --xattrs "/var/www/test/shared/deploy/" "/var/www/test/releases/20140609162536/"
INFO[c14f82bc] Finished in 0.056 seconds with exit status 0 (successful).
INFO[7dd47543] Running /usr/bin/env rsync --archive --acls --xattrs "/var/www/test/shared/deploy/" "/var/www/test/releases/20140609162536/" on myserver
DEBUG[7dd47543] Command: rsync --archive --acls --xattrs "/var/www/test/shared/deploy/" "/var/www/test/releases/20140609162536/"
INFO[7dd47543] Finished in 0.052 seconds with exit status 0 (successful).
DEBUG[4c4ef630] Running /usr/bin/env if test ! -d /var/www/test/releases/20140609162536; then echo "Directory does not exist '/var/www/test/releases/20140609162536'" 1>&2; false; fi on myserver
DEBUG[4c4ef630] Command: if test ! -d /var/www/test/releases/20140609162536; then echo "Directory does not exist '/var/www/test/releases/20140609162536'" 1>&2; false; fi
DEBUG[4c4ef630] Finished in 0.007 seconds with exit status 0 (successful).
INFO[501e7e1b] Running /usr/bin/env echo "" >> REVISION on myserver
DEBUG[501e7e1b] Command: cd /var/www/test/releases/20140609162536 && /usr/bin/env echo "" >> REVISION
INFO[501e7e1b] Finished in 0.008 seconds with exit status 0 (successful).
INFO[7dcba5ac] Running /usr/bin/env rm -rf /var/www/test/current on myserver
DEBUG[7dcba5ac] Command: /usr/bin/env rm -rf /var/www/test/current
INFO[7dcba5ac] Finished in 0.008 seconds with exit status 0 (successful).
INFO[28029ffa] Running /usr/bin/env ln -s /var/www/test/releases/20140609162536 /var/www/test/current on myserver
DEBUG[28029ffa] Command: /usr/bin/env ln -s /var/www/test/releases/20140609162536 /var/www/test/current
INFO[28029ffa] Finished in 0.009 seconds with exit status 0 (successful).
DEBUG[fd3953a8] Running /usr/bin/env ls -x /var/www/test/releases on myserver
DEBUG[fd3953a8] Command: /usr/bin/env ls -x /var/www/test/releases
DEBUG[fd3953a8]         20140609161334  20140609161407  20140609161443  20140609161503  20140609162526
DEBUG[fd3953a8]         20140609162536
DEBUG[fd3953a8] Finished in 0.009 seconds with exit status 0 (successful).
INFOKeeping 5 of 6 deployed releases on myserver
INFO[301da6e1] Running /usr/bin/env rm -rf /var/www/test/releases/20140609161334 on myserver
DEBUG[301da6e1] Command: /usr/bin/env rm -rf /var/www/test/releases/20140609161334
INFO[301da6e1] Finished in 0.008 seconds with exit status 0 (successful).
DEBUG[c64fd4e6] Running /usr/bin/env if test ! -d /var/www/test/releases; then echo "Directory does not exist '/var/www/test/releases'" 1>&2; false; fi on myserver
DEBUG[c64fd4e6] Command: if test ! -d /var/www/test/releases; then echo "Directory does not exist '/var/www/test/releases'" 1>&2; false; fi
DEBUG[c64fd4e6] Finished in 0.005 seconds with exit status 0 (successful).
INFO[37a721d7] Running /usr/bin/env echo "Branch master (at ) deployed as release 20140609162536 by lars" >> /var/www/test/revisions.log on myserver
DEBUG[37a721d7] Command: echo "Branch master (at ) deployed as release 20140609162536 by lars" >> /var/www/test/revisions.log
INFO[37a721d7] Finished in 0.005 seconds with exit status 0 (successful).

So, why the duplicated git commands (HEAD is now at f3652d4 ...) and rsync commands (INFO[c14f82bc] Running /usr/bin/env rsync ...)?

Example?

I have read the documentation but still can not get to work? Can you create a sample configuration?

Thanks!

TypeError: no _dump_data is defined for class Enumerator

It seems capistrano-rsync is not compatible with the latest version of capistrano 3.4.0.

[user@server project]$ cat Gemfile | grep cap
  gem 'capistrano', '~> 3.4.0', require: false
  gem 'capistrano-rails', '~> 1.1.2', require: false
  gem 'capistrano-bundler', '~> 1.1.4', require: false
  gem 'capistrano-rvm', '~> 0.1.2', require: false
  gem 'capistrano-sidekiq', '~> 0.4.0', require: false
  gem 'capistrano-rsync', github: 'STRML/capistrano-rsync', require: false
[user@server project]$

** Execute rsync:release
cap aborted!
TypeError: no _dump_data is defined for class Enumerator
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/dsl.rb:54:in `dump'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/dsl.rb:54:in `on'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-rsync-1.0.2/lib/capistrano/rsync.rb:89:in `block (2 levels) in <top (required)>'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:240:in `call'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:240:in `block in execute'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:235:in `each'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:235:in `execute'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:172:in `invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:201:in `block in invoke_prerequisites'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:199:in `each'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:199:in `invoke_prerequisites'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:178:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:172:in `invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:165:in `invoke'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/dsl.rb:16:in `invoke'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/tasks/deploy.rake:9:in `block (2 levels) in <top (required)>'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:240:in `call'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:240:in `block in execute'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:235:in `each'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:235:in `execute'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:172:in `invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:165:in `invoke'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/dsl.rb:16:in `invoke'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/tasks/framework.rake:65:in `block (2 levels) in <top (required)>'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/tasks/framework.rake:64:in `each'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/tasks/framework.rake:64:in `block in <top (required)>'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:240:in `call'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:240:in `block in execute'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:235:in `each'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:235:in `execute'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:179:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:172:in `invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/task.rb:165:in `invoke'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:150:in `invoke_task'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:106:in `block (2 levels) in top_level'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:106:in `each'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:106:in `block in top_level'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:115:in `run_with_threads'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:100:in `top_level'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:78:in `block in run'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:176:in `standard_exception_handling'
/usr/local/rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/rake/application.rb:75:in `run'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/lib/capistrano/application.rb:15:in `run'
/usr/local/rvm/gems/ruby-2.2.1@project/gems/capistrano-3.4.0/bin/cap:3:in `<top (required)>'
/usr/local/rvm/gems/ruby-2.2.1@project/bin/cap:23:in `load'
/usr/local/rvm/gems/ruby-2.2.1@project/bin/cap:23:in `<main>'
/usr/local/rvm/gems/ruby-2.2.1@project/bin/ruby_executable_hooks:15:in `eval'
/usr/local/rvm/gems/ruby-2.2.1@project/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => rsync:create_release => rsync:release
The deploy has failed with an error: no _dump_data is defined for class Enumerator
** Invoke deploy:failed (first_time)
** Execute deploy:failed
[user@server project]$

rsync can't use --rsh command flag to change default ssh port

The most common use-case for --rsh is to set a non-default ssh port, as such;

rsync --rsh='ssh -p 222' ...

When trying to add the --rsh flag (in such a way as above) to :rsync_options, you get the following;

rsync: Failed to exec ssh: No such file or directory (2)
rsync error: error in IPC code (code 14) at pipe.c(85) [sender=3.1.1]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in IPC code (code 14) at io.c(226) [sender=3.1.1]
rsync: Failed to exec ssh: No such file or directory (2)
rsync error: error in IPC code (code 14) at pipe.c(85) [sender=3.1.1]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in IPC code (code 14) at io.c(226) [sender=3.1.1]
...

Provide option to sync only certain folders

Can this be used without a SCM?

I have a folder full of generated files I want to send to a server, can this be used to upload the files which are not committed to SCM?

Maybe extending the upload! command might be an option otherwise...

SSH options & bundle

Hello! I read README file and all questions below, but answers do not solve mine.
So I have to connect to remote server via SSH(the same server, in example), so i use standard capistrano form for sending USER and PASSWORD information, like this:

server '127.0.0.1', user: 'user', password: 'qwerty', roles: %w{web app}

It asks me to enter password everytime, but it is in the line! At the same time, it knows username, and type it in the console. Cant solve this.

In README you marked, that "Deploy with Rsync to your server from any local (or remote) repository when using Capistrano. "
So i would like to deploy from local repository.
Here is my minimalistic deploy.rb

lock '3.1.0'
set :deploy_flag,"--development"
set :repo_url, "/home/user/api"
set :rsync_options, %w[--recursive --delete --delete-excluded --exclude .git*]
set :scm, :rsync
set :rvm, '2.0.0-p247'
set :default_env, { rvm_bin_path: '~/.rvm/bin' }
set :deploy_to, "/home/user/rsync/"

It is test example. I do NOT have repository on GIT, so just pass path to repo_url
I would like my project in api being beployed to rsync. Right folders appear in rsync, but subdirectories in release are empty, and gems from '~/.rvm/bin' are bundled.
I hope you understand what i would like to do, and i hope you help me;