aws / opsworks-cookbooks Goto Github PK

View Code? Open in Web Editor NEW

1.1K 112.0 1.2K 5.57 MB

Chef Cookbooks for the AWS OpsWorks Service

License: Other

Ruby 72.32% HTML 23.88% Perl 1.29% Shell 1.75% Python 0.70% JavaScript 0.06%

opsworks-cookbooks's Introduction

opsworks-cookbooks

This repo contains cookbooks used by AWS OpsWorks for Chef versions 11.10, 11.4 and 0.9.

To get started with AWS OpsWorks cookbooks for all versions of Chef see the cookbook documentation.

If you want to override any template (like the Rails database.yml or the Apache vhost definition), this is the place to look for the originals.

Do not reuse built-in cookbook names for custom or community cookbooks. Custom cookbooks that have the same name as built-in cookbooks might fail.

Chef version 12

For Chef 12.2 Windows and Chef 12 Linux there are no built-in cookbooks

Chef versions 11.10, 11.4 and 0.9

These branches contain the cookbooks that are used by official OpsWorks releases:

Chef 11.10: release-chef-11.10
Chef 11.4: release-chef-11.4 (deprecated)
Chef 0.9: release-chef-0.9 (deprecated)

These branches reflect the upcoming changes for the next release:

Chef 11.10: master-chef-11.10
Chef 11.4: master-chef-11.4 (deprecated)
Chef 0.9: master-chef-0.9 (deprecated)

The master branch is not used since AWS OpsWorks supports multiple configuration managers.

LICENSE: Unless otherwise stated, cookbooks/recipes originated by Amazon Web Services are licensed under the Apache 2.0 license. See the LICENSE file. Some files are just imported and authored by others. Their license will of course apply.

opsworks-cookbooks's People

Contributors

Stargazers

Watchers

Forkers

easybib headlinerfm funasaki markeymagic sliard aojha97 luisantonioa cc-sysops davebond jimflip zingcheckout techpines bearrito ravindragit binarymatt joeyaghion unfold kwarter-cookbooks slim2000 kodify awithersdd iamjoshbinder motammi hs9587 phoolish michaelkamphausen vamida heyflynn hieun agrismart meloncard gouldingken ziggythehamster danapplegate miagao leehosung gvijay tottokug abhijitr pbyrne looprock jarster2608 nivertech drewkhoury jonhyman tedder riywo arielo rpeterson anfleene cruglobal macotomii mdesjardins crowdmob lookaflyingdonkey gocarrot asabi beckoncall stempower apps4u seangaffney ramonjaker openstax niteshraw willmitchell giddyinc rud5g kbullaughey grouptools acndr timhaines kevinhodges elephantventures axomi kostia tutogogo jingu owindsor dineshlogarajcts saiaman ajohnstone jtescher rwheaton smart hippolin hemanth987kumar sethuma highfivemobile 43n79w cdelguercio ta9n wrgbh446 gogojimmy khani3s foxio okolomiets danielkleinert integrationservices rubish stembolthq

opsworks-cookbooks's Issues

`rake validate_attribute_dependencies` doesn't work

Getting the error:

undefined method 'each' for "<line_string>":String

where <line_string> appears to be the first line from an attribute file.

dynamic scaling with proper hostname/domainname scheme

in my evaluation of opsworks, i've come across a small issue and i wanted to get the opsworks crew opinion on the matter.

if you spin up a stock instance from a stock amazon linux ami and then examine the chef logs, you'll find this on the second line.

[2014-02-18T23:49:51+00:00] DEBUG: Building node object for web1.localdomain

it seems that before chef runs the first time, a process has run and done some basic setup. what i would like to do is have web1.localdomain set to my actual domain name, so when chef runs for the first time, and all future times, it's always the same node.

i have a ticket open with amazon support, and they're telling me to change the hosts.erb template in opsworks_stack_state_sync. while i agree that this would indeed change my hostname, i can't seem to make them understand this needs to be done before chef runs, and therefore this isn't the "right" way to fix this issue.

do you have any thoughts or insight on properly setting domain names, before chef runs, in the opsworks environment?

Deploy shows success even if bundle install failed

Bundle install failed on some gems but deploy shows success.
I think it is confusing and should get faled state as well.

Installing rmagick (2.13.2)
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

/usr/local/bin/ruby extconf.rb
checking for Ruby version >= 1.8.5... yes
checking for gcc... yes
checking for Magick-config... no
Can't install RMagick 2.13.2. Can't find Magick-config in /usr/local/bin:/usr/local/sbin:/opt/aws/opsworks/releases/20130726231932_203/vendor/bundle/ruby/1.8/bin:/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/bin:/bin:/usr/bin:/sbin:/usr/sbin

*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/local/bin/ruby

Gem files will remain installed in /home/deploy/.bundler/rgx1/ruby/1.9.1/gems/rmagick-2.13.2 for inspection.
Results logged to /home/deploy/.bundler/rgx1/ruby/1.9.1/gems/rmagick-2.13.2/ext/RMagick/gem_make.out
An error occurred while installing rmagick (2.13.2), and Bundler cannot
continue.
Make sure that gem install rmagick -v '2.13.2' succeeds before bundling.

Update dependencies cookbook metadata to use correct types

Looks like the metadata.rb file in the dependencies cookbook is using 'boolean' types which are not allowed. http://wiki.opscode.com/display/chef/Metadata#Metadata-attribute says that only types of 'string' and 'array' are allowed.

I'm pulling these cookbooks into a berkshelf on Vagrant to test my cookbooks out before deploying to AWS. These bad types are giving me errors when berkshelf (or its buddies) try to validate the metadata.

Notes:

#6 might take care of this altogether?
Very related pull request at scalarium: https://github.com/scalarium/cookbooks/pull/15
Similar situation: sous-chefs/rvm#20

RDS Support does not work with Rails Application and non-MySQL database

I tried to use a Postgres RDS database.

The cookbooks don't correctly set the database type in config/database.yml.

Apparently deploy["database"]["adapter"] is not set, but only deploy["database"]["type"], which is not used by the rails cookbook. The cookbooks seem to require adapter to be set and default to mysql2 when it's not:

INFO: No database adapter specified for <APP NAME>, guessing
INFO: Looks like <APP NAME> is a Rails 3 application, defaulting to mysql2

This results in the database connection failing.

As a workaround, I set deploy["database"]["adapter"] for the application in the Stack JSON.

unable to override symlink_before_migrate

Hello,
Currently we are trying to override the default behavior in opsworks for a Rails layer. We have a database.yml that we check in, and supply the credentials in through another mechanism. Opsworks default deploy is overriding this via a symlink its doing to a database.yml. We purposely do not set the database host in the custom json, which doesn't create the shared/config/database.yml which results in a symlink to nothing.

we tried setting the custom json with

"symlink_before_migrate": {}

"symlink_before_migrate": nil

no cigar and then we tried overriding this behavior with the following directly in a attributes/default.rb in one of our recipes and nothing

normal[:deploy][:phoenix][:symlink_before_migrate] = {}

connecting to the failed box, and outputting json we can see:

"symlink_before_migrate": {
        "config/database.yml": "config/database.yml",
        "config/memcached.yml": "config/memcached.yml"
      },

any clue or ideas on how we get that to not contain the database.yml line?
Thank you

Unicorn worker_processes default to node[:rails][:max_pool_size] doesn't make sense.

opsworks-cookbooks/unicorn/attributes/default.rb

Line 19 in 9454998

 default[:unicorn][:worker_processes] = node[:rails][:max_pool_size] ? node[:rails][:max_pool_size] : 4 

The line above set's the number of unicorn worker's per server instance equal to the max pool size of the database. Since unicorn uses separate processes for concurrency rather than multithreading, it seems this could cause performance problems as you increase the number of instances and pool size.

Relevant StackOverflow answer: http://stackoverflow.com/a/22404491

Apache2 restarts too often

Since dda6651 Apache will be restarted on every include_recipe "apache2".
This may lead to unwanted outages, for instance on Rails Passenger app deployments.

From our deploy log:

[Tue, 12 Nov 2013 20:18:40 +0000] INFO: execute[logdir_existence_and_restart_apache2] sending restart action to service[apache2] (delayed)

Opsworks no longer creates database.yml for Rails applications without MySQL Layer in Opsworks

We're using RDS and are setting our database configuration in the custom Stack JSON.

Until recently Opsworks generated a database.yml for our Rails application, but now it seems to only do this when a database server is set up via an Opsworks Layer. I found the following code in the Opsworks deploy cookbook (on the server, this version seems to be unpublished on GitHub):

template "#{node[:deploy][application][:deploy_to]}/shared/config/database.yml" do
  cookbook "rails"
  source "database.yml.erb"
  mode "0660"
  owner node[:deploy][application][:user]
  group node[:deploy][application][:group]
  variables(
    :database => node[:deploy][application][:database],
    :environment => node[:deploy][application][:rails_env]
  )

  only_if do
    node[:opsworks][:layers].has_key?('db-master')
  end
end.run_action(:create)

The same part of the currently published version does not contain the only_if block.

This way of configuring a Rails application with a non-Opsworks database server is explicitly documented in the Opsworks docs, so I think you should revert this change.

By the way, publishing the Opsworks cookbooks when deploying a new version would be great, otherwise they're not much help when debugging problems like this.

database.yml.erb template doesn't include port

There should be a port variable so this can be overridden if we're using RDS or some other database on a custom port.

Incorrect Gemfile changes detection

The method different_gemfile? compares the current and previous Gemfile to check if it was changed.

The problem is that the Gemfile doesn't have to be changed for the gems to be updated. The Gemfile.lock holds full data about the gems to be loaded. Gemfile is just a manifest for bundler.

As a result of this an update to a gem (vai bundle update my_gem) will not be reflected on the deployment and the old gem (and all its dependencies) will be used instead.

Failure to load custom cookbooks that are Git repositories

If I specify an HTTP archive for my custom cookbooks, and that compressed archive contains a folder which is a Git repository, it will not be properly copied to /opt/aws/opsworks/current/site-cookbooks.

The reason is that by initializing a new repository in the top-level custom cookbooks directory, and then cloning that newly-initialized repository, any Git repository subdirectories will be ignored.

Simply copying the archive to the destination would obviate this problem. Is that not possible?

nodejs provisioning has a number of false assumptions...

Chef recipe

Setup: opsworks_initial_setup, opsworks_nodejs, opsworks_nodejs::npm, ssh_users
Configure: ssh_users, agent_version, opsworks_nodejs::configure
Deploy: deploy:nodejs, photobox-checkout::default
Undeploy: deploy::nodejs-undeploy
Shutdown: opsworks_shutdown::default, deploy::nodejs-stop

This installs apache bound to port 80, which at no point do I include apache.

root@nodejs-app3:/var/log/aws/opsworks# egrep -rin apache .
./installer.log:576:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: node nodejs-app3.localdomain loading cookbook passenger_apache2's attribute file /opt/aws/opsworks/releases/20130529164351_120/cookbooks/passenger_apache2/attributes/passenger.rb
./installer.log:603:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: node nodejs-app3.localdomain loading cookbook apache2's attribute file /opt/aws/opsworks/releases/20130529164351_120/cookbooks/apache2/attributes/apache.rb
./installer.log:629:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: Loading cookbook passenger_apache2's definitions from /opt/aws/opsworks/releases/20130529164351_120/cookbooks/passenger_apache2/definitions/passenger_web_app.rb
./installer.log:636:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20130529164351_120/cookbooks/apache2/definitions/web_app.rb
./installer.log:637:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20130529164351_120/cookbooks/apache2/definitions/apache_conf.rb
./installer.log:638:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20130529164351_120/cookbooks/apache2/definitions/apache_site.rb
./installer.log:639:[Sat, 01 Jun 2013 18:18:30 +0000] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20130529164351_120/cookbooks/apache2/definitions/apache_module.rb

monit is monitoring apache not the node services

check host node_web_app_checkout with address 127.0.0.1
  start program = "/bin/sh -c 'cd /srv/www/checkout/current; /usr/bin/env NODE_PATH=/srv/www/checkout/current/node_modules:/srv/www/checkout/current /usr/local/bin/node /srv/www/checkout/current/server.js'"
  stop program  = "/usr/bin/pkill -f 'node /srv/www/checkout/current/server.js'"
  if failed port 80 protocol HTTP
    request /
    with timeout 10 seconds
    then restart
check host node_web_app_services with address 127.0.0.1
  start program = "/bin/sh -c 'cd /srv/www/services/current; /usr/bin/env NODE_PATH=/srv/www/services/current/node_modules:/srv/www/services/current /usr/local/bin/node /srv/www/services/current/server.js'"
  stop program  = "/usr/bin/pkill -f 'node /srv/www/services/current/server.js'"
  if failed port 80 protocol HTTP
    request /
    with timeout 10 seconds
    then restart

  template "#{node[:monit][:conf_dir]}/node_web_app-#{application}.monitrc" do
    source 'node_web_app.monitrc.erb'
    cookbook 'opsworks_nodejs'
    owner 'root'
    group 'root'
    mode '0644'
    variables(
      :deploy => deploy,
      :application_name => application,
      :monitored_script => "#{deploy[:deploy_to]}/current/server.js"
    )
    notifies :restart, resources(:service => 'monit'), :immediately
  end

Additionally assumes the server invocation should be server.js

./opsworks_nodejs/templates/default/node_web_app.monitrc.erb

check host node_web_app_<%= @application_name %> with address 127.0.0.1
  start program = "/bin/sh -c 'cd <%= @deploy[:deploy_to] %>/current; /usr/bin/env NODE_PATH=<%= @deploy[:deploy_to] %>/current/node_modules:<%= @deploy[:deploy_to] %>/current /usr/local/bin/node <%= @monitored_script %>'"
  stop program  = "/usr/bin/pkill -f 'node <%= @monitored_script %>'"
  if failed port 80 protocol HTTP
    request /
    with timeout 10 seconds
    then restart

This is hard coded to port 80, it also assumes that monit will be the method in which to spawn the server.

Additionally the node_modules path may not be the same in current.

As an example I'm using an upstart script.

description "<%= @alias %>"

start on runlevel [2345]
stop on shutdown

respawn

script
   <% deploy_to = node[:nodejs][@alias][:deploy_to] %>
   cd <%= deploy_to %>/current/<%= @path %>/;
   exec sudo -u www-data sh -c "NODE_ENV=production NODE_PATH=<%= deploy_to %>/current/<%= @path %>/node_modules:<%= deploy_to %>/current/<%= @path %> node <%= deploy_to %>/current/<%= @script %>  >> <%= deploy_to %>/current/log/<%= @alias %>.log 2>&1"
end script

execute[Setting sysctl: net.core.somaxconn]

I'm trying to create an Ubuntu 12.04 instance (no custom ami), chef 11.10 and I'm receiving the following error:

[2014-04-02T12:26:08+00:00] INFO: Processing directory[/etc/sysctl.d] action create (opsworks_initial_setup::sysctl line 1)
[2014-04-02T12:26:08+00:00] INFO: Processing template[/etc/sysctl.d/70-opsworks-defaults.conf] action create (opsworks_initial_setup::sysctl line 8)
[2014-04-02T12:26:08+00:00] INFO: template[/etc/sysctl.d/70-opsworks-defaults.conf] created file /etc/sysctl.d/70-opsworks-defaults.conf
[2014-04-02T12:26:08+00:00] INFO: template[/etc/sysctl.d/70-opsworks-defaults.conf] updated file contents /etc/sysctl.d/70-opsworks-defaults.conf
[2014-04-02T12:26:08+00:00] INFO: template[/etc/sysctl.d/70-opsworks-defaults.conf] owner changed to 0
[2014-04-02T12:26:08+00:00] INFO: template[/etc/sysctl.d/70-opsworks-defaults.conf] group changed to 0
[2014-04-02T12:26:08+00:00] INFO: template[/etc/sysctl.d/70-opsworks-defaults.conf] mode changed to 644
[2014-04-02T12:26:08+00:00] INFO: Processing execute[Setting sysctl: net.core.somaxconn] action run (opsworks_initial_setup::sysctl line 17)

================================================================================
Error executing action `run` on resource 'execute[Setting sysctl: net.core.somaxconn]'
================================================================================


Errno::ENOENT
-------------
No such file or directory - sysctl


Resource Declaration:
---------------------
# In /var/lib/aws/opsworks/cache/cookbooks/opsworks_initial_setup/recipes/sysctl.rb

17:   execute "Setting sysctl: #{systcl}" do
18:     command "sysctl -w #{systcl}=#{value}"
19:     action :run
20:   end
21: end



Compiled Resource:
------------------
# Declared in /var/lib/aws/opsworks/cache/cookbooks/opsworks_initial_setup/recipes/sysctl.rb:17:in `block in from_file'

execute("Setting sysctl: net.core.somaxconn") do
action [:run]
retries 0
retry_delay 2
command "sysctl -w net.core.somaxconn=1024"
backup 5
returns 0
cookbook_name "opsworks_initial_setup"
recipe_name "sysctl"
end



[2014-04-02T12:26:08+00:00] INFO: Running queued delayed notifications before re-raising exception
[2014-04-02T12:26:08+00:00] ERROR: Running exception handlers
[2014-04-02T12:26:08+00:00] ERROR: Exception handlers complete
[2014-04-02T12:26:08+00:00] FATAL: Stacktrace dumped to /var/lib/aws/opsworks/cache/chef-stacktrace.out
[2014-04-02T12:26:08+00:00] ERROR: execute[Setting sysctl: net.core.somaxconn] (opsworks_initial_setup::sysctl line 17) had an error: Errno::ENOENT: No such file or directory - sysctl
[2014-04-02T12:26:09+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

opsworks_rubygems Fallback URL Does Not Exist

If opsworks_rubygems cannot find the file rubygems-2.1.7.tgz (2.1.7 is the default version) at http://production.cf.rubygems.org/rubygems/rubygems-2.1.7.tgz, it attempts to fallback to the URL https://opsworks-instance-assets.s3.amazonaws.com/rubygems-2.1.7.tgz. However, the URL is not available, and it returns a 403 Forbidden.

The fallback code is in the default recipe of opsworks_rubygems, starting at line 8.

fallback "rubygems download fallback" do
  resource rubygems_download
  fallback_options [
    {:source => "#{node[:opsworks_commons][:assets_url]}/rubygems-#{node[:opsworks_rubygems][:version]}.tgz"}
  ]
end

Here's the relavant log entries:

[2014-01-06T18:09:41+00:00] INFO: Processing ruby_block[Fallback for remote_file[/tmp/rubygems-2.1.7.tgz]] action run (opsworks_rubygems::default line 5)
[2014-01-06T18:09:41+00:00] INFO: Processing remote_file[/tmp/rubygems-2.1.7.tgz] action create (opsworks_rubygems::default line 1)
[2014-01-06T18:09:41+00:00] DEBUG: remote_file[/tmp/rubygems-2.1.7.tgz] checking for changes
[2014-01-06T18:09:41+00:00] DEBUG: Sending HTTP Request via GET to production.cf.rubygems.org:80/rubygems/rubygems-2.1.7.tgz
[2014-01-06T18:18:06+00:00] WARN: remote_file[/tmp/rubygems-2.1.7.tgz] failed: Errno::ETIMEDOUT: remote_file[/tmp/rubygems-2.1.7.tgz] (opsworks_rubygems::default line 1) had an error: Errno::ETIMEDOUT: Error connecting to http://production.cf.rubygems.org/rubygems/rubygems-2.1.7.tgz - Connection timed out - connect(2)
[2014-01-06T18:18:06+00:00] INFO: Processing remote_file[/tmp/rubygems-2.1.7.tgz] action create (opsworks_rubygems::default line 1)
[2014-01-06T18:18:07+00:00] DEBUG: remote_file[/tmp/rubygems-2.1.7.tgz] checking for changes
[2014-01-06T18:18:07+00:00] DEBUG: Sending HTTP Request via GET to opsworks-instance-assets.s3.amazonaws.com:443/rubygems-2.1.7.tgz
[2014-01-06T18:18:07+00:00] DEBUG: Streaming download from https://opsworks-instance-assets.s3.amazonaws.com/rubygems-2.1.7.tgz to tempfile /tmp/chef-rest20140106-1487-1l1o597-0
[2014-01-06T18:18:07+00:00] WARN: remote_file[/tmp/rubygems-2.1.7.tgz] failed: Net::HTTPServerException: remote_file[/tmp/rubygems-2.1.7.tgz] (opsworks_rubygems::default line 1) had an error: Net::HTTPServerException: 403 "Forbidden"
[2014-01-06T18:18:07+00:00] INFO: Running queued delayed notifications before re-raising exception
[2014-01-06T18:18:07+00:00] DEBUG: Re-raising exception: Net::HTTPServerException - ruby_block[Fallback for remote_file[/tmp/rubygems-2.1.7.tgz]] (opsworks_rubygems::default line 5) had an error: Net::HTTPServerException: remote_file[/tmp/rubygems-2.1.7.tgz] (opsworks_rubygems::default line 1) had an error: Net::HTTPServerException: 403 "Forbidden"

Chef 11.4 Chef::Exceptions::ValidationFailed

I'm getting a failure using Chef 11.4 and a new opsworks stack.

I get the following stack trace:

# Logfile created on Thu Jul 25 17:59:37 +0000 2013 by logger.rb/1.2.6
[2013-07-25T17:59:37+00:00] INFO: *** Chef 11.4.4 ***
[2013-07-25T17:59:38+00:00] DEBUG: Building node object for cupcake.localdomain
[2013-07-25T17:59:38+00:00] DEBUG: Extracting run list from JSON attributes provided on command line
[2013-07-25T17:59:38+00:00] INFO: Setting the run_list to ["opsworks_custom_cookbooks::load", "opsworks_custom_cookbooks::execute"] from JSON
[2013-07-25T17:59:38+00:00] DEBUG: Applying attributes from json file
[2013-07-25T17:59:38+00:00] DEBUG: Platform is amazon version 2013.03
[2013-07-25T17:59:38+00:00] INFO: Run List is [recipe[opsworks_custom_cookbooks::load], recipe[opsworks_custom_cookbooks::execute]]
[2013-07-25T17:59:38+00:00] INFO: Run List expands to [opsworks_custom_cookbooks::load, opsworks_custom_cookbooks::execute]
[2013-07-25T17:59:38+00:00] INFO: Starting Chef Run for cupcake.localdomain
[2013-07-25T17:59:38+00:00] INFO: Running start handlers
[2013-07-25T17:59:38+00:00] INFO: Start handlers complete.
[2013-07-25T17:59:38+00:00] DEBUG: No chefignore file found at /opt/aws/opsworks/releases/20130724172340_202/cookbooks/chefignore no files will be ignored
[2013-07-25T17:59:39+00:00] DEBUG: No chefignore file found at /opt/aws/opsworks/releases/20130724172340_202/site-cookbooks/chefignore no files will be ignored
[2013-07-25T17:59:39+00:00] DEBUG: Re-raising exception: Chef::Exceptions::ValidationFailed - Option type must be equal to one of: string, array, hash, symbol!  You passed "boolean".
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:126:in `_pv_equal_to'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:69:in `send'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:69:in `validate'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:66:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:66:in `validate'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:56:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:56:in `validate'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/metadata.rb:388:in `attribute'
/opt/aws/opsworks/releases/20130724172340_202/site-cookbooks/dependencies/metadata.rb:18:in `from_file'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:153:in `apply_ruby_metadata'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:96:in `metadata'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:93:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:93:in `metadata'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:86:in `cookbook_version'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:74:in `tap'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:74:in `cookbook_version'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:95:in `load_cookbook'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:69:in `load_cookbooks'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:68:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:68:in `load_cookbooks'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:67:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:67:in `load_cookbooks'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:213:in `setup_run_context'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:467:in `do_run'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:200:in `run'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:190:in `run_chef_client'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:239:in `run_application'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `loop'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `run_application'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:73:in `run'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/chef-solo:25
/opt/aws/opsworks/current/bin/chef-solo:16:in `load'
/opt/aws/opsworks/current/bin/chef-solo:16
[2013-07-25T17:59:39+00:00] ERROR: Running exception handlers
[2013-07-25T17:59:39+00:00] ERROR: Exception handlers complete
[2013-07-25T17:59:39+00:00] FATAL: Stacktrace dumped to /opt/aws/opsworks/releases/20130724172340_202/chef-stacktrace.out
[2013-07-25T17:59:39+00:00] DEBUG: Chef::Exceptions::ValidationFailed: Option type must be equal to one of: string, array, hash, symbol!  You passed "boolean".
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:126:in `_pv_equal_to'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:69:in `send'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:69:in `validate'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:66:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:66:in `validate'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:56:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/params_validate.rb:56:in `validate'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/metadata.rb:388:in `attribute'
/opt/aws/opsworks/releases/20130724172340_202/site-cookbooks/dependencies/metadata.rb:18:in `from_file'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:153:in `apply_ruby_metadata'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:96:in `metadata'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:93:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:93:in `metadata'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:86:in `cookbook_version'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:74:in `tap'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook/cookbook_version_loader.rb:74:in `cookbook_version'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:95:in `load_cookbook'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:69:in `load_cookbooks'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:68:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:68:in `load_cookbooks'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:67:in `each'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_loader.rb:67:in `load_cookbooks'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:213:in `setup_run_context'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:467:in `do_run'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:200:in `run'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:190:in `run_chef_client'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:239:in `run_application'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `loop'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `run_application'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:73:in `run'
/opt/aws/opsworks/releases/20130724172340_202/vendor/gems/chef-11.4.4/bin/chef-solo:25
/opt/aws/opsworks/current/bin/chef-solo:16:in `load'
/opt/aws/opsworks/current/bin/chef-solo:16
[2013-07-25T17:59:39+00:00] FATAL: Chef::Exceptions::ValidationFailed: Option type must be equal to one of: string, array, hash, symbol!  You passed "boolean".

The offending lines in the cookbook seem to be:

https://github.com/aws/opsworks-cookbooks/blob/master/dependencies/metadata.rb#L22
https://github.com/aws/opsworks-cookbooks/blob/master/dependencies/attributes/default.rb#L7

Berkshelf cookbooks directory not deleted before new checkout

Starting with v307, Chef runs with "Manage Berkshelf" set to "yes" fail when checking out custom cookbooks. The initial Setup run when an instance boots succeeds, but the Configure run and any command after that fails.

This is caused by not deleting the /opt/aws/opsworks/current/berkshelf-cookbooks directory before calling berks vendor with Berkshelf 3.

Rails App Server with Unicorn Error

When deploying a rails app with unicorn, there is a configuration error with access to the unicorn socket.

2013/11/20 21:40:05 [crit] 4482#0: *11 connect() to unix:/srv/www/pof/shared/sockets/unicorn.sock failed (2: No such file or directory) while connecting to upstream, client: 129.74.119.192, server: pof, request: "GET / HTTP/1.1", upstream: "http://unix:/srv/www/pof/shared/sockets/unicorn.sock:/", host: "54.204.136.159

The socket file is there. From what I have read, there is an issue with permissions.

For reference...

#10

https://forums.aws.amazon.com/thread.jspa?threadID=118236

OpsWorks stays on "configure" step forever

Lately I use OpsWorks to run instances with my customs cookbooks and recipes. In particular, one of them runs a Instance with a remote_file Chef command. This command download a big file (~ 350MB) from a web URL. I dont know but that recipe is the only one staying on configure step.

The /var/lib/aws/opsworks/chef/XXXX-XX-XX-XX-XX-XX-XX.log log file says:
[2013-12-02T17:37:32+00:00] INFO: Chef Run complete in 152.972258 seconds
[2013-12-02T17:37:32+00:00] INFO: Running report handlers
[2013-12-02T17:37:32+00:00] INFO: Report handlers complete
[2013-12-02T17:37:32+00:00] DEBUG: Exiting

That means recipe finished successfully.

And the /var/log/aws/opsworks/opsworks-agent.process_command.log file says
[2013-12-02 18:12:04] ERROR [opsworks-agent(850)]: process_command: Error during perform: OpenSSL::SSL::SSLError - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed - /usr/lib/ruby/1.8/net/http.rb:586:in connect' /usr/lib/ruby/1.8/net/http.rb:586:inconnect'
/usr/lib/ruby/1.8/net/http.rb:553:in do_start' /usr/lib/ruby/1.8/net/http.rb:548:instart'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/net/http/connection_pool/session.rb:118:in start' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/net/http/connection_pool.rb:208:in_create_session'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/net/http/connection_pool.rb:193:in session_for' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/net/http/connection_pool.rb:171:inrequest'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/net/http/connection_pool/connection.rb:173:in request' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/http/net_http_handler.rb:66:inhandle'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:233:in make_sync_request' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:259:inretry_server_errors'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:226:in make_sync_request' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:451:inclient_request'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:334:in log_client_request' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:420:inclient_request'
/opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:316:in return_or_raise' /opt/aws/opsworks/current/vendor/bundle/ruby/1.8/gems/aws-sdk-1.8.1.2/lib/aws/core/client.rb:419:inclient_request'

In summary, the recipe finished and install everything, but Opsworks still on configure step and the consequence is I cannot execute more commands via run_command

Please, help.

Circular dependency: dependencies, opsworks_nodejs, deploy, dependencies

There's a circular dependency when trying to install anything that depends on dependencies with Berkshelf 3.0.0.beta8 and above: dependencies -> opsworks_nodejs -> deploy -> dependencies.

This results in Berkshelf hitting the maximum stack depth and crashing when trying to resolve transitive dependencies.

Should this be considered a bug in Berkshelf/its dependency resolver, or is it something that can be addressed by rearranging the OpsWorks cookbook dependencies?

gem_package fails outside of setup lifecycle event.

It seems that node[:dependencies] does not exist outside of the setup lifecycle event. The problem with this is that when running gem_package that attribute is now required due to a monkey patch added to GemPackage.

see: https://github.com/aws/opsworks-cookbooks/blob/master-chef-11.4/opsworks_commons/libraries/gem_package.rb#L10

      def gem_binary( gem_cmd = node[:dependencies][:gem_binary] )
        set_or_return(:gem_binary,gem_cmd,:kind_of => [ String ])
      end

What this means is that if you want to install a gem it needs to take place within a lifecycle event that includes the opsworks dependencies cookbook.

HAProxy cookbook doesn't use rpm_url attribute

Trying to install haproxy 1.5-dev19.

If I set the the rpm_url attribute, version 1.4 from the azmn-main repo still gets installed. Setting version/patchlevels doesn't seem to change what gets installed at all.

lvm cookbook inclusion causes setup failure on 11.10

It appears that there's some sort of conflict on the chef 11.10 release branch with the opscode lvm cookbook (I've tested everything from 1.0.0 to 1.0.8 with the same result).

When a cookbook that references lvm via metadata is includes in the run list, it leads to various failures, usually manifesting as failure to call split on nilClass, but in essence these are red herrings (or so it seems to me at present). Oddly, a second run resolves the issue.

Anyone else having issues with lvm on 11.10?

X-Request-Start "t=${msec}" Breaks on Ubuntu 12.04

7fa3d1f broke deployents on 12.04.

msec doesn't exist as vairable on 12.04, and this code should be conditionalized on deployments to a newer nginx.

aws-sdk gem for chef runtime is way out of date

I'd like to have my chef recipes manipulate some opsworks tasks to fill out some templates, but the version installed by opsworks for the chef runtime is pre-opsworks (1.8.1.2). I've tried:

chef_gem "aws-sdk" do
  action [:purge]
  version '1.8.1.2'
end

chef_gem "aws-sdk" do
  action [:install,:upgrade]
  version '1.29.0'
end

To no avail.

Bug: monit configuration for nodejs apps

The stop command seems wrong:

https://github.com/aws/opsworks-cookbooks/blob/master-chef-11.4/opsworks_nodejs/templates/default/node_web_app.monitrc.erb#L3

Should be wrapped in /bin/sh -c '...' as well. At least this doesn't work on precise at all. Monit will report the service is no longer monitored, but the service is still running in the background.

Hence, deploys of nodejs apps do not work either since the code is not reloaded.

opsworks_stack_state_sync does not run

Hi folks,

We rely on the local hostnames injected into /etc/hosts by the opsworks_stack_state_sync job but we've found that sometimes it doesn't run, especially if you re-run the setup using the opsworks-agent-cli.

Also, examining the output of get_json, we see for example that the monitoring_master is not defined (although the connection to our Ganglia instance seems to be up).

Can the opsworks_stack_state_sync job be run manually or more reliably?

Translate special characters to entity elements for database passwords

Our RDB database password has an ampersand. When opsworks creates my context XML file it passes in the '&' character instead of &, which causes a parsing error.

Here's a table of them for XML I found on the internet:

symbol	escaped
&	&
<	<
>	>
"	"
'	'

/etc/hosts values lost when opsworks_stack_state_sync runs

We currently have an issue where any appended values to /etc/hosts in our cookbooks are lost when the opsworks_stack_state_sync job run. We have a section for Additional hosts like this:

https://gist.github.com/khouse/6159453

We believe there needs to be a way to append the opsworks values into /etc/hosts without losing all the current content

https://github.com/aws/opsworks-cookbooks/tree/master-chef-11.4/opsworks_stack_state_sync

Haproxy Cookbook Version is wrong

I'm working on getting haproxy 1.5-dev17 installed so I can use some of the newer features. In the process I realized that the haproxy cookbook in this repo doesn't match the installed cookbook on my haproxy LB instances. In fact, the installed version already has functionality for specifying a different rpm to install. Please update this repo.

Define the number of releases kept

According to the docs on http://docs.opscode.com/resource_deploy.html#attributes chef allows to define the number of releases kept on the server with the keep_releases attribute which defaults to five.

Unfortunately, there seems to be no way to define a different value as the deploy resource definition doesn't care about it. see https://github.com/aws/opsworks-cookbooks/blob/master-chef-11.4/deploy/definitions/opsworks_deploy.rb#L65

I'm currently working under heavy storage constraints and can't afford to keep more than one or two old releases. Adding support for this attribute would be awesome.

Custom cookbook installation fails on second start with EBS

Steps to repro:

Have at least one custom cookbook.
Create and start an EBS backed instance.
Stop instance after it reaches online state.
Add at least one additional custom cookbook.
Start the EBS backed instance again.
The new custom cookbook is not available.

Expected:
Any new cookbooks or updates to existing cookbooks would be available in /opt/aws/opsworks/current/site-cookbooks.

This appears to be an issue with the only_if guard here: https://github.com/aws/opsworks-cookbooks/blob/master/opsworks_custom_cookbooks/recipes/checkout.rb#L51

When using EBS backed instances, it will see that the directory already exists and fail to update the custom cookbooks.

Update Node.js to v0.10.21 security update

This is also needed by all node.js apps that run on 0.10 because this also fixes a bug with closing connections too soon.

Add cache-control: public to nginx for Rails apps

...because any apps that use asset precompilation will benefit immensely from having the cache-control: public header set on all static assets. These assets will be postfixed with a hash of their contents anyway, so max-age can also be set to a year.

At the very least, this can be an optional. There's no other way to solve this problem: even with a CDN, changing headers usually isn't allowed, with unnecessarily results in more requests, less speed, and higher costs.

NodeJS package gets reinstalled on each deploy

The recipe for installing the node js package installs it even if it's the same version as already installed. (I think..)

Switching to using the package resource install action and specifying the source would prevent this.

It's here: https://github.com/aws/opsworks-cookbooks/blob/master/opsworks_nodejs/recipes/default.rb

Custom recipe fails during instance setup but works from "execute_recipes" in Stack

Trying to install mongodb from https://github.com/alexgorbatchev/d-opsworks

Here's my layer config

Here's the error I get

...
[Thu, 02 May 2013 23:00:20 +0000] INFO: service[monit]: restarted successfully
[Thu, 02 May 2013 23:00:20 +0000] DEBUG: Processing ruby_block[restart node.js application tinytypo_test] on nodejs-app1.localdomain
[Thu, 02 May 2013 23:00:20 +0000] INFO: restart node.js via: monit restart node_web_app_tinytypo_test
[Thu, 02 May 2013 23:00:21 +0000] INFO: 
[Thu, 02 May 2013 23:00:21 +0000] INFO: apt_repository[10gen] sending run action to execute[apt-get update]immediately (delayed)
[Thu, 02 May 2013 23:00:21 +0000] ERROR: Running exception handlers
[Thu, 02 May 2013 23:00:21 +0000] ERROR: Exception handlers complete
[Thu, 02 May 2013 23:00:27 +0000] DEBUG: Re-raising exception: NoMethodError - undefined method `run_action' for #<Hash:0x7fd312efcbe8>
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/runner.rb:53:in `run_action'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/runner.rb:101:in `converge'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/runner.rb:97:in `each'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/runner.rb:97:in `converge'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/client.rb:268:in `converge'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/client.rb:158:in `run'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/application/solo.rb:190:in `run_application'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/application/solo.rb:181:in `loop'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/application/solo.rb:181:in `run_application'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/../lib/chef/application.rb:62:in `run'
/opt/aws/opsworks/releases/20130429142335_115/vendor/bundle/ruby/1.8/gems/chef-0.9.15.5/bin/chef-solo:25
/opt/aws/opsworks/current/bin/chef-solo:16:in `load'
/opt/aws/opsworks/current/bin/chef-solo:16

And here's the same two recipes executed in from the stack's run command.

I would love it to be consistent... not working so that I can fix it, or working so that I don't have to :)

Any idea? What am i doing wrong?

runit cookbook not working on Amazon Linux

The runit cookbook doesn't support Amazon Linux, so when you run it on AL, it just fails silently. runit package won't be installed at all.

opsworks_ganglia fails not_if test for remote_file fetch on Ubuntu

This happens because the following line checks for the package architecture. Ubuntu's default format for package queries doesn't include the architecture.

`dpkg-query --show #{package_name} | cut -f 2`.chomp.eql?(node[:ganglia][:package_arch])

memcached cluster not supported

As we have a couple of OpsWorks instances running that host a Rails application, I wanted to utilize the spare memory of each of these instances for memory caching. The instances have enough memory free that it seems unreasonable to launch a separate instance just for memcached. So I assigned all instances the "Memcached" layer in addition to "Rails App Server".

Memcached itself as well as the Ruby client Dalli support clustering. OpsWorks doesn't provide the info: the memcached.yml contain only one IP. I guess it's just the IP of the first Memcached instance. Additional instances are not listed.

As mentioned in https://forums.aws.amazon.com/message.jspa?messageID=428260 the json contains the key node[:opsworks][:layers][:memcached][:instances] which lists all instances.

I'd like to have the generated memcached.yml not only contain one IP address but the IP addresses of all instances of the "Memcached" layer.

Multiple PHP Apps fails on configure

Leaving the "DocumentRoot" empty.

DocumentRoot <%= @params[:docroot] %>

PS: I fixed and I will send a pull request next.
PS 2: Support Case ID 123158551

ruby_enterprise is always installed on Chef11

Summary

Please fix dependencies cookbook in order for OpsWorks users to include/exclude ruby_enterprise depending on node[:opsworks][:ruby_version]. Or alternatively, it's nice as well if ruby_enterprise::default recipe checks node[:opsworks][:ruby_version] before its installation.

Description

I've found out an issue where OpsWorks with Chef11 always runs site-cookbooks on Ruby 1.8. As long as I looked into the issue, this is caused because the following entry is included in dependencies/metadata.rb:

depends "ruby_enterprise"

It is impossible to change Ruby version even though I specify the following in the custom JSON:

 "opsworks" : {
    "ruby_version" : "1.9.3"
}

This is the output of ruby -v:

ubuntu@akkorokamui:~$ ruby -v
ruby 1.8.7 (2012-02-08 MBARI 8/0x6770 on patchlevel 358) [x86_64-linux], MBARI 0x6770, Ruby Enterprise Edition 2012.02

The reason why I need to remove Ruby 1.8.7 is I want to install gems requiring Ruby 1.9+ such as fog.

Here is a log when an error occurred because of version mismatch (--disable-gems works only for 1.9 or later whereas the installed ruby /usr/local/bin/ruby is 1.8.7).

[2013-08-01T06:39:25+00:00] DEBUG: Mixlib::ShellOut::ShellCommandFailed: ruby_block[Compile Custom OpsWorks Run List]
(opsworks_custom_cookbooks::execute line 3) had an error: Mixlib::ShellOut::ShellCommandFailed: execute[Updating Rubygems to 1.8.24]
(opsworks_rubygems::default line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '2'
---- Begin output of /usr/bin/env LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 /usr/local/bin/ruby --disable-gems setup.rb --no-rdoc --no-ri ----
STDOUT: 
STDERR: /usr/local/bin/ruby: invalid option --disable-gems  (-h will show valid options)
---- End output of /usr/bin/env LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 /usr/local/bin/ruby --disable-gems setup.rb --no-rdoc --no-ri ----
Ran /usr/bin/env LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 /usr/local/bin/ruby --disable-gems setup.rb --no-rdoc --no-ri returned 2
/opt/aws/opsworks/releases/20130726231932_203/vendor/bundle/ruby/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:248:in `invalid!'
/opt/aws/opsworks/releases/20130726231932_203/vendor/bundle/ruby/1.8/gems/mixlib-shellout-1.1.0/lib/mixlib/shellout.rb:234:in `error!'
(snip)

Hope this will be fixed.

Fix rails deploy permissions

Add group to deploy resource, without this deploy will chown deploy.root everything.

diff --git a/deploy/definitions/opsworks_deploy.rb b/deploy/definitions/opsworks
index d7d0c05..39b07c2 100644
--- a/deploy/definitions/opsworks_deploy.rb
+++ b/deploy/definitions/opsworks_deploy.rb
@@ -60,6 +60,7 @@ define :opsworks_deploy do
deploy deploy[:deploy_to] do
repository deploy[:scm][:repository]
user deploy[:user]

```
 group deploy[:group]
```
revision deploy[:scm][:revision]
migrate deploy[:migrate]
migration_command deploy[:migrate_command]

Ruby patchlevels should be upgraded due to security update

There's a vulnerability in the current patchlevels of the installed ruby versions: https://www.ruby-lang.org/en/news/2013/11/22/heap-overflow-in-floating-point-parsing-cve-2013-4164/

I believe there should be a way to override the patch level of the ruby being installed in the stack configuration? However it seems that this would only work if the correct package is uploaded to the assets s3 bucket.

How to increase deploy timeout limit at AWS Opsworks?

I would like to increase the deploy time, in a stack layer that hosts many apps (AWS Opsworks). Currenlty I get the following error:

 Mixlib::ShellOut::CommandTimeout: Command timed out after 600s:

Split out cookbooks?

Hey guys, thanks for sharing all of these, but any chance you can split them out? I think that's now the official opscode recommendation :)

Help on opsworks_java

Hi,
I am trying to deploy an app named abcd with artifact as abcd.war. I want to configure to an external datasource. Below is my abcd.war/META-INF/context.xml file

Context
ResourceLink global="jdbc/abcdDataSource1" name="jdbc/abcdDataSource1" type="javax.sql.DataSource" ResourceLink global="jdbc/abcdDataSource2" name="jdbc/abcdDataSource2" type="javax.sql.DataSource"
Context

I configured the below custom JSON during a deployment

{
"datasources": {
"fa": "jdbc/abcdDataSource1",
"fa": "jdbc/abcdDataSource2"
},
"deploy": {
"fa": {
"database": {
"username": "un",
"password": "pass",
"database": "ds1",
"host": "reserved-alpha-db.abcd.us-east-1.rds.amazonaws.com",
"adapter": "mysql"
},
"database": {
"username": "un",
"password": "pass",
"database": "ds2",
"host": "reserved-alpha-db.abcd.us-east-1.rds.amazonaws.com",
"adapter": "mysql"
}
}
}
}

I also added the recipe opsworks_java::context during configure phase. But it doesnt seem like working and I always get the message as below

[2014-01-11T16:12:48+00:00] INFO: Processing template[context file for abcd] action create (opsworks_java::context line 16)
[2014-01-11T16:12:48+00:00] DEBUG: Skipping template[context file for abcd] due to only_if ruby block

Can anyone please help on what I am missing with OpsWorks configuration?

Suggested changes for NodeJs

There are quite a few things in the initial setup that are inconsistent with how NodeJs apps are usually set up. I have created a fork of the official cookbook to fix these things and also added MongoDb and Redis.

I will try to split these changes into separate pull requests so you can accept them individually but for anyone that are interested in the meantime, please give your thoughts to the implementation here:
irony/opsworks-cookbooks

Github Repository is behind

I've been doing a lot of cookbook development on top of this repository the last few days. I just discovered that the cookbooks that are actually installed on my instances are ahead of what is available here. It would really help to have those changes published here at the same time they're pushed out to our instances.

Node.js App Server layer includes deprecated opsworks_nodejs::npm recipe (twice)

It doesn't do anything, no need to be there, yes?

https://github.com/aws/opsworks-cookbooks/blob/master/opsworks_nodejs/recipes/npm.rb

opsworks_custom_cookbooks fails with S3 zip files

I'm using a zip file to provide custom cookbooks.
During instance setup, everything goes fine until 'scm "Download Custom Cookbooks" '
https://github.com/aws/opsworks-cookbooks/blob/master/opsworks_custom_cookbooks/recipes/checkout.rb#L27

At this point, Chef fails because git submodule init && git submodule update returns a non-zero status.

See the logs below. It starts right after prepare_s3_checkouts.

[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: STDERR: 
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: ---- End output of git init; git add .; git commit -m 'Create temporary repository from downloaded contents.' ----
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Ran git init; git add .; git commit -m 'Create temporary repository from downloaded contents.' returned 0
[Tue, 26 Feb 2013 01:38:03 +0000] INFO: Ran execute[create git repository] successfully
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Processing scm[Download Custom Cookbooks] on mynode-00.localdomain
[Tue, 26 Feb 2013 01:38:03 +0000] INFO: Cloning repo /tmp/opsworks20130226-834-1e0sh27/archive.d to /opt/aws/opsworks/current/site-cookbooks
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Executing git clone  /tmp/opsworks20130226-834-1e0sh27/archive.d /opt/aws/opsworks/current/site-cookbooks
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: ---- Begin output of git clone  /tmp/opsworks20130226-834-1e0sh27/archive.d /opt/aws/opsworks/current/site-cookbooks ----
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: STDOUT: Cloning into '/opt/aws/opsworks/current/site-cookbooks'...
done.
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: STDERR: 
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: ---- End output of git clone  /tmp/opsworks20130226-834-1e0sh27/archive.d /opt/aws/opsworks/current/site-cookbooks ----
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Ran git clone  /tmp/opsworks20130226-834-1e0sh27/archive.d /opt/aws/opsworks/current/site-cookbooks returned 0
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Executing git ls-remote /tmp/opsworks20130226-834-1e0sh27/archive.d HEAD
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Executing git ls-remote /tmp/opsworks20130226-834-1e0sh27/archive.d HEAD
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: ---- Begin output of git ls-remote /tmp/opsworks20130226-834-1e0sh27/archive.d HEAD ----
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: STDOUT: 183e5968347b2c5eb192855230937507f55f32b3   HEAD
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: STDERR: 
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: ---- End output of git ls-remote /tmp/opsworks20130226-834-1e0sh27/archive.d HEAD ----
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Ran git ls-remote /tmp/opsworks20130226-834-1e0sh27/archive.d HEAD returned 0
[Tue, 26 Feb 2013 01:38:03 +0000] INFO: Checking out branch: HEAD reference: 183e5968347b2c5eb192855230937507f55f32b3
[Tue, 26 Feb 2013 01:38:03 +0000] DEBUG: Executing git checkout -b deploy 183e5968347b2c5eb192855230937507f55f32b3
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: ---- Begin output of git checkout -b deploy 183e5968347b2c5eb192855230937507f55f32b3 ----
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: STDOUT: 
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: STDERR: Switched to a new branch 'deploy'
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: ---- End output of git checkout -b deploy 183e5968347b2c5eb192855230937507f55f32b3 ----
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: Ran git checkout -b deploy 183e5968347b2c5eb192855230937507f55f32b3 returned 0
[Tue, 26 Feb 2013 01:38:04 +0000] INFO: Enabling git submodules
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: Executing git submodule init && git submodule update
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: ---- Begin output of git submodule init && git submodule update ----
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: STDOUT: 
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: STDERR: No submodule mapping found in .gitmodules for path 'mongodb'
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: ---- End output of git submodule init && git submodule update ----
[Tue, 26 Feb 2013 01:38:04 +0000] DEBUG: Ran git submodule init && git submodule update returned 1
[Tue, 26 Feb 2013 01:38:04 +0000] ERROR: scm[Download Custom Cookbooks] (/opt/aws/opsworks/releases/20130225142901_104/cookbooks/opsworks_custom_cookbooks/recipes/checkout.rb:27:in `from_file') had an error:
git submodule init && git submodule update returned 1, expected 0
---- Begin output of git submodule init && git submodule update ----
STDOUT: STDERR: No submodule mapping found in .gitmodules for path 'mongodb'
---- End output of git submodule init && git submodule update ----

/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/mixin/command.rb:184:in `handle_command_failures'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/mixin/command.rb:131:in `run_command'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/provider/git.rb:128:in `enable_submodules'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/provider/git.rb:44:in `action_checkout'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource.rb:406:in `send'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource.rb:406:in `run_action'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/runner.rb:53:in `run_action'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/runner.rb:89:in `converge'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/runner.rb:89:in `each'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/runner.rb:89:in `converge'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/runner.rb:84:in `converge'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/client.rb:268:in `converge'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/client.rb:158:in `run'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/application/solo.rb:190:in `run_application'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/application/solo.rb:181:in `loop'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/application/solo.rb:181:in `run_application'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/../lib/chef/application.rb:62:in `run'
/opt/aws/opsworks/releases/20130225142901_104/vendor/bundle/ruby/1.8/gems/chef-0.9.15.3/bin/chef-solo:25
/opt/aws/opsworks/current/bin/chef-solo:16:in `load'
/opt/aws/opsworks/current/bin/chef-solo:16