Git Product home page Git Product logo

application_python's Introduction

Poise

Build Status Gem Version Cookbook Version Coverage Gemnasium License

What is Poise?

The poise cookbook is a set of libraries for writing reusable cookbooks. It provides helpers for common patterns and a standard structure to make it easier to create flexible cookbooks.

Writing your first resource

Rather than LWRPs, Poise promotes the idea of using normal, or "heavy weight" resources, while including helpers to reduce much of boilerplate needed for this. Each resource goes in its own file under libraries/ named to match the resource, which is in turn based on the class name. This means that the file libraries/my_app.rb would contain Chef::Resource::MyApp which maps to the resource my_app.

An example of a simple shell to start from:

require 'poise'
require 'chef/resource'
require 'chef/provider'

module MyApp
  class Resource < Chef::Resource
    include Poise
    provides(:my_app)
    actions(:enable)

    attribute(:path, kind_of: String)
    # Other attribute definitions.
  end

  class Provider < Chef::Provider
    include Poise
    provides(:my_app)

    def action_enable
      notifying_block do
        ... # Normal Chef recipe code goes here
      end
    end
  end
end

Starting from the top, first we require the libraries we will be using. Then we create a module to hold our resource and provider. If your cookbook declares multiple resources and/or providers, you might want additional nesting here. Then we declare the resource class, which inherits from Chef::Resource. This is similar to the resources/ file in an LWRP, and a similar DSL can be used. We then include the Poise mixin to load our helpers, and then call provides(:my_app) to tell Chef this class will implement the my_app resource. Then we use the familiar DSL, though with a few additions we'll cover later.

Then we declare the provider class, again similar to the providers/ file in an LWRP. We include the Poise mixin again to get access to all the helpers and call provides() to tell Chef what provider this is. Rather than use the action :enable do ... end DSL from LWRPs, we just define the action method directly. The implementation of action comes from a block of recipe code wrapped with notifying_block to capture changes in much the same way as use_inline_resources, see below for more information about all the features of notifying_block.

We can then use this resource like any other Chef resource:

my_app 'one' do
  path '/tmp'
end

Helpers

While not exposed as a specific method, Poise will automatically set the resource_name based on the class name.

Notifying Block

As mentioned above, notifying_block is similar to use_inline_resources in LWRPs. Any Chef resource created inside the block will be converged in a sub-context and if any have updated it will trigger notifications on the current resource. Unlike use_inline_resources, resources inside the sub-context can still see resources outside of it, with lookups propagating up sub-contexts until a match is found. Also any delayed notifications are scheduled to run at the end of the main converge cycle, instead of the end of this inner converge.

This can be used to write action methods using the normal Chef recipe DSL, while still offering more flexibility through subclassing and other forms of code reuse.

Include Recipe

In keeping with notifying_block to implement action methods using the Chef DSL, Poise adds an include_recipe helper to match the method of the same name in recipes. This will load and converge the requested recipe.

Resource DSL

To make writing resource classes easier, Poise exposes a DSL similar to LWRPs for defining actions and attributes. Both actions and default_action are just like in LWRPs, though default_action is rarely needed as the first action becomes the default. attribute is also available just like in LWRPs, but with some enhancements noted below.

One notable difference over the standard DSL method is that Poise attributes can take a block argument.

Template Content

A common pattern with resources is to allow passing either a template filename or raw file content to be used in a configuration file. Poise exposes a new attribute flag to help with this behavior:

attribute(:name, template: true)

This creates four methods on the class, name_source, name_cookbook, name_content, and name_options. If the name is set to '', no prefix is applied to the function names. The content method can be set directly, but if not set and source is set, then it will render the template and return it as a string. Default values can also be set for any of these:

attribute(:name, template: true, default_source: 'app.cfg.erb',
          default_options: {host: 'localhost'})

As an example, you can replace this:

if new_resource.source
  template new_resource.path do
    source new_resource.source
    owner 'app'
    group 'app'
    variables new_resource.options
  end
else
  file new_resource.path do
    content new_resource.content
    owner 'app'
    group 'app'
  end
end

with simply:

file new_resource.path do
  content new_resource.content
  owner 'app'
  group 'app'
end

As the content method returns the rendered template as a string, this can also be useful within other templates to build from partials.

Lazy Initializers

One issue with Poise-style resources is that when the class definition is executed, Chef hasn't loaded very far so things like the node object are not yet available. This means setting defaults based on node attributes does not work directly:

attribute(:path, default: node['myapp']['path'])
...
NameError: undefined local variable or method 'node'

To work around this, Poise extends the idea of lazy initializers from Chef recipes to work with resource definitions as well:

attribute(:path, default: lazy { node['myapp']['path'] })

These initializers are run in the context of the resource object, allowing complex default logic to be moved to a method if desired:

attribute(:path, default: lazy { my_default_path })

def my_default_path
  ...
end

Option Collector

Another common pattern with resources is to need a set of key/value pairs for configuration data or options. This can done with a simple Hash, but an option collector attribute can offer a nicer syntax:

attribute(:mydata, option_collector: true)
...

my_app 'name' do
  mydata do
    key1 'value1'
    key2 'value2'
  end
end

This will be converted to {key1: 'value1', key2: 'value2'}. You can also pass a Hash to an option collector attribute just as you would with a normal attribute.

Debugging Poise

Poise has its own extra-verbose level of debug logging that can be enabled in three different ways. You can either set the environment variable $POISE_DEBUG, set a node attribute node['POISE_DEBUG'], or touch the file /POISE_DEBUG. You will see a log message Extra verbose logging enabled at the start of the run to confirm Poise debugging has been enabled. Make sure you also set Chef's log level to debug, usually via -l debug on the command line.

Upgrading from Poise 1.x

The biggest change when upgrading from Poise 1.0 is that the mixin is no longer loaded automatically. You must add require 'poise' to your code is you want to load it, as you would with normal Ruby code outside of Chef. It is also highly recommended to add provides(:name) calls to your resources and providers, this will be required in Chef 13 and will display a deprecation warning if you do not. This also means you can move your code out of the Chef module namespace and instead declare it in your own namespace. An example of this is shown above.

Sponsors

The Poise test server infrastructure is generously sponsored by Rackspace. Thanks Rackspace!

License

Copyright 2013-2016, Noah Kantrowitz

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

application_python's People

Contributors

amccloud avatar bastien-jove-cbp avatar coderanger avatar comandrei avatar farshidce avatar guilhem avatar jblaine avatar lamont-granquist avatar mbacchi avatar mpaolini avatar rcloran avatar sethvargo avatar tinnet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

application_python's Issues

Seems like git is broken

Thoughts or ideas why I would get this?

Generated at 2016-06-30 19:26:02 +0000
Mixlib::ShellOut::ShellCommandFailed: git[https://github.com/moos3/captain-slackhook.git] (wiw_slackhook::default line 7) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '128'
---- Begin output of git ls-remote "" "HEAD" ----
STDOUT:
STDERR: fatal: No path specified. See 'man git-pull' for valid url syntax
---- End output of git ls-remote "" "HEAD" ----
Ran git ls-remote "" "HEAD" returned 128
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:289:in `invalid!'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout.rb:276:in `error!'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/shell_out.rb:56:in `shell_out!'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:286:in `git_ls_remote'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:244:in `remote_resolve_reference'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:230:in `target_revision'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider/git.rb:65:in `block (2 levels) in define_resource_requirements'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:231:in `call'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:231:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:321:in `block in run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:320:in `each'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/mixin/why_run.rb:320:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider.rb:153:in `process_resource_requirements'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/provider.rb:132:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource.rb:596:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:69:in `run_action'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:97:in `block (2 levels) in converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:97:in `each'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:97:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/resource_list.rb:84:in `block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/resource_collection/resource_list.rb:82:in `execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/runner.rb:96:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:667:in `block in converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:662:in `catch'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:662:in `converge'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:701:in `converge_and_save'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/client.rb:281:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:285:in `block in fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:273:in `fork'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:273:in `fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:238:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:226:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:301:in `block in interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:290:in `loop'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:290:in `interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application/solo.rb:269:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/lib/chef/application.rb:58:in `run'
/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.9.41/bin/chef-solo:25:in `<top (required)>'

Sync/Migrate Multiple Databases?

I have an django application that uses multiple databases and requires multiple syncdb/migrate commands to be executed to sync all of the databases. Is there a possible work-around for this?

systemd service name collisions

On an Ubuntu 16.04 system, I observed that an application created via this cookbook will attempt to create multiple systemd services with the same name.

Am I misunderstanding the README by mixing a gunicorn and celery_worker resource in the same application resource?

Given the following cookbook code:

application '/opt/myapp' do
  git '[email protected]:myorg/myapp.git'
  pip_requirements
  python '2.7'
  gunicorn do
    port 9001
  end
  celery_worker do
   app_module 'app.tasks'
  end
end

this will happen:

* application_gunicorn[/opt/myapp] action enable
  * python_package[gunicorn] action install (up to date)
  * poise_service[/opt/myapp] action enable
    * execute[systemctl daemon-reload] action nothing (skipped due to action :nothing)
    * template[/etc/systemd/system/myapp.service] action create
      - create new file /etc/systemd/system/myapp.service
      - update content in file /etc/systemd/system/myapp.service from none to bd37a3
      --- /etc/systemd/system/myapp.service	2017-05-29 17:46:47.668598009 +0000
      +++ /etc/systemd/system/.chef-myapp20170529-24061-1i17040.service	2017-05-29 17:46:47.636598009 +0000
      @@ -1 +1,15 @@
      +[Unit]
      +Description=myapp
      +
      +[Service]
      +Environment=
      +ExecStart=/usr/bin/python2.7 -m gunicorn.app.wsgiapp --bind 0.0.0.0:9001 setup
      +ExecReload=/bin/kill -HUP $MAINPID
      +KillSignal=TERM
      +User=root
      +WorkingDirectory=/opt/myapp
      +Restart=on-failure
      +
      +[Install]
      +WantedBy=multi-user.target
      - change mode from '' to '0644'
      - change owner from '' to 'root'
      - change group from '' to 'root'
    * execute[systemctl daemon-reload] action run
      - execute systemctl daemon-reload
    * service[myapp] action enable
      - enable service service[myapp]
    * service[myapp] action start
      - start service service[myapp]
  

* application_celery_worker[/opt/myapp] action enable
  * poise_service[/opt/myapp] action enable
    * execute[systemctl daemon-reload] action nothing (skipped due to action :nothing)
    * template[/etc/systemd/system/myapp.service] action create
      - update content in file /etc/systemd/system/myapp.service from bd37a3 to a61b3d
      --- /etc/systemd/system/myapp.service	2017-05-29 17:46:47.636598009 +0000
      +++ /etc/systemd/system/.chef-myapp20170529-24061-1fov4ul.service	2017-05-29 17:46:50.100598009 +0000
      @@ -3,7 +3,7 @@

[Service]
Environment=
      -ExecStart=/usr/bin/python2.7 -m gunicorn.app.wsgiapp --bind 0.0.0.0:9001 setup
      +ExecStart=/usr/bin/python2.7 -m celery --app=app.tasks worker
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=TERM
User=root
    * execute[systemctl daemon-reload] action run
      - execute systemctl daemon-reload
    * service[myapp] action enable (up to date)
    * service[myapp] action start (up to date)

COOK-4059: application_python's gunicorn.rb attempts to install requirements attached to django block as root, system-wide, outside of the virtualenv

When attempting to install a Django application using application_python, with a recipe based on the packaginator example, the recipe will install requirements listed in the django block as follows:

  • 1st: From django.rb, install the requirements inside django's Virtualenv, as the owner in the application block, using the GIT_SSH wrapper if one is specified
  • 2nd: From gunicorn.rb, install the requirements inside django block, as root, system-wide (outside of Virtualenv).

It appears that there is some code that attempts to pull in Virtualenv details from the django block if app_module == :django, but this doesn't seem to be used properly in providers/gunicorn.rb line 135-140.
Also, git may refuse to use the wrapper script as root if it is owned by a different application.owner.

This is a problem because it means:

  • Everything in the requirements.txt file is installed twice (once in the virtualenv and once system-wide)
  • The system-wide installation cannot pull from SSH-based git repositories.

This means that code stored in private repositories referenced from a requirements.txt on GitHub/Bitbucket/whatever (git+ssh) will fail to install and have an error of Permission denied (publickey).

As a workaround in my recipe, I've put in the gunicorn block reqirements false. This seems to prevent gunicorn.rb from trying to install any dependencies, or try to be smart by looking for requirements.txt at the root of the repository.

If this is all intended behaviour, then maybe the example for packaginator should be updated and this behaviour be documented.

Understanding 4.0.0 'local_settings'

Having read the poise-developer-centric link to "Poise template property" I don't really understand how to use local_settings:

local_settings โ€“ A Poise template property for the content of the local settings configuration file.

Can someone shed some more light on the topic?

  NoMethodError
  -------------
  undefined method `local_settings' for PoiseApplicationPython::Resources::Django::Resource
   40:    django do
   41:      migrate false
   42>>     local_settings 'settings.py.erb'
   43:      debug true
   44:      database do
   45:        host 'hostdb'
   46:        database 'hostdb'
   47:        adapter  'mysql'
   48:        username chef_vault_item('secrets', 'hostdb')['username']
   49:        password chef_vault_item('secrets', 'hostdb')['password']
   50:        time_zone 'America/New_York'
   51:      end

Possible fix for virtualenv references

I spent some time thinking about how to solve the virtualenv issues with gunicorn installing on the host vs the virtualenv associated with the (django) application.

If we modify the resource to support a symbol argument we can use that symbol to identify another resource in the application resource. As an example you could specify a virtualenv of :django to install to the same virtualenv as that resource.

Check my commit here:
BNOTIONS@a65a9e6

Unable to install local wheel package via "python_package" resource

Before requesting admin upload a new Python package to their repo I'm testing via local install. Installing from command line like so works fine:

source /home/fooapp/env/bin/activate && pip install /tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl

but when I attempt to do via python_package resource I have problems.

Input

# no support for source property so using absolute filename here
python_package /tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl do
  virtualenv conf['virtualenv']
end

Output

================================================================================
Error executing action `install` on resource 'python_package[/tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl]'
================================================================================

Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of ["/home/fooapp/env/bin/python", "-", "/tmp/kitchen/cache/cefpython3-31.2-cp27-none-linux_x86_64.whl"] ----
STDOUT:
STDERR: /home/fooapp/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
 SNIMissingWarning
/home/fooapp/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
 InsecurePlatformWarning
No handlers could be found for logger "pip.index"
Traceback (most recent call last):
 File "<stdin>", line 38, in <module>
 File "/home/fooapp/env/lib/python2.7/site-packages/pip/index.py", line 491, in find_requirement
'No matching distribution found for %s' % req

I use this resource with remote packages all the time with no issues so assume I'm just misusing or otherwise misunderstanding. Appreciate the assistance.

How to add settings to local_settings.py?

In the old application_python django resource, you could specify your own settings.py.erb and pass in variables for it in additional_settings resource attribute. How would one do this with the new poise/application_python?

CONTRIBUTING.md directs people with bugs to Opscode's JIRA instance

When attempting to report a bug following instructions in CONTRIBUTING.md, I recieved the following message:

Seth Vargo closed COOK-4059.
----------------------------

    Resolution: Duplicate

Ohai!

The new maintainer for this cookbook, [~coderanger], has moved the existing issues and Pull Requests to GitHub.

Is current master branch usable?

First, thank you for your work. I am new to chef and i am currently implementing a new cookbook to deploy our django based applications. As the last release is 3.0.0 and it fundamentally differs from the current implementation in the master branch i am wondering if i can use the version in master branch already. I tried to include it via the git repo but chef complains about the missing metadata.rb. So my question is: should i use the old version 3.0.0 or the current master branch? If i can use the master branch, how is that possible?

Thanks much for your help!
Jakob

gunicorn resource "config" option breaks things?

Does anyone have any thoughts how to debug this?

  gunicorn do
    app_module 'hostdb.wsgi'
    port 9999
    config "/root/hostdb.new/gunicorn_config.py"
  end

This results in the service failing due to too many respawns:

Jan 28 17:17:53 rcf-webapps init: hostdb.new main process (5075) terminated with status 1
Jan 28 17:17:53 rcf-webapps init: hostdb.new main process ended, respawning
...
Jan 28 17:17:55 rcf-webapps init: hostdb.new respawning too fast, stopped
# hostdb.new generated by poise-service for poise_service[/root/hostdb.new]

description "hostdb.new"

start on runlevel [2345]
stop on runlevel [!2345]

respawn
respawn limit 10 5
umask 022
chdir /root/hostdb.new
env DJANGO_SETTINGS_MODULE="hostdb.settings"
env PATH="/opt/rh/python27/root/usr/bin:/usr/lib/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/bin:/bin:/sbin:/usr/sbin:/usr/rcf-krb5/bin:/root/bin:/opt/chef/embedded/bin:/opt/chef/embedded/bin"
env LD_LIBRARY_PATH="/opt/rh/python27/root/usr/lib64"
env MANPATH="/opt/rh/python27/root/usr/share/man:"
env XDG_DATA_DIRS="/opt/rh/python27/root/usr/share"
env PKG_CONFIG_PATH="/opt/rh/python27/root/usr/lib64/pkgconfig"

exec /opt/chef/embedded/bin/ruby -e 'Process::UID.change_privilege("apache"); ENV["HOME"] = Dir.home("apache") rescue nil; exec(*["/root/hostdb.new/.virtualenv/bin/python", "-m", "gunicorn.app.wsgiapp", "--config", "/root/hostdb.new/gunicorn_config.py", "--bind", "0.0.0.0:9999", "hostdb.wsgi"])'

If I run gunicorn --config /root/hostdb.new/gunicorn_config.py --bind 0.0.0.0:9999 hostdb.wsgi by hand, it works.

If I comment out the 'config' property in the resource definition, everything works (but without the desired configuration as indicated in the gunicorn config file of course).

The config file contents are:

syslog = True
syslog_prefix = "hostdb_gunicorn"
syslog_facility = "daemon"
loglevel = "info"
errorlog = "/var/log/gunicorn_error.log"
accesslog = "/var/log/gunicorn_access.log"
worker_class = 'gevent'

I have to believe then that something is wrong with the generated upstart init file?

Non-simplistic gunicorn example?

depends 'application', '> 5.0'
depends 'application_python', '
> 4.0'

This is working for me:

...
  gunicorn do
    app_module 'gravatarFlask:app'
    port 80
  end
...

This isn't working for me:

...
  gunicorn do
    app_module 'gravatarFlask:app'
    config '--error-logfile /var/log/gunicorn_error.log --log-syslog-to tcp://speaker.our.org:514 --log-level info --log-syslog-prefix gravatar_gunicorn'
    port 80
  end
...

Any ideas what am I doing wrong?

Chef 14 - #<NameError: instance variable @cwd not defined>

Issue opened after a talk with coderanger, here's my summary, workaround as comment on the issue:

With the rewrite of the execute resource in Chef 14, the following lines create an error:

remove_instance_variable(:@cwd)

Error as seen from chefspec test:

 expected no Exception, got #<NameError: instance variable @cwd not defined> with backtrace:
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application_python/files/halite_gem/poise_application_python/resources/python_execute.rb:43:in `remove_instance_variable'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application_python/files/halite_gem/poise_application_python/resources/python_execute.rb:43:in `initialize'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:140:in `block in declare_resource'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:121:in `declare_resource'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:175:in `public_send'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:175:in `block (2 levels) in _rewire_dsl!'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/app-python/recipes/default.rb:24:in `block in from_file'
         # C:/Users/xxx/AppData/Local/Temp/chefspec20181009-7988-1eo94amfile_cache_path/cookbooks/app-python/recipes/default.rb:21:in `from_file'
         # ./spec/unit/recipes/default_spec.rb:29:in `block (3 levels) in <top (required)>'
         # ./spec/unit/recipes/default_spec.rb:33:in `block (4 levels) in <top (required)>'
         # ./spec/unit/recipes/default_spec.rb:33:in `block (3 levels) in <top (required)>'
     # ./spec/unit/recipes/default_spec.rb:33:in `block (3 levels) in <top (required)>'

COOK-4094 : python_pip resource(s) for packages have no support for pip options

python_pip has the 'options' attribute

This option is not catered to in the protected install_packages() in providers.rb ( https://github.com/poise/application_python/blob/master/providers/django.rb#L111 )

I cannot, therefore, install any pip packages as I am behind an HTTP proxy. And so I am dead in the water and not sure what the right/acceptable approach to the code would be.

Reported here: https://tickets.opscode.com/browse/COOK-4094

application_python failing to install gunicorn on first deploy

The first deploy of an app that has the gunicorn service enabled fails with the following error:

[2013-12-15T07:15:38+00:00] FATAL: Chef::Exceptions::EnclosingDirectoryDoesNotExist: directory[/srv/splots/shared/env](/opt/aws/opsworks/releases/20131125133823_214/site-cookbooks/python/providers/virtualenv.rb line 31) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /srv/splots/shared does not exist, cannot create /srv/splots/shared/env

There is also a longer description of this bug on stackoverflow: http://stackoverflow.com/questions/20351292/application-python-failing-to-install-gunicorn-on-first-deploy

There is a way to give celery worker parameters?

Hi,

I'm using application_celery_worker and I need to set --concurrency parameter, but I didn't found any place talking about parameters to celery service.

There is a way to set this parameter to celery service?

Thanks!

manage.py syncdb --noinput fails to run

during a chef run syncdb fails to run. this is the error, i cannot see what is causing this... any ideas ? cheers

Chef::Exceptions::Exec: deploy_revision[datagether](/var/chef/cache/cookbooks/application/providers/default.rb line 122) had an error: Chef::Exceptions::Exec: /home/wolfpack/datagether/shared/env/bin/python manage.py syncdb --noinput returned 1, expected 0

Application environment not carried over to Django resource

I use an environment variable in my manage.py to determine a specific set of credentials to load. It appears that the environment attribute of the application resource isn't used by the django resource as it has no impact on the django python commands that are execute for collectstatic and migrate.

Here's the example recipe:

application release_dir do
  python '2.7'
  environment ({
    :SITE_CONFIG => deploy_bag.fetch('default_site_config', 'abc'),
  })
...
  django do
    path release_dir
    collectstatic true
    local_settings_path false
    migrate false
  end
end

I don't get a SITE_CONFIG environment variable when manage.py is run from these resources.

unable to deploy python application

Hi,
I'm trying to install a python application from a private git repo.
According to application_python documentation, this is my recipe

application '/opt/sd-cif' do
  packages   ['git', 'swig', 'openssl-devel']
  owner      'centos'
  group      'centos'
  path       '/opt/sd-cif'
  git '/opt/sd-cif' do
    repository '.......'
    deploy_key 'ssh-rsa ..........'
    revision   'master'
  end  
  python_runtime 'sd-cif' do
  provider :scl
  version '3'
  end
  pip_requirements '/opt/sd-cif/requirements.txt'  
  python_virtualenv '/opt/sd-cif'    
end

Error is

[2015-09-30T17:14:02+00:00] DEBUG: Resources for generic application resource enabled on node include: [LWRP resource application from cookbook application]
[2015-09-30T17:14:02+00:00] DEBUG: Resource for application is LWRP resource application from cookbook application
[2015-09-30T17:14:02+00:00] DEBUG: Resources for generic git resource enabled on node include: [Chef::Resource::Git]
[2015-09-30T17:14:02+00:00] DEBUG: Resource for git is Chef::Resource::Git
[2015-09-30T17:14:02+00:00] DEBUG: filtered backtrace of compile error: /var/chef/cache/cookbooks/application_cif/recipes/default.rb:17:in `block (2 levels) in from_file',/var/chef/cache/cookbooks/application_cif/recipes/default.rb:15:in `block in from_file',/var/chef/cache/cookbooks/application_cif/recipes/default.rb:10:in `from_file'
[2015-09-30T17:14:02+00:00] DEBUG: backtrace entry for compile error: '/var/chef/cache/cookbooks/application_cif/recipes/default.rb:17:in `block (2 levels) in from_file''
[2015-09-30T17:14:02+00:00] DEBUG: Line number of compile error: '17'

================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/application_cif/recipes/default.rb
================================================================================

NoMethodError
-------------
undefined method `deploy_key' for Chef::Resource::Git

Environment vars not accessible by service

Hi guys!

I'm trying to fill systemd Environment variables from celery_worker. I saw it has ServiceMixin from poise-service interface but unfortunately I can't find a way to pass my env vars to service environment.

I'm trying to do like this:

application 'blabla' do
   environment ({
     :REMAP_SIGTERM => 'SIGQUIT'
   })

   celery_worker do
     app_module 'tasks'
     user 'celery'
     service_name "myapp"
   end
end

But my Environment entry on systemd service conf still empty.

Am I doing something wrong?

Thanks in advance!
Alisson

Application_python cookbook should take environment variables in file.

I am creating gunicorn service for flask application. My application needs a couple of env variables whose length is increasing 2048, which is systemd LINE_MAX.
If we pass environment variables in form of a file, then this issue can be solved.
This can be the enhancement in application_python to overcome line_max issue.

Allow private GIT repo in pip_requeriments

This is my application definition. All code is in a private GIT server, so I need to use application_git cookbook who provides deploy_key resource to checkout code successfully.

application  "/data/envs/app_name" do
   application_git "/data/envs/app_name" do
         repository  "[email protected]:atomic/app_name.git"
         revision 'develop'
         deploy_key key
   end
   virtualenv
   pip_requirements "/data/envs/app_name/requeriments/development.txt"
   django do
      database do
              engine 'mysql'
              username 'someuser'
              password 'somepassword'
              host     'localhost'
        end
   end
  gunicorn do
        app_module 'django'
        port 8100
        timeout 800
  end
end

So far, so good. The problem arise when pip_requirements finds this line into requeriments/development.txt, who points to an egg archive into another private GIT repo:

-e git+ssh://[email protected]/atomic/puzzlequestserver.git@develop#egg=pqs

In this case, the command fails because lack of permissions in the GIT repo

  Permission denied, please try again.
  Permission denied, please try again.
  Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  Command "git clone -q ssh://[email protected]/atomic/puzzlequestserver.git /data/envs/app_name/.virtualenv/src/pqs" failed with error code 128 in None
       ---- End output of ["/data/envs/app_name/.virtualenv/bin/python", "-m", "pip.__main__", "install", "--requirement", "/data/envs/app_name/requeriments/development.txt"] ----
       Ran ["/data/envs/app_name/.virtualenv/bin/python", "-m", "pip.__main__", "install", "--requirement", "/data/envs/app_name/requeriments/development.txt"] returned 1

How can I install this egg if cookbook don't provides this functionality ?

Django app fails when specifying Postgres port

When I specify a port for postgres, URI parse :5432 as an integer.
The code however checks to see if the value is empty?, which causes the following error:

:15:in `block in parse_database_url': undefined method `empty?' for 5432:Integer (NoMethodError)
	from delme.rb:4:in `tap'
	from delme.rb:4:in `parse_database_url'
	from delme.rb:19:in `<main>'

metadata.rb missing

Hello,

My org maintains an internal supermarket with strict license restrictions, automated by checking the metadata.rb file. This repo doesn't have metadata.rb so I am not able to pull in the cookbook internally despite your licensing being compatible.

Would you please consider adding metadata.rb and tagging a new release?

Thank you!

Cannot find a resource matching python_runtime

Hi, I'm getting a weird error trying to use application_python

Cannot find a resource matching python_runtime[myapp] (did you define it first?)

I'm using Chef: 12.7.2. Here is the relevant part of my recipe...

python_runtime 'myapp' do
  version '2.7'
  options :system, dev_package: false
end

application '/var/www/myapp' do
  python 'myapp'

  pip_requirements do
    path '/var/www/myapp/requirements_prod.txt'
    python 'myapp'
  end

  gunicorn do
    service_name "myapp"
    user app.user
    path "#{app.path}/myapp"
    config "/etc/gunicorn.d/myapp.py"
    bind '127.0.0.1'
    port 9009
  end

  django do
    manage_path '/var/www/myapp/gallery.py'
    debug node[:env] != "prod"
    secret_key app.secret

    database do
      engine 'django.db.backends.mysql'
      host db.host
      name db.name
      user db.user
      password db.password
    end

    migrate true
    collectstatic true
  end
end

Recipe Compile Error on deploying gunicorn

My recipe:

application app_path do
  path app_path
  owner "www-data"
  group "www-data"
  package "git"

  git app_path do
    repository "~~~~"
    user "www-data"
    group "www-data"
  end
  pip_requirements

  gunicorn do
    port 9001
    app_module "manage:app"
  end

  environment ({'HOME' => '/var/www'})
end

Error message:

================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb
================================================================================

Errno::ENOENT
-------------
No such file or directory @ dir_initialize - /usr/share/classifier-api

Cookbook Trace:
---------------

  /tmp/kitchen/cache/cookbooks/application_python/files/halite_gem/poise_application_python/resources/gunicorn.rb:59:in `entries'
  /tmp/kitchen/cache/cookbooks/application_python/files/halite_gem/poise_application_python/resources/gunicorn.rb:59:in `default_app_module'

  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lazy_default.rb:50:in `instance_eval'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lazy_default.rb:50:in `set_or_return'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/lwrp_polyfill.rb:114:in `block in attribute'
  /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb:70:in `block (2 levels) in from_file'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:149:in `instance_exec'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:149:in `block (3 levels) in _rewire_dsl!'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:125:in `instance_exec'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:125:in `block (2 levels) in declare_resource'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:115:in `block in declare_resource'
  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subcontext_block.rb:54:in `instance_eval'

  /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/subresources/container.rb:96:in `declare_resource'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:144:in `public_send'
  /tmp/kitchen/cache/cookbooks/application/files/halite_gem/poise_application/resources/application.rb:144:in `block (2 levels) in _rewire_dsl!'
  /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb:68:in `block in from_file'
  /tmp/kitchen/cache/cookbooks/classifier-api/recipes/api.rb:55:in `from_file'


----------------------
/tmp/kitchen/cache/cookbooks/application_python/files/halite_gem/poise_application_python/resources/gunicorn.rb:

 52:          # checks the app state and then looks for commonly used filenames.
 53:          # Raises an exception if no default can be found.

 55:          # @return [String]
 56:          def default_app_module
 57:            # If set in app_state, use that.
 58:            return app_state[:python_wsgi_module] if app_state[:python_wsgi_module]
 59>>           files = Dir.entries(path)
 60:            # Try to find a known filename.
 61:            candidate_file = %w{wsgi.py main.py app.py application.py}.find {|file| files.include?(file) }
 62:            # Try the first Python file. Do I really want this?
 63:            candidate_file ||= files.find {|file| file.end_with?('.py') }
 64:            if candidate_file
 65:              ::File.basename(candidate_file, '.py')
 66:            else
 67:              raise PoiseApplicationPython::Error.new("Unable to determine app module for #{self}")
 68:            end

It occurs because the deploy target directory is not created yet on the compilation time.
It even happens even if specified app_module attribute.
How can I avoid it? How to specify python_wsgi_module of app_state?

Can not install python v3

Error:

PoiseLanguages::Error
---------------------
Package python would install 2.7.11-1, which does not match 3.4.3. Please set the package_name or package_version provider options.

Code:

application "/srv/xyz" do
  only_if { app['deploy'] }

  git url do
    deploy_key deploy_key
  end
  python '3' do
    version '3.4.3'
  end
  virtualenv
  pip_requirements
  django 'app' do
  end
  gunicorn 'app' do
    port 8000
  end
end

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.