hw-cookbooks / chef-server-populator Goto Github PK

Populate an OSS Chef Server

Ruby 100.00%

chef-server-populator's Introduction

Chef Server Populator

Creates orgs, clients, and admin users and installs provided public keys. Simplifies managing and recreating Chef Server nodes. Provides backup and restore recipes for complete Chef Server recovery.

New Chef 12 Support

Chef 12 is supported in version 1.0 and above. If you need Chef 11 support, please pin your environment to version 0.4.0.

Usage

When bootstrapping with the chef-server cookbook and chef-solo:

Download and unpack chef-server, chef-server-ingredient, packagecloud, and chef-server-populator cookbooks
Upload public keys to be used by users, org-validator, and clients (optionally)
Create json for organization, user, and (optionally) client(s)
Run chef-solo

See the default[:chef_server_populator][:solo_org] and default[:chef_server_populator][:solo_org_user] attribute hashes in attributes/default.rb for the required attribute structure.

When converging with chef-client:

Create data bag to hold data bag items with user, org, and client information
Create data bag items with user, org, and client information
Set data bag related attributes

Applicable attributes:

node[:chef_server_populator][:databag] - name of the data bag

Structure of the data bag item:

User:

{
  "id": "user_name",
  "chef_server": {
    "full_name": "User Name",
    "email": "[email protected]",
    "client_key": "public key contents",
    "type": [
      "user"
    ],
    "orgs": {
      "organization": {
        "enabled": true,
        "admin": true
      }
    }
  }
}

Note: While users can belong to multiple organizations, and the above hash structure allows you to define multiple associations, the chef-server-populator currently only supports the first organization that is defined in the data bag.

Client:

{
  "id": "client_name",
  "chef_server": {
    "client_key": "public key contents",
    "type": [
      "client"
    ],
    "orgs": [ "organization" ]
   }
}

Note: If no organization is specified for a client, it will be added to the default organization. The client enabled and admin settings can be set at the top level of the chef_server hash or in and orgs hash as in the User example.

Org:

{
  "id": "org_name",
  "chef_server": {
    "full_name": "Organization Name",
    "client_key": "public key contents",
    "type": [
      "org"
    ],
    "enabled": true
  }
}

Note: Creating the org will create a client called <org>-validator which uses the public key specified when creating the org. In addition, there is currently a bug in Chef server 12.1 which means only the first word in the full name will be used, as the option is not parsed correctly

Restoring from a backup:

Set path to restore file with node[:chef_server_populator][:restore][:file]
The restore recipe is run if a restore file is set
The restore file can be remote or local

When enabling backups:

Include chef-server-populator::restore recipe
Set backup cron interval with node[:chef_server_populator][:schedule]
Optionally set a remote storage location with node[:chef_server_populator][:backup][:remote][:connection]
Backups include both a pg_dump of the entire chef database and a tarball of the Chef data directory

Public Key Format

The format of the public key specified with the json object needs to be a single line string with new lines represented with the \n character

You can use one of the below commands to convert your public key file into the correct string format (credit to the certificates cookbook for these)

cat <filename> | sed s/$/\\\\n/ | tr -d '\n'
-OR-
/usr/bin/env ruby -e 'p ARGF.read' <filename>
-OR-
perl -pe 's!(\x0d)?\x0a!\\n!g' <filename>

If you need to obtain the public key string for your private key first, then run the following on the .pem file containing the private key

openssl rsa -in <path_to_keyfile>.pem -pubout

Extras

Need to use the IP address of the node for a bit, or another name instead of having node[:fqdn]?

node[:chef_server_populator][:servername_override]

Keep chef server configured via chef client:

node[:chef_server_populator][:chef_server]

If the hash is non-empty, it will write the chef-server dna.json and trigger a reconfigure when ever the attributes are updated.

Known Issues

As mentioned above, user and client data bag items currently only support the first organization provided. Multi-org support is forthcoming.

Examples

Take a look in the examples directory for basic bootstrap templates that will build a new erchef server, using existing keys and client, and register itself, or restore an existing chef server from a backup.

Info

Repository: https://github.com/hw-cookbooks/chef-server-populator
IRC: Freenode @ #heavywater

chef-server-populator's People

Contributors

Stargazers

Watchers

Forkers

alex-chef alexdavis-bf-chef pmacdougall johnbellone cwjohnston yifangd yoshiwaan petecheslock the-smooth-operator adam-carbone aimtheory etna-alternance darkskyapp rudymccomb checkmypi isabella232

chef-server-populator's Issues

Backups path

Is it work with Chef 12 ?
https://github.com/hw-cookbooks/chef-server-populator/blob/master/recipes/backups.rb#L59

Should be:

cron 'Chef Server Backups' do
...
  path "$PATH:/opt/opscode/embedded/bin/"
end

Packagecloud.io no longer a viable source for chef-server

We have a metadata dependency on chef-server cookbook ~> 4.0, which results in our attempting to install Chef Server via Packagecloud.io repositories which are no longer available.

Version 5.0.1 of chef-server cookbook passes our unit tests but the newer repositories it provides access to do not seem to have the versions of Chef we specify in our default and old-chef suites.

Client recipe fails if client_key is missing from a data bag item.

I'm guessing this is a regression from the refactor of the recipe. Need to make sure we're only interacting with items that have a chef_server hash.

Invalid only_if check in client.rb recipe for add org validator key

Hi,

The only_if guard in the client.rb recipe on line 81 is a syntax error. It should be wrapped in {}.
Currently this generates an error as it tries to execute a bash command with the same name as the value of the pub_key variable.

Random password generation can fail

'password' => item.fetch('password', SecureRandom.urlsafe_base64(23))

The above line is used to generate a password for a user if none is specified in a data bag item.
As the string is random it is possible the first character of this password can be a hyphen.
Because of the use case of this password (passing as a command line parameter) the hyphen beginning the string can be interpreted as a switch parameter to the chef-server-ctl command and cause the chef run to fail.

Any leading hyphens should be stripped from the generated password string

Skipping organization creation

When using the cookbook, my organization was never created, in fact it was skipped.
The organization data bag has the parameter "enabled": "true.

Debugging the code I've found that changing true for "true" in the next line: https://github.com/hw-cookbooks/chef-server-populator/blob/master/recipes/client.rb#L33 was solving the issue.
I did not submit a pull requests because I am not sure if this is a bug.

chef-server-populator public key name not compatible with Chef Vault

Hi all! I'm trying to use this cookbook to set up a Chef server and then store secrets using Chef Vault. Unfortunately, Chef Vault only supports using the default public key (https://github.com/chef/chef-vault/blob/c98d4fbe4f7443f7341b278b436aa19ef4240e06/lib/chef-vault/chef_key.rb#L39) and this cookbook deletes it & creates a new one under the name of populator (https://github.com/hw-cookbooks/chef-server-populator/blob/develop/recipes/client.rb#L104-L115). I'll look into adding the default key back with my wrapper cookbook, but thought you'd want to be aware. Thanks!

Client recipe does not properly assign users to orgs

Another regression from the refactor.

Broken tests following merge of #69

I broke the build by merging #69. I selfishly wanted to see Nellie run the tests. Unfortunately I think the build failed before it could report that the following unit tests are broken:

Failures:                                                                                                                                                                                                   [34/1797]

  1) chef-server-populator::client a user is defined in the data bag the user is enabled creates the user
     Failure/Error:
       expect(chef_run).to run_execute("create user: #{test_user_name}").with(
         :command => "chef-server-ctl user-create #{test_user_name} #{test_user_item['chef_server']['full_name'].split(' ').first} #{test_user_item['chef_server']['full_name'].split(' ').last} #{test_user_item['c$
ef_server']['email']} #{test_user_item['chef_server']['password']} > /dev/null 2>&1"
       )

       expected "execute[create user: murph]" with action :run to be in Chef run. Other execute resources:

         execute[create org: endeavor]
         execute[add org validator key: endeavor]
         execute[remove org default validator key: endeavor]
         execute[create org: nasa]
         execute[add org validator key: nasa]
         execute[remove org default validator key: nasa]
         execute[create client: case]
         execute[set client key: case]
         execute[delete default client key: case]
         execute[create client: tars]
         execute[set client key: tars]
         execute[delete default client key: tars]

     # ./test/unit/client_spec.rb:122:in `block (4 levels) in <top (required)>'

  2) chef-server-populator::client a user is defined in the data bag the user is enabled the user has a client key specified creates the user's client key file
     Failure/Error:
       expect(chef_run).to create_file(test_user_pub_key_path).with(
         :content => test_user_item['chef_server']['client_key']
       )

       expected "file[/var/folders/y4/ccjrk8mn4c74ynp2lc7kpnm80000gn/T/d20160511-49588-p3cehy/murph.pub]" with action :create to be in Chef run. Other file resources:

         file[/var/folders/y4/ccjrk8mn4c74ynp2lc7kpnm80000gn/T/d20160511-49588-p3cehy/chef-server-core.firstrun]
         file[/var/folders/y4/ccjrk8mn4c74ynp2lc7kpnm80000gn/T/d20160511-49588-p3cehy/endeavor.pub]
         file[/var/folders/y4/ccjrk8mn4c74ynp2lc7kpnm80000gn/T/d20160511-49588-p3cehy/nasa.pub]
         file[/var/folders/y4/ccjrk8mn4c74ynp2lc7kpnm80000gn/T/d20160511-49588-p3cehy/case.pub]
         file[/var/folders/y4/ccjrk8mn4c74ynp2lc7kpnm80000gn/T/d20160511-49588-p3cehy/tars.pub]

     # ./test/unit/client_spec.rb:130:in `block (5 levels) in <top (required)>'

  3) chef-server-populator::client a user is defined in the data bag the user is enabled the user has a client key specified sets inserts the client key as the user's populator key
     Failure/Error:
       expect(chef_run).to run_execute("set user key: #{test_user_name}").with(
         :command => "chef-server-ctl add-user-key #{test_user_name} --public-key-path #{test_user_pub_key_path} --key-name populator"
       )

       expected "execute[set user key: murph]" with action :run to be in Chef run. Other execute resources:

         execute[create org: endeavor]
         execute[add org validator key: endeavor]
         execute[remove org default validator key: endeavor]
         execute[create org: nasa]
         execute[add org validator key: nasa]
         execute[remove org default validator key: nasa]
         execute[create client: case]
         execute[set client key: case]
         execute[delete default client key: case]
         execute[create client: tars]
         execute[set client key: tars]
         execute[delete default client key: tars]

  4) chef-server-populator::client a user is defined in the data bag the user is enabled the user has a client key specified deletes the user's default key
     Failure/Error: expect(chef_run).to run_execute("delete default user key: #{test_user_name}")

       expected "execute[delete default user key: murph]" with action :run to be in Chef run. Other execute resources:

         execute[create org: endeavor]
         execute[add org validator key: endeavor]
         execute[remove org default validator key: endeavor]
         execute[create org: nasa]
         execute[add org validator key: nasa]
         execute[remove org default validator key: nasa]
         execute[create client: case]
         execute[set client key: case]
         execute[delete default client key: case]
         execute[create client: tars]
         execute[set client key: tars]
         execute[delete default client key: tars]

     # ./test/unit/client_spec.rb:142:in `block (5 levels) in <top (required)>'

Finished in 59.01 seconds (files took 1.82 seconds to load)
62 examples, 4 failures

Failed examples:

rspec ./test/unit/client_spec.rb:121 # chef-server-populator::client a user is defined in the data bag the user is enabled creates the user
rspec ./test/unit/client_spec.rb:129 # chef-server-populator::client a user is defined in the data bag the user is enabled the user has a client key specified creates the user's client key file
rspec ./test/unit/client_spec.rb:135 # chef-server-populator::client a user is defined in the data bag the user is enabled the user has a client key specified sets inserts the client key as the user's populator key
rspec ./test/unit/client_spec.rb:141 # chef-server-populator::client a user is defined in the data bag the user is enabled the user has a client key specified deletes the user's default key

Are the tests or fixtures in need of updating with this change?

Update the admin.pem to pivotal.pem

I believe the current default uses

/etc/opscode/pivotal.pem for the signing-key

So shouldn't the following be updated or at least add a comment so that people know for chef-server 12
default[:chef_server_populator][:user] = 'admin'
default[:chef_server_populator][:pem] = '/etc/chef-server/admin.pem'

Version evaluating to 0.0, causing the wrong commands to run

node['chef-server'][:version].to_f is used in the recipes to determine whether or not it is running in a chef server >= 12.1. This determines which commands are run when creating users, clients, organizations. I have an issue where the --public-key is not being passed in the command so the public key is not being used.

I'm willing to do the work to fix the issue if you can point me in the direction as to where the node['chef-server'][:version] is being set!

logging from inside the recipe
Chef-version: 0.0
Add client key command 'chef-server-ctl add-client-key lmid acarbone-dev /var/chef/cache/acarbone-dev.pub --key-name populator'

Pin dependencies to route around recent chef-ingredient renaming.

Chef updated the dependencies for chef-server and renamed chef-server-ingredient. We should pin chef-server below 4.0 and chef-server-ingredient to 0.4.0 to maintain functionality prior to a refactor.

Chef provisioning support?

In lieu of jtimberman's and John Kiesner's talks at last year's Chef conferences, do you have any intention of turning this repo towards using the chef provisioning model such as is found in the the example repo?

Key not added for client/user in 12.1

Hi,

The command for adding a key to a user or client has changed in 12.1 from earlier versions. The --public-key-path flag is now required, instead of simply adding an extra parameter.

On Amazon linux I also found that the new chef-server v4.0.0 cookbook doesn't work, although it may work fines for other distros. There were quite a few changes from 3.1.1 to 4.0.0 so the chef-server cookbook version might need locking for now if it introduces breaking changes.

#42

I would really appreciate a quick merge for this as I discovered the problem while deploying our production Chef server >.<

Regards,
Yoshi

Skip user when they have no client key

In this example the client_key is empty:

"chef_server": {
   "client_key": "",
   "type": [ "user"],
   "orgs": {
     "myorg": {
       "enabled": true,
       "admin": true
     }
   }
}

Which causes an error from the chef-server-ctl command to fail the chef run.

Data bag integration with user cookbook

Hey guys,

I'm building a PoC chef server and attempting to deploy the whole thing from code using this cookbook, chef-server, etc.

It's not clear to me if there is an intended integration or interoperability between the data bags used in this cookbook for user/client registration (`node[:chef_server_populator][:databag]) and the user data bags from Chef's users?

I'd prefer not to create another data bag with duplicate users and have to maintain code in two places if I can help it. Is there any integration available/intended with this or other chef user management tools, at this time?

I'm assuming not. But it would be really handy if I could use the same data bag for each and have this cookbook only register clients, keys, etc, if a certain value existed in each user's file.

Example user entry in the format used by the Chef-maintained user cookbook:

{
  "id": "user_id",
  "ssh_keys": [
    "key_value"
    ],
  "groups": [ "developers" ],
  "uid": 607,
  "shell": "\/bin\/bash",
  "comment": "comment_valuel"
}

Which I'm thinking could become:

{
    "id": "user_id",
    "full_name": "User Name",
    "ssh_keys": [
        "key_value"
    ],
    "groups": [
        "developers"
    ],
    "uid": 607,
    "email": "[email protected]",
    "shell": "/bin/bash",
    "comment": "comment_value",
    "chef_server": {
        "client_key": "public key contents",
        "type": [
            "user"
        ],
        "orgs": {
            "organization": {
                "enabled": true,
                "admin": true
            }
        }
    }
}

User is created as admin if attribute is set to false in data bag

Hi,

The current logic check for making a user in the client recipe incorrectly creates a user as an admin even if the admin attribute is set to false.
This is because the current logic looks in a different place for the admin key than what is specified in the examples:

    item.merge('client' => data_bag_item(node[:chef_server_populator][:databag], bag_item)['id'],
      'pub_key' => item['client_key'],
      'enabled' => item['enabled'],
      'admin' => item.fetch('admin', true),
      'password' => item.fetch('password', SecureRandom.urlsafe_base64(23)),
      'orgs' => item.fetch('orgs', {}))
    ...
    execute "set user org: #{item['client']}" do
      command "chef-server-ctl org-user-add #{item['org']} #{item['client']} #{'--admin' if item['admin']}"
    end

In the data bag the admin flag is set under the org:

{
  "id": "<user>",
    "orgs": {
      "<org>": {
        "enabled": true,
        "admin": false  <---
      }
    }
  }
}

Either the examples need updating or the code does, but you take away the ability to have a user be an admin in one org but not another if you don't update the code.