chef / chef-vault Goto Github PK

I'm trying to understand exactly how I can use chef-vault in our environment.

Can you clarify what key is used to encrypt/decrypt on a client when I specify encrypt for all clients of type X please?

• Is it the chef client.pem? If not, what key is it, and where is it stored?
• What happens if I create a new client of type X? Do I need to re-encrypt the vault for my new client to gain access?

Many thanks, and apologies if this is documented somewhere I haven't found.

"OpenSSL::PKey::RSAError: padding check failed" error received if the private key used to try and decrypt the value is not the pair of the public key used to encrypt the value. This can be received if the client/admin pem is regenerated after doing the encryption with chef-vault and the vault is not updated.

This is a VALID error, but need to add a better exception message!

I'm going to submit a PR for this shortly.. the general idea is that I wanted to be able to print out the contents of the chef-vault databag in JSON format in order to make larger changes.

An example of how I see it working is this:

$ knife decrypt testing test --mode client -Fj
{
  "id": "test",
  "alpha": {
    "beta": "gamma"
  },
  "gamma": {
    "beta": "alpha"
  }
}

Leaving the VALUE off would print the contents of the databag in whatever format you want (specified with -F). Adding a VALUE back in would give you the current chef-vault abbreviated output.

I created a vault (passwords) and item (root) using the following command - as per the syntax specified in KNIFE_EXAMPLES.md

$ knife vault create passwords root '{"username": "root", "password": "mypassword"}' -S "chef_environment:DEV AND name:MYTESTNODE04_DEV" --mode client

Vault item is created successfully - but I experience following error when I try to update the same vault item later on

$ knife vault update passwords root '{"username": "root", "password": "mypassword"}' -S "chef_environment:DEV AND name:MYTESTNODE04_DEV" --mode client
ERROR: ChefVault::Exceptions::SecretDecryption: passwords/root is not encrypted with your public key. Contact an administrator of the vault item to encrypt for you!

I don't understand what is the point of saying that ADMINS is an optional thing - if I cannot update the vault item later it means specifying an ADMIN is mandatory. Is that right?

My problem is that I cannot specify a list of ADMIN users beforehand - as I cannot be sure who would be updating the vault item eventually in production environment. Production support team keep changing, so I need to be able to specify a group or a dynamic list of users.

1- Allow everyone to be able to update the vault item - is there a way to specify a wild card option to allow anyone to update the vault item? e.g. -A "%" or -A "*" or not specifying -A means all?
2- Allow a group ( not a list of users ) to be able to update the vault item - where members of the group can change but anyone who belongs to that group should be able to manage the vault item. Is that possible in current version of chef-vault?

Can someone plz throw some light on this ASAP? My main problem is that I cannot specify a fix list of ADMINs beforehand in an environment where team members keep changing.

Thanks

Kevin,

This is to fix the issue we identified at the last Chef meetup.

In chef-vault.rb, use data_bag and chef_config_file getters instead of instance vars per Practical Object-Oriented Design in Ruby (POODR) guidelines.

"Released" date of version "v2.2.0" is showing as "## v2.2.0 / 2013-01-21" which is not correct. Year should be 2014 not 2013.

Chef, not chef

See this Github blog post

Read these two articles:

1. Clarifying the Roles of the .gemspec and Gemfile
2. Gem Development Best Practices

:~$ knife --help
Usage: knife sub-command (options)
-s, --server-url URL Chef Server URL
-k, --key KEY API Client Key
--[no-]color Use colored output, defaults to enabled
-c, --config CONFIG The configuration file to use
--defaults Accept default values for all questions
-d, --disable-editing Do not open EDITOR, just accept the data as is
-e, --editor EDITOR Set the editor to use for interactive commands
-E, --environment ENVIRONMENT Set the Chef environment
-F, --format FORMAT Which format to use for output
-u, --user USER API Client Username
--print-after Show the data after a destructive operation
-V, --verbose More verbose output. Use twice for max verbosity
-v, --version Show chef version
-y, --yes Say yes to all prompts for confirmation
-h, --help Show this message

/home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-vault-1.2.3/lib/chef/knife/EncryptPassword.rb:23:in <class:EncryptPassword>': uninitialized constant EncryptPassword::ChefVault (NameError) from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-vault-1.2.3/lib/chef/knife/EncryptPassword.rb:18:in<top (required)>'
from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/knife/core/subcommand_loader.rb:37:in load' from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/knife/core/subcommand_loader.rb:37:inblock in load_commands'
from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/knife/core/subcommand_loader.rb:37:in each' from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/knife/core/subcommand_loader.rb:37:inload_commands'
from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/knife.rb:114:in load_commands' from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/knife.rb:134:inlist_commands'
from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/application/knife.rb:179:in print_help_and_exit' from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/application/knife.rb:146:invalidate_and_parse_options'
from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/lib/chef/application/knife.rb:121:in run' from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/gems/chef-10.24.0/bin/knife:25:in<top (required)>'
from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/bin/knife:19:in load' from /home/rhass/.rvm/gems/ruby-1.9.3-p194@chef/bin/knife:19:in

Hi,

When is the 2.1.0 scheduled for release in Rubygems?

Regards.

When modifying 1.x items instead of blowing up with a .empty? on nil error it should exit gracefully with a meaningful error

This is a pretty minor change.

The readme should say that knife commands are not support, but should say supported:

NOTE: chef-vault 1.0 knife commands are not support! Please use chef-vault 2.0 commands.

but should say

NOTE: chef-vault 1.0 knife commands are not supported! Please use chef-vault 2.0 commands.

Repro Steps:

1. Verify you don't have a certs or passwords directory under your chef-repo/data_bags dir.
2. $ knife encrypt cert -S "name:my_server.example.com" --cert my_cert.pem --name my_cert_pub_key --admins "admin1, admin2"

Result

INFO: Writing ./data_bags/certs/my_cert_pub_key_keys.json...
ERROR: Errno::ENOENT: No such file or directory - ./data_bags/certs/my_cert_pub_key_keys.json

Expect

Please create the certs (or passwords) directory for me, or at least give me a specific error message about the missing directory.

Hello,

I'm trying to try out chef-vault on a vagrant windows 2012 client using the chef solo provisioner. I'm running chef-vault in solo mode but I don't see a way to list the chef solo node since the search option looks on the chef server. Am I missing something or do I have to use a chef server?

Thanks,

David

knife encrypt cert and knife encrypt password leave in extra spaces when --admins string has spaces, e.g. --admins 'alice, bob, carol'

The usage text shows knife rotate secret [VAULT] [ITEM] --mode MODE", and it should be knife encrypt rotate keys [VAULT] [ITEM] --mode MODE.

If you run knife -h you can see that knife commands are usually in the form of knife NOUN VERB, such as knife environment list and knife data bag show.

The current chef-vault commands are a bit disjointed with most being under knife encrypt VAULTNAME ITEM. One is under knife decrypt VAULTNAME ITEM VALUE and then there is knife rotate keys which may not be valid at the moment.

I suggest we standardize all chef-vault commands as knife vault OPERATION VAULTNAME ...

e.g.:
knife vault encrypt vault1 item --json item.json --admin gmanfunky --mode client
knife vault decrypt vault1 id,foo,bar --mode client
knife vault rotate vault1 --mode client

Or consider revamping knife use-cases more thoroughly to continue the analogy of a data bag command overlay. We can get rid of encrypt+decrypt sub-commands and try to match the existing knife data bag create,delete,edit,from file, show. Note that chef-vault's knife plugins go beyond knife data bag parity to enable individual value modification.

In order to facilitate encrypting files without having to convert new lines to \n adding a file-content option to the knife commands to let the code deal with line conversion for ease of encrypting a file

Would add a key called "file-content" in the json hash

As a bit of metadata for later use, knife encrypt should store the search query, so that:

1. The user could decrypt the data bag item to find it, a la:

knife decrypt vault mything search_query

1. The user wouldn't have to remember what the search query was every time, and reuse that for updating nodes.

How do you run integration tests on recipes?

In chef-vault 2.1.0, it's possible to create an encrypted item with illegal characters. This makes it impossible to retrieve or delete that databag item, and it also breaks the Chef web interface.

Example:

$ knife encrypt create service_passwords 'Guggenheim.TradeDM-PROD'' '{"password":"supersecretpass"}' --search 'hostname:foobar' --mode client

Results in:

$ knife decrypt service_passwords 'Guggenheim.TradeDM-TST' 'password' --mode client
ERROR: Chef::Exceptions::InvalidDataBagItemID: Data Bag items must have an id matching /^[-[:alnum:]_]+$/, you gave: "Guggenheim.TradeDM-TST_keys"

$ knife data bag show service_passwords
Guggenheim.TradeDM-PROD
Guggenheim.TradeDM-PROD_keys

$ knife data bag delete service_passwords Guggenheim.TradeDM-PROD

Do you really want to delete Guggenheim.TradeDM-PROD? (Y/N) y
ERROR: Chef::Exceptions::InvalidDataBagItemID: Data Bag items must have an id matching /^[-[:alnum:]_]+$/, you gave:
"Guggenheim.TradeDM-PROD"

http://about.travis-ci.org/docs/user/languages/ruby/

@moserke Would it be possible to get a new gem released with the code from #71 #72 #73 and #74 included?

The current README on here is incorrect and could be confusing for users.

Thank you.

Hello,

I've recently started testing with chef-vault on my local windows 2012 virtual box client. After I add the chef_gem "chef-vault" and require "chef-vault" statements, chef solo runs fine the 1st time but after that I get the following error.

C:/opscode/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb:1637:in `raise_if_conflicts': Unable to activate mixlib-shellout-1.2.0-x86-mingw32, because windows-pr-1.2.1 conflicts with windows-pr (> 1.2.2), win32-process-0.6.5 conflicts with win32-process (> 0.7.0) (Gem::LoadError)'

Here is my gem list on the windows 2012 client
bigdecimal (1.1.0)
builder (3.2.2)
bundler (1.1.5)
chef (11.6.2 x86-mingw32)
chef-vault (2.0.2)
chef-zero (1.7.1, 1.6)
coderay (1.0.9)
diff-lcs (1.2.4, 1.1.3)
erubis (2.7.0)
ffi (1.3.1 x86-mingw32, 1.0.9 x86-mingw32)
hashie (2.0.5)
highline (1.6.19)
hpricot (0.8.6)
io-console (0.3)
ipaddress (0.8.0)
json (1.7.7, 1.5.5)
method_source (0.8.2)
mime-types (1.25)
minitest (2.5.1)
mixlib-authentication (1.3.0)
mixlib-cli (1.3.0)
mixlib-config (2.0.0, 1.1.2)
mixlib-log (1.6.0)
mixlib-shellout (1.2.0 x86-mingw32, 1.1.0 x86-mingw32)
moneta (0.6.0)
multi_json (1.8.1)
mustache (0.99.4)
net-ssh (2.7.0)
net-ssh-gateway (1.2.0)
net-ssh-multi (1.1)
ohai (6.18.0)
pry (0.9.12.2 i386-mingw32)
puma (1.6.3)
rack (1.5.2)
rake (10.1.0, 0.9.2.2)
rdiscount (2.1.6)
rdoc (3.12.2, 3.9.5)
rdp-ruby-wmi (0.3.1)
rest-client (1.6.7)
ronn (0.7.3)
rspec (2.12.0)
rspec-core (2.12.2)
rspec-expectations (2.12.1)
rspec-mocks (2.12.2)
rspec_junit_formatter (0.1.6)
sdoc (0.3.20)
simplecov (0.7.1)
simplecov-html (0.7.1)
slop (3.4.6)
systemu (2.5.2, 2.2.0)
test-unit (2.5.5)
win32-api (1.4.8 x86-mingw32)
win32-dir (0.4.5, 0.3.7)
win32-event (0.6.1, 0.5.2)
win32-ipc (0.6.1)
win32-mmap (0.4.0)
win32-mutex (0.4.1, 0.3.1)
win32-process (0.7.3, 0.6.5)
win32-service (0.8.2, 0.7.2 x86-mingw32)
win32console (1.3.2 x86-mingw32)
windows-api (0.4.2, 0.4.0)
windows-pr (1.2.2, 1.2.1)
yajl-ruby (1.1.0 x86-mingw32)
yard (0.8.7.2)

This will make Travis CI integration easier.

I can update an existing vault item from json that contains multiple values, e.g.: zs-api.json:

{
  "id": "zs-api",
  "aws_access_key": "TRYNUMBERTWO",
  "aws_secret_key": "herewegoonceagainmyfriend"
}

then

    knife encrypt update pdbtest zs-api --mode client --json zs-api.json

It seems I also should be able to emit json for bulk edits and re-uploading, e.g: the following should work:

    knife decrypt pdbtest zs-api --mode client -F json > new.json

so I can bulk edit and update the vault more consistently.

I was trying to explain the knife vault rotate keys command the other day, and was having troubles fully understanding its purpose.

This command does not change who can access they vault, does it?

Is it for the case of client keys that have been regenerated on the chef server?

Thanks for any clarification.

Currently vault creation is successful even If you don't provide --ADMINS option with 'knife vault create' but later on 'knife vault update' fails with following error:

"ERROR: ChefVault::Exceptions::SecretDecryption: DATA_BAG/ITEM is not encrypted with your public key. Contact an administrator of the vault item to encrypt for you!"

--ADMINS must be as mandatory option and 'knife vault create' must fail if at least 1 admin is not provided.

I try to create a new vault but run into errors. I am running knife and chef-vault through bundler if that helps.

/Users/mhenrixon/.rvm/gems/ruby-2.1.1/gems/chef-vault-2.2.1/lib/chef/knife/vault_create.rb:50:in `run': undefined method `join' for nil:NilClass (NoMethodError)
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/gems/chef-11.10.4/lib/chef/knife.rb:491:in `run_with_pretty_exceptions'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/gems/chef-11.10.4/lib/chef/knife.rb:174:in `run'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/gems/chef-11.10.4/lib/chef/application/knife.rb:135:in `run'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/gems/chef-11.10.4/bin/knife:25:in `<top (required)>'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/bin/knife:23:in `load'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/bin/knife:23:in `<main>'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/bin/ruby_executable_hooks:15:in `eval'
    from /Users/mhenrixon/.rvm/gems/ruby-2.1.1/bin/ruby_executable_hooks:15:in `<main>'

This is my knife.rb file

# knife.rb
log_level                :info
log_location             STDOUT
node_name                'mhenrixon'
client_key               '/Users/mhenrixon/code/rushplay/casino-saga-chef/.chef/mhenrixon.pem'
validation_client_name   'mhenrixon-validator'
validation_key           '/Users/mhenrixon/code/rushplay/casino-saga-chef/.chef/mhenrixon-validator.pem'
chef_server_url          'https://chef.casinosaga:443'
syntax_check_cache_path  '/Users/mhenrixon/code/rushplay/casino-saga-chef/.chef/syntax_check_cache'

knife[:vault_mode] = 'client'

I have no issues encrypting the data bag when I just specify the search string -S "role:base". However, when I attempt to add -A or --admins the command returns the JSON::ParseError.

command:

 ubuntu@jaryd:~/ddg-chef$ knife encrypt create priv ssh_keys -J key.json -S "role:base" --admins "jaryd" -M "client"

chef (11.6.0, 11.4.0)
server version 10.18.2

I am unable to specify client admins in the --admins flag. chef-vault 1.x would search for the specified admins in users then clients.

https://github.com/Nordstrom/chef-vault/blob/master/lib/chef-vault/item.rb#L134-L136

From http://www.ruby-doc.org/stdlib-1.9.3/libdoc/openssl/rdoc/OpenSSL/PKey/RSA.html,

Generates or loads an RSA keypair. If an integer key_size is given it represents the desired key size. Keys less than 1024 bits should be considered insecure.

What about adding gpg support for public/private pairs using https://github.com/ueno/ruby-gpgme? This would potentially address #58.

If I were to add GPG support and submit the patches as a PR, would you consider it for inclusion, or is GPG support something you would consider out-of-scope entirely?

I am trying to test/verify that everything is production ready but the below code

chef_vault_item 'passwords', 'postgres_master'

generates the following error:

[2014-03-09T21:33:59+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: http://127.0.0.1:8889/data/passwords/postgres_master_keys

       ================================================================================
       Recipe Compile Error in /tmp/kitchen/cache/cookbooks/db_server/recipes/master.rb
       ================================================================================


       ChefVault::Exceptions::KeysNotFound
       -----------------------------------
       passwords/postgres_master_keys could not be found


       Cookbook Trace:
       ---------------
  /tmp/kitchen/cache/cookbooks/chef-vault/libraries/chef_vault_item.rb:43:in `chef_vault_item'
         /tmp/kitchen/cache/cookbooks/db_server/recipes/master.rb:4:in `from_file'

Even though I have the data_bags folder with the in my book correct name like in the image below.

Could anyone shed any light on what I am doing wrong?

Kevin,

I'm adding RSpec tests.

Doug

Add logging subsystem so users can control logging level and have more consistent logging.