test-kitchen / busser Goto Github PK

Kitchen Busser - Runs tests for projects in test-kitchen

License: Other

Ruby 78.05% Gherkin 10.99% HTML 10.96%

managed-by-terraform hacktoberfest ruby-gem test-kitchen testing ruby

busser's Introduction

Test Kitchen


Website	https://kitchen.ci/
Source Code	https://kitchen.ci/docs/getting-started/introduction/
Slack	#test-kitchen channel on Chef Community Slack

Test Kitchen is an integration tool for developing and testing infrastructure code and software on isolated target platforms.

Getting Started Guide

To learn how to install and setup Test Kitchen for developing infrastructure code, check out the Getting Started Guide.

If you want to get going super fast, then try the Quick Start next...

Quick Start

Test Kitchen is a RubyGem and can be installed with:

gem install test-kitchen

If you use Bundler, you can add gem "test-kitchen" to your Gemfile and make sure to run bundle install.

Next add support to your library, Chef cookbook, or empty project with kitchen init:

kitchen init

A kitchen.yml file will be created in your project base directory. This file describes your testing configuration; what you want to test and on which target platforms. Each of these suite and platform combinations are called instances. By default your instances will be converged with Chef Solo and run in Vagrant virtual machines.

Get a listing of your instances with:

kitchen list

Run Chef Infra Client on an instance, in this case default-ubuntu-2004, with:

kitchen converge default-ubuntu-2004

Destroy all instances with:

kitchen destroy

You can clone a Chef Infra cookbook project that contains Test Kitchen support and run through all the instances in serial by running:

kitchen test

Usage

There is help included with the kitchen help subcommand which will list all subcommands and their usage:

kitchen help test

More verbose logging for test-kitchen can be specified when running test-kitchen from the command line using:

kitchen test -l debug

For the provisioner (e.g. chef-solo or chef-infra) add a log_level item to the provisioner section of the .kitchen.yml For more information on setting log_level see the configuration documentation.

Documentation

Documentation is being added on the Test Kitchen website. Please read and contribute to improve them!

Versioning

Test Kitchen aims to adhere to Semantic Versioning 2.0.0.

Community and Ecosystem

If you would like to see a few of the plugins or ecosystem helpers, please look at ECOSYSTEM.md.

Development

Source hosted at GitHub
Report issues/questions/feature requests on GitHub Issues

Pull requests are very welcome! Make sure your patches are well tested. Ideally create a topic branch for every separate change you make. For example:

Fork the repo
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Added some feature')
Push to the branch (git push origin my-new-feature)
Create a new Pull Request

Authors

Created by Fletcher Nichol ([email protected]).

Maintained by Chef Software and a growing community of contributors.

License

Apache License, Version 2.0 (see LICENSE)

busser's People

Contributors

Stargazers

Watchers

Forkers

arangamani higanworks obazoud alex-chef scotthain mbrukman jordansissel fwiesel fatsoma allenbhuiyan neillturner transportapi languageplayground redox isabella232 iq-scm

busser's Issues

Busser/Thor 0.19 incompatability?

Got this when I just tried to use bats for the first time; apparently Thor 19 only got released in the last 20 minutes or so, so perhaps I'm just really unlucky.

I had a quick look at busser and thor to try and workout where the Pathname originated, but it's a lot of code to step through as both libs use pathname in some capacity.

Decided to make an issue now incase someone with more knowledge can see a trivial fix.

-----> Verifying <supervisor-program-enable-ubuntu-1204>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
/tmp/busser/gems/gems/thor-0.19.0/lib/thor/shell/basic.rb:81:in `say': undefined method `end_with?' for #<Pathname:/tmp/busser/suites> (NoMethodError)
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/shell.rb:63:in `say'
        from /tmp/busser/gems/gems/busser-0.6.0/lib/busser/command/suite_path.rb:34:in `path'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `block in invoke_all'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `each'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `map'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `invoke_all'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/group.rb:232:in `dispatch'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:115:in `invoke'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor.rb:40:in `block in register'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor.rb:359:in `dispatch'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:115:in `invoke'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor.rb:235:in `block in subcommand'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor.rb:359:in `dispatch'
        from /tmp/busser/gems/gems/thor-0.19.0/lib/thor/base.rb:440:in `start'
        from /tmp/busser/gems/gems/busser-0.6.0/bin/busser:8:in `<top (required)>'
        from /tmp/busser/gems/bin/busser:23:in `load'
        from /tmp/busser/gems/bin/busser:23:in `<main>'
Uploading /bats/check_program.bats (mode=0664)

Unable to resolve dependencies: busser requires thor (<= 0.19.0)

Installing busser fails

Kitchen::ActionFailed: SSH exited (1) for command: [env http_proxy=http://172.30.73.53:8123 https_proxy=http://172.30.73.53:8123 sh -c '
BUSSER_ROOT="/tmp/busser" GEM_HOME="/tmp/busser/gems" GEM_PATH="/tmp/busser/gems" GEM_CACHE="/tmp/busser/gems/cache"
export BUSSER_ROOT GEM_HOME GEM_PATH GEM_CACHE
gem_bindir=/opt/chef/embedded/bin/ruby -rrubygems -e "puts Gem.bindir"

if ! sudo -E /opt/chef/embedded/bin/gem list busser -i >/dev/null; then
sudo -E /opt/chef/embedded/bin/gem install busser --no-rdoc --no-ri
fi

How to configure busser plugins

is it possible to pass parameters from the kitchen.yml to the busser plugins so they can be configured.

Return helpful message and fail when a plugin (gem install) is not successful

This includes the following cases:

when a gem does not exist
when a gem is missing dependencies, like native extensions that cannot be installed
when a RubyGems issue occurs and a legitimate gem cannot be installed

References:

readlink command missing '-f' in binstub?

I'm having problems with the generated binstub after running busser setup (part of kitchen verify). In one of my Wercker box repositories, I install chef-dk and some plugins, followed by running serverspec to validate correct installation by checking the command line versions.

I install busser on a system ruby since it doesn't support Chef-DK yet. Since upgrading my base OS from Ubuntu 12.04 to 14.04, the binstub fails, because of a different setup of the ruby softlinks:

Ubuntu 12.04:

ubuntu@ip-10-209-166-44:~$ ls -l /usr/bin/ruby
lrwxrwxrwx 1 root root 22 Sep 14 16:26 /usr/bin/ruby -> /etc/alternatives/ruby
ubuntu@ip-10-209-166-44:~$ ls -l /etc/alternatives/ruby
lrwxrwxrwx 1 root root 18 Sep 14 16:26 /etc/alternatives/ruby -> /usr/bin/ruby1.9.1
ubuntu@ip-10-209-166-44:~$ SOURCE=/usr/bin/ruby
ubuntu@ip-10-209-166-44:~$ while [ -h "$SOURCE" ] ; do SOURCE="`readlink "$SOURCE"`"; done
ubuntu@ip-10-209-166-44:~$ dirname $SOURCE
/usr/bin
ubuntu@ip-10-209-166-44:~$ echo $SOURCE
/usr/bin/ruby1.9.1

Ubuntu 14.04:

ubuntu@ip-10-209-52-131:~$ ls -l /usr/bin/ruby
lrwxrwxrwx 1 root root 9 Feb 15  2014 /usr/bin/ruby -> ruby1.9.1
ubuntu@ip-10-209-52-131:~$ SOURCE="/usr/bin/ruby"
ubuntu@ip-10-209-52-131:~$ while [ -h "$SOURCE" ] ; do SOURCE="`readlink "$SOURCE"`"; done
ubuntu@ip-10-209-52-131:~$ dirname $SOURCE
.
ubuntu@ip-10-209-52-131:~$ echo $SOURCE
ruby1.9.1

Adding the -f option to the readlink command solves my problem, but I'm not sure if this option is supported on all the different Unix flavors.

Comments?

Busser needs to be Proxy aware

note by @cheeseplus - copying from test-kitchen/test-kitchen#821

So the client I'm currently working for has an obnoxious proxy that requires creds etc.

Environment

Mac OS X Yosemite
ChefDK 0.7.0
- Kitchen 1.4.2
- kitchen-vagrant 0.18.0
VirtualBox 5.0.0
Vagrant 1.7.4
Charles Proxy 3.10.2
CentOS 6.6/7.1 Guest

Kitchen Config (.kitchen.yml)

provisioner:
  name: chef_zero
  require_chef_omnibus: 12.3.0
  http_proxy: http://10.0.2.2:8888
  https_proxy: http://10.0.2.2:8888
  no_proxy: 'localhost,127.0.0.1,10.0.2.,33.33.33.,.customerdomain.com'

Proxy Settings Not Included in Setup/Verfy Stage

Adding the above to the .kitchen.yml fixes everything up to the phase where it needs to download Gems for ServerSpec. It seems the proxy is not set during the _setup_ stage.

As a test I added the following to the vagrant user _~/.bashrc_ and it _Fixes_ the problem.

export http_proxy=http://10.0.2.2:8888
export https_proxy=https://10.0.2.2:8888
export no_proxy=localhost,127.0.0.1,10.0.2.,33.33.33.,.customerdomain.com

Charles Proxy is basically a application that runs your own local proxy that you point all you Dev tools to and it saves your creds securely and in one spot (Easier for when changing password). So from a Vagrant guest to get to the hosted system it's IP 10.0.2.2 blah, and whatever port you tell Charles Proxy to listen on 8888 blah.

Bam, that fixes the issue. Which proves that the proxy is not getting set by the Kitchen code during the _setup_ stage.

I have not had time to dig into the Kitchen code to find exactly where the problem is... If I have time I will.

Errror

-----> Starting Kitchen (v1.4.2)
-----> Setting up <base-rh-centos-66>...
       Finished setting up <base-rh-centos-66> (0m0.00s).
-----> Verifying <base-rh-centos-66>...
       Preparing files for transfer
-----> Installing Busser (busser)
       ERROR:  Could not find a valid gem 'busser' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - no such name (https://rubygems.org/specs.4.8.gz)
       sh: line 23: /tmp/verifier/gems/bin/busser: No such file or directory
       Installing Busser plugins: busser-serverspec
       sudo: /tmp/verifier/bin/busser: command not found
>>>>>> Verify failed on instance <base-rh-centos-66>.
>>>>>> Please see .kitchen/logs/base-rh-centos-66.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/opt/chef/embedded/bin/ruby"
gem="/opt/chef/embedded/bin/gem"
version="busser"
gem_install_args="busser --no-rdoc --no-ri"
busser="sudo -E /tmp/verifier/bin/busser"
plugins="busser-serverspec"

$gem list busser -i 2>&1 >/dev/null
if test $? -ne 0; then
  echo "-----> Installing Busser ($version)"
  $gem install $gem_install_args
else
  echo "-----> Busser installation detected ($version)"
fi

if test ! -f "$BUSSER_ROOT/bin/busser"; then
  gem_bindir=`$ruby -rrubygems -e "puts Gem.bindir"`
  $gem_bindir/busser setup
fi

echo "       Installing Busser plugins: $plugins"
$busser plugin install $plugins
']
>>>>>> ----------------------
zlib(finalizer): the stream was freed prematurely.

Thanks in advance for any help!

-=Levon

Busser should be able to install gems from alternate sources

If I don't want to point to (or can't get to) RubyGems.org, I should be able to point to either local file system paths, or an internal gem server.

plugin_usage broken link at README.md.erb

from test-kitchen/busser-serverspec#10

https://github.com/test-kitchen/busser/blob/master/templates/plugin/README.md.erb

this link no longer exists. but i do not find correct plugin_usage page.

/usr/bin/env sh shouldn't assume bourne shell

While /usr/bin/env sh may resolve to Bourne shell or a Bourne-like shell on many platforms, it doesn't on Solaris/illumos systems, such as OmniOS.

$ ls -l /usr/bin/sh
lrwxrwxrwx   1 root     root           9 May 16  2013 /usr/bin/sh -> i86/ksh93

This causes the following error:

-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       /tmp/busser/bin/busser: line 15: : cannot execute [Is a directory]
       Usage: dirname [-frx] string
       /tmp/busser/bin/busser[15]: /opt/chef/embedded/bin/ruby` && pwd: not found [No such file or directory]
       /tmp/busser/bin/busser[33]: exec: /ruby: not found
>>>>>> Setup failed on instance <client-omnios-r151006c>.
>>>>>> Please see .kitchen/logs/client-omnios-r151006c.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (127) for command: [sh -c 'BUSSER_ROOT="/tmp/busser" GEM_HOME="/tmp/busser/gems" GEM_PATH="/tmp/busser/gems" GEM_CACHE="/tmp/busser/gems/cache" ; export BUSSER_ROOT GEM_HOME GEM_PATH GEM_CACHE; if ! sudo -E /opt/chef/embedded/bin/gem list busser -i >/dev/null; then sudo -E /opt/chef/embedded/bin/gem install busser --no-rdoc --no-ri; fi; gem_bindir=`/opt/chef/embedded/bin/ruby -rrubygems -e "puts Gem.bindir"`; sudo -E ${gem_bindir}/busser setup; sudo -E /tmp/busser/bin/busser plugin install busser-bats']
>>>>>> ----------------------

"busser plugin install" doesn't respect system gemrc

I'm not sure if it should respect that or it should be in another entry in .kitchen.yml. But here's the difference of the two:

gem install - https://github.com/rubygems/rubygems/blob/master/lib/rubygems/commands/install_command.rb#L22-L28
busser plugin install - https://github.com/test-kitchen/busser/blob/master/lib/busser/rubygems.rb#L76-L78

This difference might explain what I mentioned earlier in #18

How to install additional gems for busser execution?

Maybe it's specified somewhere in the documentation, or it's a test-kitchen issue and not a busser one, but I'm not able to install additional gems that might be needed for busser execution, see for instance test-kitchen/busser-cucumber#2

Could you help?

Thanks.

Busser/Thor Error.

ERROR:  Error installing busser: invalid gem format for /tmp/busser/gems/cache/thor-0.19.0.gem       
sudo: /tmp/busser/gems/bin/busser: command not found       
sudo: /tmp/busser/bin/busser: command not found

I think this to be related to #7, but I am unable to currently test. Could it be that busser needs to be bumped to use thor 0.19.1?

Testing LWRPs, not recipes

Fletcher,
thank you for test-kitchen.

Is there any way to test LWRPs instead of recipes?
I looked at the busser-minitest source code, and it's not supported there.
My "dream functionality" would be to to test a single LWRP multiple times with different attributes.

I understand I could create a test recipe for each LWRP invocation, but that does not seem like a good solution.

New Serverspec Gem Breaks Older Chef Versions

Serverspec have released an updated gem which relies on a new version of net-ssh. The new version of net-ssh requires Ruby >= 2.0.

Older versions of Chef use Ruby < 2.0, so the installation fails.

I wasn't sure whether to report this here or against serverspec. It looks like the serverspec change was introduced pretty recently, and may well be this one mizzy/serverspec@35989d7. I'm only asserting that because serverspec has that commit tagged for 2.25, and that version of the gem was dropped into rubygems today. Unfortunately, I'm having difficulty figuring out whether the check there for ruby version is actually working as expected i.e. whether net-ssh ~> 2.7 is being installed or not.

My rationale for reporting this here is that it's the test-kitchen project that's breaking, so you're probably most invested right now. If there were a way to specify what versions of gems to install into the virtual machines, that'd provide a workaround for this for now.

kitchen verify default
-----> Starting Kitchen (v1.4.2)
-----> Verifying <default-oel-6-4>...
   Preparing files for transfer
-----> Busser installation detected (busser)
   Installing Busser plugins: busser-serverspec
   Plugins: busser-serverspec
   Plugin serverspec already installed
   Removing /tmp/verifier/suites/serverspec
   Transferring files to <default-oel-6-4>
-----> Running serverspec test suite
-----> Installing Serverspec..
   /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/installer.rb:388:in `ensure_required_ruby_version_met': net-ssh requires Ruby version >= 2.0. (Gem::InstallError)
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/installer.rb:156:in `install'
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/dependency_installer.rb:297:in `block in install'
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/dependency_installer.rb:270:in `each'
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/dependency_installer.rb:270:in `each_with_index'
    from /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/dependency_installer.rb:270:in `install'
    from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/rubygems.rb:44:in `install_gem'
    from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/helpers.rb:57:in `install_gem'
    from /tmp/verifier/gems/gems/busser-serverspec-0.5.7/lib/busser/runner_plugin/serverspec.rb:60:in `install_serverspec'
    from /tmp/verifier/gems/gems/busser-serverspec-0.5.7/lib/busser/runner_plugin/serverspec.rb:33:in `test'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `block in invoke_all'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `each'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `map'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `invoke_all'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/group.rb:232:in `dispatch'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:115:in `invoke'
    from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/test.rb:43:in `block in perform'
    from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/test.rb:35:in `each'
    from /tmp/verifier/gems/gems/busser-0.7.1/lib/busser/command/test.rb:35:in `perform'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `block in invoke_all'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `each'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `map'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:133:in `invoke_all'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/group.rb:232:in `dispatch'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:115:in `invoke'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor.rb:40:in `block in register'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/command.rb:27:in `run'
   from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/invocation.rb:126:in `invoke_command'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor.rb:359:in `dispatch'
    from /tmp/verifier/gems/gems/thor-0.19.0/lib/thor/base.rb:440:in `start'
    from /tmp/verifier/gems/gems/busser-0.7.1/bin/busser:8:in `<top (required)>'
    from /tmp/verifier/gems/bin/busser:23:in `load'
    from /tmp/verifier/gems/bin/busser:23:in `<main>'

Verify failed on instance .

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

bundler

Gemfile

Check this box to trigger a request for Renovate to run again on this repository

Permission denied - stub_specification

I'm not sure which repo to report this to. Here is what I'm doing:

RHEL 6.7
EC2 Host

Kitchen create (creates the Ec2 Host)
Kitchen verify ( 1st time - works)
Kitchen verify ( 2nd time - ERROR Permission denied)

Here is the error:

D      [SSH] opening connection to [email protected] <{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60,
 :timeout=>15, :keys_only=>true, :keys=>["/PEM_FILE.pem"], :auth_methods=>["publickey"]}>
       /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/stub_specification.rb:75:in `initialize': Permission denied @ rb_sysopen - /tmp/verifier/gems/specifications/rspec-core-3.4.4.gemspec (Errno::EACCES)
        from /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/stub_specification.rb:75:in `open'
        from /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/stub_specification.rb:75:in `data'
        from /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/stub_specification.rb:178:in `valid?'
        from /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:731:in `block in each_stub'
        from /opt/chef/embedded/lib/ruby/site_ruby/2.1.0/rubygems/specification.rb:723:in `block (2 levels) in each_gemspec'
-----> Installing Busser (busser)

The permissions of /tmp/verifier/gems/specifications/rspec-core-3.4.4.gemspec

-rw-r----- 1 root     root     4793 Jun 11 23:57 rspec-core-3.4.4.gemspec

To resolve this issue I had to give everyone read access to the gemspec's.

$ chmod o+r /tmp/verifier/gems/specifications/*

Just not sure how these files were created in the first place. Anyone?

Busser only uses /tmp/verfier/gems as it's local source

I'd like to be able to reduce my spin up the verify times of kitchen suites.
Baking the busser/busser-serverspec etc gems into the chef omnibus installer would be a nice option as it will persist between destroys.
I there a way to override the BUSSER_GEM, GEM_PATH, GEM_HOME, GEM_CAHE variables?

Assuming that:

chef-client is installed into the image
busser and busser-serverspec gems are installed

kitchen would find those gems. For a slow connection, or a non-existent connection to the internet (corp firewall, no-internet etc) this would be super useful.

If this doesn't belong here, but in the kitchen-ci repo I'm happy to move it.

Busser issues when /tmp is mounted with noexec

Copied from: test-kitchen/test-kitchen#749

I´ve got the problem that busser installation fails due permission error:

      Preparing files for transfer
-----> Installing Busser (busser)
Fetching: thor-0.19.0.gem (100%)
       Successfully installed thor-0.19.0
Fetching: busser-0.7.1.gem (100%)
       Successfully installed busser-0.7.1
       2 gems installed
       sh: 24: /tmp/verifier/gems/bin/busser: Permission denied
       Installing Busser plugins: busser-serverspec
       sudo: /tmp/verifier/bin/busser: command not found
>>>>>> Verify failed on instance <xyz-x86-64>.
>>>>>> Please see .kitchen/logs/default-xyz-x86-64.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/opt/chef/embedded/bin/ruby"
gem="/opt/chef/embedded/bin/gem"
version="busser"
gem_install_args="busser --no-rdoc --no-ri"
busser="sudo -E /tmp/verifier/bin/busser"
plugins="busser-serverspec"

$gem list busser -i 2>&1 >/dev/null
if test $? -ne 0; then
  echo "-----> Installing Busser ($version)"
  $gem install $gem_install_args
else
  echo "-----> Busser installation detected ($version)"
fi

if test ! -f "$BUSSER_ROOT/bin/busser"; then
  gem_bindir=`$ruby -rrubygems -e "puts Gem.bindir"`
  $gem_bindir/busser setup
fi

echo "       Installing Busser plugins: $plugins"
$busser plugin install $plugins
']
>>>>>> ----------------------

sudo: /tmp/verifier/bin/busser: command not found
busser bin is installed in /tmp/verifier/gems/bin/busser

A link of the dirs doesn´t solve the problem.

What can I do?

test group(plugin) priority

It will be great if I can specify plugins execution order.

For example test/integration/default/00_serverspec and test/integration/default/01_bats. Because I want that first of all my chef cookbook will be tested on serverspec and if all ok, then I want to do more complex tests with bats.

Thanks.

Binstubs in 'verifier' do not point at a valid ruby interpreter on Windows

Originally reported at test-kitchen/test-kitchen#616 but re-reporting here as the symptoms of that bug are now fixed, but the cause remains.

On Windows, the batchfile binstubs generated by busser in %TEMP%\verifier\gems\bin\ do not point at a valid ruby. The ruby.exe that they try to call does not exist on the machine.

In my case, bundle.bat looks like this:

C:\Users\vagrant>more C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\bundle.bat                                                                                                                                               
@ECHO OFF                                                                                                                                                                                                                            
IF NOT "%~f0" == "~f0" GOTO :WinNT                                                                                                                                                                                                   
@"C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\ruby.exe" "C:/Users/vagrant/AppData/Local/Temp/verifier/gems/bin/bundle" %1 %2 %3 %4 %5 %6 %7 %8 %9                                                                          
GOTO :EOF                                                                                                                                                                                                                            
:WinNT                                                                                                                                                                                                                               
@"%~dp0ruby.exe" "%~dpn0" %*

Where bundle in the same directory, looks like this:

C:\Users\vagrant>more C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\bundle                                                                                                                                                   
#!C:/opscode/chef/embedded/bin/ruby.exe                                                                                                                                                                                              
#                                                                                                                                                                                                                                    
# This file was generated by RubyGems.                                                                                                                                                                                               
#                                                                    
....

The latter is correct.

This leads to errors when trying to run binaries, even with the correct environment. An example would be running serverspec tests with a Gemfile present (which runs 'bundle'):

-> bundle exec kitchen verify default-opentable-win-2012r2-standard-amd64-nocm
-----> Starting Kitchen (v1.4.0.rc.1)
-----> Verifying <default-opentable-win-2012r2-standard-amd64-nocm>...
       Preparing files for transfer
-----> Busser installation detected (busser)
       Installing Busser plugins: busser-serverspec
       Plugin serverspec already installed
       Removing C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec
       Transferring files to <default-opentable-win-2012r2-standard-amd64-nocm>
-----> Running serverspec test suite
-----> Bundle Installing..
         run  bundle install --gemfile C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/Gemfile --local || bundle install --gemfile C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/Gemfile from "."
       C:/opscode/chef/embedded/bin/ruby.exe -IC:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec -I'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-support-3.2.2/lib';'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib' 'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/exe/rspec' --pattern 'C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/**/*_spec.rb' --color --format documentation --default-path C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec
$$$$$$ C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/serverspec.rb:49:in `run_bundle_install': Use RbConfig instead of obsolete and deprecated Config.
$$$$$$ C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/serverspec.rb:49:in `run_bundle_install': Use RbConfig instead of obsolete and deprecated Config.
$$$$$$ '"C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\ruby.exe"' is not recognized as an internal or external command,
$$$$$$ operable program or batch file.
$$$$$$ '"C:\Users\vagrant\AppData\Local\Temp\verifier\gems\bin\ruby.exe"' is not recognized as an internal or external command,
$$$$$$ operable program or batch file.
$$$$$$ C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/spec_helper.rb:17:in `<top (required)>': uninitialized constant Rspec (NameError)
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/default_spec.rb:1:in `require_relative'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/default_spec.rb:1:in `<top (required)>'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1226:in `load'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1226:in `block in load_spec_files'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1224:in `each'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1224:in `load_spec_files'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:97:in `setup'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:85:in `run'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:70:in `run'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:38:in `invoke'
$$$$$$  from C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/exe/rspec:4:in `<main>'
$$$$$$ C:/opscode/chef/embedded/bin/ruby.exe -IC:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec -I'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-support-3.2.2/lib';'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/lib' 'C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/rspec-core-3.2.3/exe/rspec' --pattern 'C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec/**/*_spec.rb' --color --format documentation --default-path C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec failed
$$$$$$ !!!!!! Ruby Script [C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-serverspec-0.5.5/lib/busser/runner_plugin/../serverspec/runner.rb C:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec] exit code was 1
>>>>>> Verify failed on instance <default-opentable-win-2012r2-standard-amd64-nocm>.
>>>>>> Please see .kitchen/logs/default-opentable-win-2012r2-standard-amd64-nocm.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: WinRM exited (1) for command: [
$env:BUSSER_ROOT = "$env:TEMP\verifier"
$env:GEM_HOME = "$env:TEMP\verifier\gems"
$env:GEM_PATH = "$env:TEMP\verifier\gems"
$env:GEM_CACHE = "$env:TEMP\verifier\gems\cache"

& $env:TEMP\verifier\bin\busser.bat test
]
>>>>>> ----------------------

Note, this error no longer occurs as it was worked around by test-kitchen/busser-serverspec#27 which calls bundler explicitly via /path/to/ruby.exe /path/to/bundle/script instead.

kitchen test with serverspec fails: `run_ruby_script!': undefined method `success?' for nil:NilClass (NoMethodError)

As per suggestion of @cl-lab-k I forwarded the original issue (test-kitchen/busser-serverspec#22) here.

In my scenario kitchen test reproducably fails on the first run, but succeeds on subsequent runs. See description in the original issue for more details.

Any ideas?

Support for sharing code between test suites

It is common for many different test kitchen suites to require verifying the same or similar behaviors. AFAIK, there isn't currently a way to share code between suites, because serverspec uploads the suite folder and nothing outside/above it.

It would be very useful and reduce duplication to be able to share a spec_helper between suites per test framework/plugin.

suite_cleanup does not work on windows

It seems suite_path will return the path using \ on windows, but this does not work for globbing (see: http://ruby-doc.org/core-2.1.1/Dir.html#method-c-glob which notes "Note that this means you cannot use backslash on windows as part of a glob, i.e. Dir["c:\foo_"] will not work, use Dir["c:/foo_"] instead.").

Example:

From: C:/Users/vagrant/AppData/Local/Temp/verifier/gems/gems/busser-0.7.0/lib/busser/command/suite_cleanup.rb @ li
Busser::Command::SuiteCleanup#cleanup:

    31:       def cleanup
    32:         if suite_path.directory?
    33:
    34:                 require 'pry'
 => 35:                 binding.pry
    36:
    37:           Pathname.glob(suite_path + "*").each do |dir|
    38:             info "Removing #{dir}"
    39:             dir.rmtree
    40:           end
    41:         else
    42:           info "Suite path directory #{suite_path} does not exist, skipping."
    43:         end
    44:       end
[1] pry(#<Busser::Command::SuiteCleanup>)> suite_pathpath
=> #<Pathname:C:\Users\vagrant\AppData\Local\Temp\verifier/suites>
[2] pry(#<Busser::Command::SuiteCleanup>)> Pathname.glob(suite_path + "*")
=> []
[3] pry(#<Busser::Command::SuiteCleanup>)> p = Pathname.new('c:/Users/vagrant/AppData/Local/Temp/verifier/suites')
=> #<Pathname:c:/Users/vagrant/AppData/Local/Temp/verifier/suites>
[4] pry(#<Busser::Command::SuiteCleanup>)> Pathname.glob(p + "*")
=> [#<Pathname:c:/Users/vagrant/AppData/Local/Temp/verifier/suites/serverspec>]
[5] pry(#<Busser::Command::SuiteCleanup>)>

Investigate support for running suites as specific non-root users

Originally came in as test-kitchen/busser-bats#4 but should be generalized if possible to all runner plugins.

/cc @diclophis

Add a busser for testinfra

Testinfra is a new project featured in this weeks DevOps Weekly newsletter. It may be useful to have a busser for it. It's alpha software.

@fnichol I didn't know where the best place to open this issue is, so feel free to reopen elsewhere if there's a better project for tracking this. 🍨

Kitchen and Busser are ignoring the http_proxy parameter (and it doesn't have a gem source option)

Hello there,

Recently, I started to use Kitchen to make some tests on our cookbooks here. As part of our framework, these cookbooks must be tested on our AWS environment, to assure their reliability and working.

Given that, I made the following kitchen yml file:


---
driver:
  driver_config:
  name: ec2
  aws_ssh_key_id: xxxxxxxxxxxxKey
  shared_credentials_profile: default
  require_chef_omnibus: true
  region: us-east-1
  iam_profile_name: xxxxxxxxxxIAM
  instance_type: t2.large
  associate_public_ip: false
  interface: private


transport:
  ssh_key: '/Users/xxxxx/workspace/Keys/xxxxxx.pem'
  connection_timeout: 10
  connection_retries: 5
  ssh_timeout: 5
  username: centos


provisioner:
  name: chef_zero
  http_proxy: 'http://xxxxxx:3128'
  https_proxy: 'http://xxxx:3128'
  require_chef_omnibus: '12.12.15'
  nodes_path: 'test/nodes'

platforms:
  - name: centos-7.1
    driver:
      image_id: ami-xxxxxxxx
      subnet_id:  subnet-4xxxxxx
      security_group_ids: ["sg-xxxxxxx"]

suites:
  - name: default
    http_proxy: 'http://xxxx:3128'
    https_proxy: 'http://xxxx:3128'
    run_list:
      - recipe[squid-aws::default]
    attributes:
      ddnsupdate:
        no_ddnssec: true
        use_resolv_conf: true
        ttl: 300
      yum:
        main:
          proxy: 'http://xxxxx:3128'

Everything works and has been converged, and we need proxy for every installation at this machine, because our instances there aren't exposed to the internet directly.

But, when we hit the Busser installation stage, we got this:

-----> Verifying <default-centos-71>...
       Preparing files for transfer
D      Creating local sandbox in /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/default-centos-71-sandbox-20160720-72793-1x6udaz
D      [SSH] reusing existing connection [email protected]<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60, :timeout=>10, :keys_only=>true, :keys=>["/Users/dduarte/workspace/Keys/3cSharedLab.pem"], :auth_methods=>["publickey"], :user=>"centos"}>
D      [SSH] [email protected]<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60, :timeout=>10, :keys_only=>true, :keys=>["/Users/dduarte/workspace/Keys/3cSharedLab.pem"], :auth_methods=>["publickey"], :user=>"centos"}> (sh -c '
BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE
ruby="/opt/chef/embedded/bin/ruby"
gem="/opt/chef/embedded/bin/gem"
version="busser"
gem_install_args="busser --no-rdoc --no-ri --no-format-executable -n /tmp/verifier/bin --no-user-install"
busser="sudo -E /tmp/verifier/bin/busser"
plugins="busser-serverspec"

$gem list busser -i 2>&1 >/dev/null
if test $? -ne 0; then
  echo "-----> Installing Busser ($version)"
  $gem install $gem_install_args
else
  echo "-----> Busser installation detected ($version)"
fi

if test ! -f "$BUSSER_ROOT/bin/busser"; then
  $busser setup
fi

echo "       Installing Busser plugins: $plugins"
$busser plugin install $plugins
')
-----> Installing Busser (busser)
^CD      Cleaning up local sandbox in /var/folders/tl/lcf69pls00q84xm27bxc9b75_fdjrs/T/default-centos-71-sandbox-20160720-72793-1x6udaz
D      [SSH] shutting previous connection centos@xxxxxxxxxxxxx<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60, :timeout=>10, :keys_only=>true, :keys=>["/Users/xxxx/workspace/Keys/xxxxxxxx.pem"], :auth_methods=>["publickey"], :user=>"centos"}>
D      [SSH] closing connection to centos@1xxxxxxxx<{:user_known_hosts_file=>"/dev/null", :paranoid=>false, :port=>22, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60, :timeout=>10, :keys_only=>true, :keys=>["/Users/dxxxxxxx/workspace/Keys/xxxxxxxx.pem"], :auth_methods=>["publickey"], :user=>"centos"}>

And the installs hangs for good on "installing busser"

I tried a lot of http_proxy combinations on kitchen yml file, but they haven't worked, and Busser's installation behaves as the same way, not using the proxy.

Any suggestions?