voxpupuli / beaker-rspec Goto Github PK
View Code? Open in Web Editor NEWbeaker-rspec is a bridge between the puppet acceptance test harness
License: Apache License 2.0
beaker-rspec is a bridge between the puppet acceptance test harness
License: Apache License 2.0
In allot of cases when i run multiple test instances via jenkins they all want to take port 2222.
Would it be an idea to be able to set the SSH Port number via a config or an ENV variable?
Or even chose a random number would help.
if it can be via an EVN variable I could have some kind of increment thing happening
hi there,
i am using the example from the beaker-rspec and having problems finding my class.
here is my error:
[/Users/nayeem.syed/Documents/bgch/puppet-installer:feature/add-resolv-conf:d62c707:+I+M+U]$ rspec spec/acceptance/bgch_bastion_rspec.rb
Beaker::Hypervisor, found some vagrant boxes to create
==> ubuntu-server-1404-x64: Forcing shutdown of VM...
==> ubuntu-server-1404-x64: Destroying VM and associated drives...
Bringing machine 'ubuntu-server-1404-x64' up with 'virtualbox' provider...
==> ubuntu-server-1404-x64: Importing base box 'puppetlabs/ubuntu-14.04-64-nocm'...
==> ubuntu-server-1404-x64: Matching MAC address for NAT networking...
==> ubuntu-server-1404-x64: Checking if box 'puppetlabs/ubuntu-14.04-64-nocm' is up to date...
==> ubuntu-server-1404-x64: Setting the name of the VM: defaultyml_ubuntu-server-1404-x64_1425394474386_17455
==> ubuntu-server-1404-x64: Clearing any previously set network interfaces...
==> ubuntu-server-1404-x64: Preparing network interfaces based on configuration...
ubuntu-server-1404-x64: Adapter 1: nat
ubuntu-server-1404-x64: Adapter 2: hostonly
==> ubuntu-server-1404-x64: Forwarding ports...
ubuntu-server-1404-x64: 22 => 2222 (adapter 1)
==> ubuntu-server-1404-x64: Running 'pre-boot' VM customizations...
==> ubuntu-server-1404-x64: Booting VM...
==> ubuntu-server-1404-x64: Waiting for machine to boot. This may take a few minutes...
ubuntu-server-1404-x64: SSH address: 127.0.0.1:2222
ubuntu-server-1404-x64: SSH username: vagrant
ubuntu-server-1404-x64: SSH auth method: private key
ubuntu-server-1404-x64: Warning: Connection timeout. Retrying...
==> ubuntu-server-1404-x64: Machine booted and ready!
==> ubuntu-server-1404-x64: Checking for guest additions in VM...
==> ubuntu-server-1404-x64: Setting hostname...
==> ubuntu-server-1404-x64: Configuring and enabling network interfaces...
==> ubuntu-server-1404-x64: Mounting shared folders...
ubuntu-server-1404-x64: /vagrant => /Users/nayeem.syed/Documents/bgch/puppet-installer/.vagrant/beaker_vagrant_files/default.yml
Allocated a PTY on 10.255.44.172 for "sudo su -c \"cp -r .ssh /root/.\""
Allocated a PTY on 10.255.44.172 for "sudo su -c \"sed -ri 's/^#?PermitRootLogin no|^#?PermitRootLogin yes/PermitRootLogin yes/' /etc/ssh/sshd_config\""
Allocated a PTY on 10.255.44.172 for "sudo su -c \"service ssh restart\""
localhost $ scp /Users/nayeem.syed/Documents/bgch/puppet-installer/modules/bgch_bastion ubuntu-server-1404-x64:/etc/puppetlabs/puppet/modules/bgch_bastion {:ignore => [".bundle", ".git", ".idea", ".vagrant", ".vendor", "vendor", "acceptance", "bundle", "spec", "tests", "log", ".", ".."]}
bgch_bastion class
running puppet code
localhost $ scp /var/folders/cg/j_8tw58n60nc098k36yfvwvm000120/T/beaker20150303-25498-7dl2fx ubuntu-server-1404-x64:/tmp/apply_manifest.pp.kVqx1G {:ignore => }
should work with no errors (FAILED - 1)
Destroying vagrant boxes
==> ubuntu-server-1404-x64: Forcing shutdown of VM...
==> ubuntu-server-1404-x64: Destroying VM and associated drives...
Failures:
1) bgch_bastion class running puppet code should work with no errors
On host ``
Failure/Error: apply_manifest(pp, :catch_failures => true)
Beaker::Host::CommandFailure: Host 'ubuntu-server-1404-x64' exited with 1 running:
puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.kVqx1G
Last 10 lines of output were:
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
(at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1139:in `issue_deprecation_warning')
Info: Loading facts
Error: Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find declared class bgch_bastion at /tmp/apply_manifest.pp.kVqx1G:1 on node ubuntu-server-1404-x64
Wrapped exception:
Could not find declared class bgch_bastion
Error: Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find declared class bgch_bastion at /tmp/apply_manifest.pp.kVqx1G:1 on node ubuntu-server-1404-x64
Host 'ubuntu-server-1404-x64' exited with 1 running:
puppet apply --verbose --detailed-exitcodes /tmp/apply_manifest.pp.kVqx1G
Last 10 lines of output were:
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
(at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1139:in `issue_deprecation_warning')
Info: Loading facts
Error: Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find declared class bgch_bastion at /tmp/apply_manifest.pp.kVqx1G:1 on node ubuntu-server-1404-x64
Wrapped exception:
Could not find declared class bgch_bastion
Error: Puppet::Parser::AST::Resource failed with error ArgumentError: Could not find declared class bgch_bastion at /tmp/apply_manifest.pp.kVqx1G:1 on node ubuntu-server-1404-x64
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/host.rb:290:in `exec'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/helpers.rb:88:in `block in on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/shared/host_manager.rb:95:in `run_block_on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/patterns.rb:32:in `block_on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/helpers.rb:78:in `on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/helpers.rb:953:in `block in apply_manifest_on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/shared/host_manager.rb:95:in `run_block_on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/patterns.rb:32:in `block_on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/helpers.rb:882:in `apply_manifest_on'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/beaker-2.5.0/lib/beaker/dsl/helpers.rb:960:in `apply_manifest'
# ./spec/acceptance/bgch_bastion_rspec.rb:11:in `block (3 levels) in <top (required)>'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/extensions/instance_eval_with_args.rb:16:in `instance_exec'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/extensions/instance_eval_with_args.rb:16:in `instance_eval_with_args'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example.rb:116:in `block in run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example.rb:248:in `with_around_each_hooks'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example.rb:113:in `run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:515:in `block in run_examples'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:511:in `map'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:511:in `run_examples'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:496:in `run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:497:in `block in run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:497:in `map'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/example_group.rb:497:in `run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/command_line.rb:24:in `block (2 levels) in run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/command_line.rb:24:in `map'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/command_line.rb:24:in `block in run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/reporter.rb:58:in `report'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/command_line.rb:21:in `run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/runner.rb:103:in `run'
# /Users/nayeem.syed/gems/ruby/2.1.0/gems/rspec-core-2.99.2/lib/rspec/core/runner.rb:17:in `block in autorun'
Finished in 14.42 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/acceptance/bgch_bastion_rspec.rb:5 # bgch_bastion class running puppet code should work with no errors
here is my spec file:
require 'spec_helper_acceptance'
describe 'bgch_bastion class' do
describe 'running puppet code' do
it 'should work with no errors' do
pp = <<-EOS
class { 'bgch_bastion': }
EOS
# Run it twice and test for idempotency
apply_manifest(pp, :catch_failures => true)
expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero
end
end
end
and here is the spec_helper_acceptance.rb:
require 'beaker-rspec'
require 'pry'
hosts.each do |host|
# Install Puppet
on host, install_puppet
on hosts, "mkdir -p #{host['distmoduledir']}"
end
RSpec.configure do |c|
module_root = File.expand_path(File.join(File.dirname(__FILE__), '..'))
module_name = module_root.split('/').last
c.formatter = :documentation
# Configure all nodes in nodeset
c.before :suite do
# Install module
# puppet_module_install(:source => module_root, :module_name => module_name)
puppet_module_install(:source => "#{module_root}/modules/bgch_bastion", :module_name => 'bgch_bastion')
hosts.each do |host|
shell("/bin/touch #{default['puppetpath']}/hiera.yaml")
on host, puppet('module','install','puppetlabs-stdlib'), { :acceptable_exit_codes => [0,1] }
# binding.pry
end
end
end
When i try enabling pry like mentioned on the doc I get the error below:
[1] pry(#<RSpec::Core::ExampleGroup>)> hosts
NoMethodError: undefined method `process_line' for #<SpecInfra::Command::Ubuntu:0x007faba537bee8>
from /Users/nayeem.syed/gems/ruby/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:404:in `process_command'
any ideas why i am having this problem... it seems quite straightforward on the doc but not quite working.
I tried asking around on #puppet-dev irc with no luck either... :-(
Allow setting colors on/off with beaker-rspec, to help clear up Jenkins/Travis ci etc output
When running rspec tests the results are outputted twice. Running in debug shows them as only being run once.
Adapter
Adapter
Port "9700"
Port "9700"
default_host 23:45:36$ /bin/sh -c netstat\ -tunl\ \|\ grep\ --\ :9700\\\
tcp 0 0 0.0.0.0:9700 0.0.0.0:* LISTEN
default_host executed in 0.04 seconds
should be listening on port 9700
should be listening on port 9700
The test that creates this looks like this:
describe 'Adapter' do
describe port('9700') do
it 'should be listening on port 9700' do should be_listening end
end
end
beaker-rspec 4.0.0 has dependencies that cannot be satisfied.
beaker-rspec requires beaker 2.0 which require rspec ~> 3.0
beaker-rspec also requires serverspec ~> 1.0 which requires rspec ~> 2.99
The install_puppet function does not yet understand the new package repository (i.e. PC1), package name (i.e. puppetserver), or the new filesystem paths for puppet 4. When can we expect an update for this?
beaker-rspec should support the same semantics for :pre_condition
like rspec-puppet already does.
$ bundle exec rspec spec/acceptance/server_spec.rb
/opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/beaker-rspec-2.0.0/lib/beaker-rspec/beaker_shim.rb:35:in `validate': uninitialized constant Beaker::Utils (NameError)
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/beaker-rspec-2.0.0/lib/beaker-rspec/spec_helper.rb:22:in `block in <top (required)>'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core.rb:79:in `configure'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/beaker-rspec-2.0.0/lib/beaker-rspec/spec_helper.rb:5:in `<top (required)>'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/beaker-rspec-2.0.0/lib/beaker-rspec.rb:5:in `require'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/beaker-rspec-2.0.0/lib/beaker-rspec.rb:5:in `<module:BeakerRSpec>'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/beaker-rspec-2.0.0/lib/beaker-rspec.rb:1:in `<top (required)>'
from /Users/peterso/Projects/puppetlabs-mongodb/spec/spec_helper_acceptance.rb:2:in `require'
from /Users/peterso/Projects/puppetlabs-mongodb/spec/spec_helper_acceptance.rb:2:in `<top (required)>'
from /Users/peterso/Projects/puppetlabs-mongodb/spec/acceptance/server_spec.rb:1:in `require'
from /Users/peterso/Projects/puppetlabs-mongodb/spec/acceptance/server_spec.rb:1:in `<top (required)>'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/configuration.rb:1051:in `load'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/configuration.rb:1051:in `block in load_spec_files'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/configuration.rb:1051:in `each'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/configuration.rb:1051:in `load_spec_files'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/runner.rb:97:in `setup'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/runner.rb:85:in `run'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/runner.rb:70:in `run'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/lib/rspec/core/runner.rb:38:in `invoke'
from /opt/boxen/rbenv/versions/2.0.0-p451/lib/ruby/gems/2.0.0/gems/rspec-core-3.0.0/exe/rspec:4:in `<top (required)>'
from /opt/boxen/rbenv/versions/2.0.0-p451/bin/rspec:23:in `load'
from /opt/boxen/rbenv/versions/2.0.0-p451/bin/rspec:23:in `<main>'
I'm working on adding tests for an existing module that sets up a cluster. There are cluster roles such as master, peer, searchhead and forwarder.
I've got my nodeset configured to bring up the appropriate vagrant VMs. What I need to do is run certain test cases on all node and others on hosts with a specific role.
I can see confine and confine_block block methods in the Beaker DSL but have no idea how to use them in beaker-rspec. I've tried the following without success:
confine_block :to, role: 'searchhead' do
describe service('myservice') do
it { should be_running }
end
end
The other thing I could do split out tests for individual roles into different spec files. But even then I'm not sure how to run specific spec files against specific roles without some nasty hackery.
Any suggestions appreciated.
While I understand using environmental variables to pass options such as "provision" or "destroy" to beaker via beaker-rspec, this isn't efficient if I want to make "provision=no" and/or "destroy=no" a default when I run my tests.
Beaker-rpec should support setting these defaults via the acceptance spec helper which can be merged with the spec_helper.rb defaults and over-written by any environmental variables.
I'm not sure how to accomplish this, since the spec_helper.rb gets executed before my acceptance spec helper.
Similar to the rspec-system one: https://github.com/puppetlabs/rspec-system/blob/master/lib/rspec-system/rake_task.rb
I've Upgraded beaker 1.8.0 to 1.9.0 and the run fails because it needs beaker-rspec 2.2.1
So wanted to upgrade beaker-rspec since it was still on 2.2.0.
bundle update failed so pinned beaker-rspec in the Gemfile to 2.2.1 and now getting this error:
Resolving dependencies...
Bundler could not find compatible versions for gem "beaker":
In Gemfile:
beaker-rspec (~> 2.2.1) ruby depends on
beaker (~> 1.8.2) ruby
beaker (1.9.0)
/Users/petersouter/.bundler/ruby/2.0.0/beaker-rspec-bbf806a4fc12/lib/beaker-rspec/helpers/serverspec.rb:12: warning: already initialized constant Module::VALID_OPTIONS_KEYS
/Library/Ruby/Gems/2.0.0/gems/specinfra-2.11.5/lib/specinfra/configuration.rb:4: warning: previous definition of VALID_OPTIONS_KEYS was here
Not sure if this is related to #58 or not
With the latest 5.0.0 release, I'm getting duplicated rspec output - if I pin my Gemfile to 4.0.0, this doesn't happen at all.
Example of output:
centos7 executed in 0.02 seconds
should be installed
should be installed
Service "jenkins"
Service "jenkins"
centos7 07:24:13$ /bin/sh -c systemctl\ --plain\ list-dependencies\ multi-user.target\ \|\ grep\ \'\\\(\^\\\|\ \\\)jenkins.service\$\'
jenkins.service
centos7 executed in 0.03 seconds
should be enabled
should be enabled
centos7 07:24:13$ /bin/sh -c systemctl\ is-active\ jenkins
tive
▽
centos7 executed in 0.02 seconds
should be running
should be running
File "/var/lib/jenkins/jobs/SEED_generate_jobs"
File "/var/lib/jenkins/jobs/SEED_generate_jobs"
centos7 07:24:13$ /bin/sh -c test\ -d\ /var/lib/jenkins/jobs/SEED_generate_jobs
centos7 executed in 0.02 seconds
should be a directory
should be a directory
Finished in 10 minutes 1 second (files took 49.03 seconds to load)
6 examples, 0 failures
Finished in 10 minutes 1 second (files took 49.03 seconds to load)
6 examples, 0 failures
Gemfile:
source "https://rubygems.org"
group :test do
gem "rake"
gem "puppet", ENV['PUPPET_VERSION'] || '~> 3.6.0'
gem "puppet-lint"
gem "puppet-lint-indent-check"
gem "rspec", '< 3.2.0'
gem "rspec-puppet", :git => 'https://github.com/rodjek/rspec-puppet.git'
gem "puppet-syntax"
gem "puppetlabs_spec_helper"
gem "hiera-puppet-helper", :git => 'https://github.com/bobtfish/hiera-puppet-helper.git'
end
group :development do
gem "travis"
gem "travis-lint"
gem "beaker", :git => "https://github.com/puppetlabs/beaker.git",
gem "beaker-rspec"
gem "docker-api"
gem "vagrant-wrapper"
gem "puppet-blacksmith"
gem "guard-rake"
end
Is this perhaps related to the change to using serverspec
> 2.x ? That appears to be the main difference when comparing the Gemfile.lock
when the beaker-rspec
gem is pinned to version 4.0.0
beaker-rspec (4.0.0)
beaker (~> 2.0)
rspec
serverspec (~> 1.0)
specinfra (~> 1.0)
vs
beaker-rspec (5.0.2)
beaker (~> 2.0)
rspec
serverspec (~> 2)
specinfra (~> 2)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.