Module: Beaker::DSL::Helpers

Included in:
Beaker::DSL
Defined in:
lib/beaker/dsl/helpers.rb

Overview

This is the heart of the Puppet Acceptance DSL. Here you find a helper to proxy commands to hosts, more commands to move files between hosts and execute remote scripts, confine test cases to certain hosts and prepare the state of a test case.

To mix this is into a class you need the following:

  • a method hosts that yields any hosts implementing Host‘s interface to act upon.

  • a method logger that yields a logger implementing Logger‘s interface.

Instance Method Summary collapse

Instance Method Details

#apply_manifest_on(host, manifest, opts = {}, &block) ⇒ Object

Runs ‘puppet apply’ on a remote host, piping manifest through stdin

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.

  • :parseonly (Boolean) — default: false —

    If this key is true, the “–parseonly” command line parameter will be passed to the ‘puppet apply’ command.

  • :trace (Boolean) — default: false —

    If this key exists in the Hash, the “–trace” command line parameter will be passed to the ‘puppet apply’ command.

  • :catch_failures (Boolean) — default: false —

    By default “puppet –apply” will exit with 0, which does not count as a test failure, even if there were errors applying the manifest. This option enables detailed exit codes and causes a test failure if “puppet –apply” indicates there was a failure during its execution.



437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
 
# File 'lib/beaker/dsl/helpers.rb', line 437

def apply_manifest_on(host, manifest, opts = {}, &block)
  on_options = {:stdin => manifest + "\n"}
  on_options[:acceptable_exit_codes] = opts.delete(:acceptable_exit_codes)
  args = ["--verbose"]
  args << "--parseonly" if opts[:parseonly]
  args << "--trace" if opts[:trace]

  if opts[:catch_failures]
    args << '--detailed-exitcodes'

    # From puppet help:
    # "... an exit code of '2' means there were changes, an exit code of
    # '4' means there were failures during the transaction, and an exit
    # code of '6' means there were both changes and failures."
    # We're after failures specifically so catch exit codes 4 and 6 only.
    on_options[:acceptable_exit_codes] |= [0, 2]
  end

  # Not really thrilled with this implementation, might want to improve it
  # later.  Basically, there is a magic trick in the constructor of
  # PuppetCommand which allows you to pass in a Hash for the last value in
  # the *args Array; if you do so, it will be treated specially.  So, here
  # we check to see if our caller passed us a hash of environment variables
  # that they want to set for the puppet command.  If so, we set the final
  # value of *args to a new hash with just one entry (the value of which
  # is our environment variables hash)
  if opts.has_key?(:environment)
    args << { :environment => opts[:environment]}
  end

  on host, puppet( 'apply', *args), on_options, &block
end

#check_for_package(host, package_name) ⇒ Boolean

Check to see if a package is installed on a remote host



126
127
128
 
# File 'lib/beaker/dsl/helpers.rb', line 126

def check_for_package host, package_name
  host.check_for_package package_name
end

#confine(type, criteria, host_array = nil, &block) ⇒ Array<Host>

Note:

This will modify the TestCase#hosts member in place unless an array of hosts is passed into it and TestCase#logger yielding an object that responds like Logger#warn, as well as Outcomes#skip_test, and optionally TestCase#hosts.

Limit the hosts a test case is run against

Examples:

Basic usage to confine to debian OSes.

confine :to, :platform => 'debian'

Confining to anything but Windows and Solaris

confine :except, :platform => ['windows', 'solaris']

Using additional block to confine to Solaris global zone.

confine :to, :platform => 'solaris' do |solaris|
  on( solaris, 'zonename' ) =~ /global/
end

Raises:

  • (SkipTest) —

    Raises skip test if there are no valid hosts for this test case after confinement.



227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
 
# File 'lib/beaker/dsl/helpers.rb', line 227

def confine(type, criteria, host_array = nil, &block)
  provided_hosts = host_array ? true : false
  hosts_to_modify = host_array || hosts
  criteria.each_pair do |property, value|
    case type
    when :except
      hosts_to_modify = hosts_to_modify.reject do |host|
        inspect_host host, property, value
      end
      if block_given?
        hosts_to_modify = hosts_to_modify.reject do |host|
          yield host
        end
      end
    when :to
      hosts_to_modify = hosts_to_modify.select do |host|
        inspect_host host, property, value
      end
      if block_given?
        hosts_to_modify = hosts_to_modify.select do |host|
          yield host
        end
      end
    else
      raise "Unknown option #{type}"
    end
  end
  if hosts_to_modify.empty?
    logger.warn "No suitable hosts with: #{criteria.inspect}"
    skip_test 'No suitable hosts found'
  end
  self.hosts = hosts_to_modify
  hosts_to_modify
end

#create_remote_file(hosts, file_path, file_content, opts = {}) ⇒ Result

Note:

This method uses Tempfile in Ruby’s STDLIB as well as #scp_to.

Create a remote file out of a string

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.



151
152
153
154
155
156
157
 
# File 'lib/beaker/dsl/helpers.rb', line 151

def create_remote_file(hosts, file_path, file_content, opts = {})
  Tempfile.open 'beaker' do |tempfile|
    File.open(tempfile.path, 'w') {|file| file.puts file_content }

    scp_to hosts, tempfile.path, file_path, opts
  end
end

#curl_with_retries(desc, host, url, desired_exit_codes, max_retries = 60, retry_interval = 1) ⇒ Object 



554
555
556
 
# File 'lib/beaker/dsl/helpers.rb', line 554

def curl_with_retries(desc, host, url, desired_exit_codes, max_retries = 60, retry_interval = 1)
  retry_command(desc, host, "curl #{url}", desired_exit_codes, max_retries, retry_interval)
end

#install_package(host, package_name) ⇒ Result

Install a package on a host



136
137
138
 
# File 'lib/beaker/dsl/helpers.rb', line 136

def install_package host, package_name
  host.install_package package_name
end

#on(host, command, opts = {}, &block) ⇒ Result

The primary method for executing commands on some set of hosts.

Examples:

Most basic usage

on hosts, 'ls /tmp'

Allowing additional exit codes to pass

on agents, 'puppet agent -t', :acceptable_exit_codes => [0,2]

Using the returned result for any kind of checking

if on(host, 'ls -la ~').stdout =~ /\.bin/
  ...do some action...
end

Using TestCase helpers from within a test.

agents.each do |agent|
  on agent, 'cat /etc/puppet/puppet.conf' do
    assert_match stdout, /server = #{master}/, 'WTF Mate'
  end
end

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.

Raises:

  • (FailTest) —

    Raises an exception if command obviously fails.



59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'lib/beaker/dsl/helpers.rb', line 59

def on(host, command, opts = {}, &block)
  unless command.is_a? Command
    cmd_opts = opts[:environment] ? { 'ENV' => opts.delete(:environment) } : Hash.new
    command = Command.new(command.to_s, [], cmd_opts)
  end
  if host.is_a? Array
    host.map { |h| on h, command, opts, &block }
  else
    @result = host.exec(command, opts)

    # Also, let additional checking be performed by the caller.
    yield self if block_given?

    return @result
  end
end

#port_open_within?(host, port = 8140, seconds = 120) ⇒ Boolean

Blocks until the port is open on the host specified, returns false on failure



403
404
405
406
407
 
# File 'lib/beaker/dsl/helpers.rb', line 403

def port_open_within?( host, port = 8140, seconds = 120 )
  repeat_for( seconds ) do
    host.port_open?( port )
  end
end

#retry_command(desc, host, command, desired_exit_codes = 0, max_retries = 60, retry_interval = 1) ⇒ Object 



558
559
560
561
562
563
564
565
566
567
568
569
570
 
# File 'lib/beaker/dsl/helpers.rb', line 558

def retry_command(desc, host, command, desired_exit_codes = 0, max_retries = 60, retry_interval = 1)
  desired_exit_codes = [desired_exit_codes].flatten
  result = on host, command, :acceptable_exit_codes => (0...127)
  num_retries = 0
  until desired_exit_codes.include?(result.exit_code)
    sleep retry_interval
    result = on host, command, :acceptable_exit_codes => (0...127)
    num_retries += 1
    if (num_retries > max_retries)
      fail("Unable to #{desc}")
    end
  end
end

#run_agent_on(host, arg = '--no-daemonize --verbose --onetime --test', options = {}, &block) ⇒ Object

Deprecated. 


471
472
473
474
475
476
477
478
 
# File 'lib/beaker/dsl/helpers.rb', line 471

def run_agent_on(host, arg='--no-daemonize --verbose --onetime --test',
                 options={}, &block)
  if host.is_a? Array
    host.each { |h| run_agent_on h, arg, options, &block }
  else
    on host, puppet_agent(arg), options, &block
  end
end

#run_script_on(host, script, opts = {}, &block) ⇒ Result

Note:

this relies on #on and #scp_to

Move a local script to a remote host and execute it

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.



170
171
172
173
174
175
176
177
178
179
180
181
 
# File 'lib/beaker/dsl/helpers.rb', line 170

def run_script_on(host, script, opts = {}, &block)
  # this is unsafe as it uses the File::SEPARATOR will be set to that
  # of the coordinator node.  This works for us because we use cygwin
  # which will properly convert the paths.  Otherwise this would not
  # work for running tests on a windows machine when the coordinator
  # that the harness is running on is *nix. We should use
  # {Beaker::Host#temp_path} instead. TODO
  remote_path = File.join("", "tmp", File.basename(script))

  scp_to host, script, remote_path
  on host, remote_path, opts, &block
end

#scp_from(host, from_path, to_path, opts = {}) ⇒ Result

Note:

If using Host for the hosts scp is not required on the system as it uses Ruby’s net/scp library. The net-scp gem however is required (and specified in the gemspec).

Move a file from a remote to a local path

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.



89
90
91
92
93
94
95
96
 
# File 'lib/beaker/dsl/helpers.rb', line 89

def scp_from host, from_path, to_path, opts = {}
  if host.is_a? Array
    host.each { |h| scp_from h, from_path, to_path, opts }
  else
    @result = host.do_scp_from(from_path, to_path, opts)
    @result.log logger
  end
end

#scp_to(host, from_path, to_path, opts = {}) ⇒ Result

Note:

If using Host for the hosts scp is not required on the system as it uses Ruby’s net/scp library. The net-scp gem however is required (and specified in the gemspec.

Move a local file to a remote host

Options Hash (opts):

  • :silent (Boolean) — default: false —

    Do not produce log output

  • :acceptable_exit_codes (Array<Fixnum>) — default: [0] —

    An array (or range) of integer exit codes that should be considered acceptable. An error will be thrown if the exit code does not match one of the values in this list.

  • :environment (Hash{String=>String}) — default: {} —

    These will be treated as extra environment variables that should be set before running the command.



111
112
113
114
115
116
117
118
 
# File 'lib/beaker/dsl/helpers.rb', line 111

def scp_to host, from_path, to_path, opts = {}
  if host.is_a? Array
    host.each { |h| scp_to h, from_path, to_path, opts }
  else
    @result = host.do_scp_to(from_path, to_path, opts)
    @result.log logger
  end
end

#sign_certificate(host) ⇒ Object

prompt the master to sign certs then check to confirm the cert for this host is signed



607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
 
# File 'lib/beaker/dsl/helpers.rb', line 607

def sign_certificate(host)
  return if [master, dashboard, database].include? host
 
  hostname = Regexp.escape host.node_name
 
  last_sleep = 0
  next_sleep = 1
  (0..10).each do |i|
    fail_test("Failed to sign cert for #{hostname}") if i == 10
 
    on master, puppet("cert --sign --all"), :acceptable_exit_codes => [0,24]
    break if on(master, puppet("cert --list --all")).stdout =~ /\+ "?#{hostname}"?/
    sleep next_sleep
    (last_sleep, next_sleep) = next_sleep, last_sleep+next_sleep
  end
end

#sleep_until_puppetdb_started(host) ⇒ Object 



548
549
550
551
552
 
# File 'lib/beaker/dsl/helpers.rb', line 548

def sleep_until_puppetdb_started(host)
  curl_with_retries("start puppetdb", host, "http://localhost:8080", 0, 120)
  curl_with_retries("start puppetdb (ssl)",
                    host, "https://#{host.node_name}:8081", [35, 60])
end

#stop_agent(agent) ⇒ Object

stops the puppet agent running on the host



573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
 
# File 'lib/beaker/dsl/helpers.rb', line 573

def stop_agent(agent)
  vardir = agent.puppet['vardir']
  agent_running = true
  while agent_running
    result = on agent, "[ -e '#{vardir}/state/agent_catalog_run.lock' ]", :acceptable_exit_codes => [0,1]
    agent_running = (result.exit_code == 0)
    sleep 2 unless agent_running
  end
 
  if agent['platform'].include?('solaris')
    on(agent, '/usr/sbin/svcadm disable -s svc:/network/pe-puppet:default')
  elsif agent['platform'].include?('aix')
    on(agent, '/usr/bin/stopsrc -s pe-puppet')
  elsif agent['platform'].include?('windows')
    on(agent, 'net stop pe-puppet', :acceptable_exit_codes => [0,2])
  else
    # For the sake of not passing the PE version into this method,
    # we just query the system to find out which service we want to
    # stop
    result = on agent, "[ -e /etc/init.d/pe-puppet-agent ]", :acceptable_exit_codes => [0,1]
    service = (result.exit_code == 0) ? 'pe-puppet-agent' : 'pe-puppet'
    on(agent, "/etc/init.d/#{service} stop")
  end
end

#stub_forge_on(machine) ⇒ Object

This wraps the method ‘stub_hosts_on` and makes the stub specific to the forge alias.



544
545
546
547
 
# File 'lib/beaker/dsl/helpers.rb', line 544

def stub_forge_on(machine)
  @forge_ip ||= Resolv.getaddress(forge)
  stub_hosts_on(machine, 'forge.puppetlabs.com' => @forge_ip)
end

#stub_hosts_on(machine, ip_spec) ⇒ Object

This method accepts a block and using the puppet resource ‘host’ will setup host aliases before and after that block.

A teardown step is also added to make sure unstubbing of the host is removed always.

Examples:

Stub puppetlabs.com on the master to 127.0.0.1

stub_hosts_on(master, 'puppetlabs.com' => '127.0.0.1')



524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
 
# File 'lib/beaker/dsl/helpers.rb', line 524

def stub_hosts_on(machine, ip_spec)
  ip_spec.each do |host, ip|
    logger.notify("Stubbing host #{host} to IP #{ip} on machine #{machine}")
    on( machine,
        puppet('resource', 'host', host, 'ensure=present', "ip=#{ip}") )
  end

  teardown do
    ip_spec.each do |host, ip|
      logger.notify("Unstubbing host #{host} to IP #{ip} on machine #{machine}")
      on( machine,
          puppet('resource', 'host', host, 'ensure=absent') )
    end
  end
end

#wait_for_host_in_dashboard(host) ⇒ Object

wait for a given host to appear in the dashboard



600
601
602
603
 
# File 'lib/beaker/dsl/helpers.rb', line 600

def wait_for_host_in_dashboard(host)
  hostname = host.node_name
  retry_command("Wait for #{hostname} to be in the console", dashboard, "! curl --sslv3 -k -I https://#{dashboard}/nodes/#{hostname} | grep '404 Not Found'")
end

#with_puppet_running_on(host, conf_opts, testdir = host.tmpdir(File.basename(@path)), &block) ⇒ Object

Test Puppet running in a certain run mode with specific options. This ensures the following steps are performed:

  1. The pre-test Puppet configuration is backed up

  2. A new Puppet configuraton file is layed down

  3. Puppet is started or restarted in the specified run mode

  4. Ensure Puppet has started correctly

  5. Further tests are yielded to

  6. Revert Puppet to the pre-test state

  7. Testing artifacts are saved in a folder named for the test

Examples:

A simple use case to ensure a master is running

with_puppet_running_on( master ) do
    ...tests that require a master...
end

Fully utilizing the possiblities of config options

with_puppet_running_on( master,
                        :main => {:logdest => '/var/blah'},
                        :master => {:masterlog => '/elswhere'},
                        :agent => {:server => 'localhost'} ) do

  ...tests to be ran...
end



316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
 
# File 'lib/beaker/dsl/helpers.rb', line 316

def with_puppet_running_on host, conf_opts, testdir = host.tmpdir(File.basename(@path)), &block
  begin
    backup_file host, host['puppetpath'], testdir, 'puppet.conf'
    lay_down_new_puppet_conf host, conf_opts, testdir

    if host.is_pe?
      bounce_service( 'pe-httpd' )

    else
      start_puppet_from_source_on!( host )
    end

    yield self if block_given?
  ensure
    restore_puppet_conf_from_backup( host )

    if host.is_pe?
      bounce_service( 'pe-httpd' )

    else
      stop_puppet_from_source_on( host )
    end
  end
end