Module: Beaker::DSL::Helpers::PuppetHelpers

Included in:
Beaker::DSL::Helpers
Defined in:
lib/beaker/dsl/helpers/puppet_helpers.rb

Overview

Methods that help you interact with your puppet installation, puppet must be installed for these methods to execute correctly

Instance Method Summary collapse

Instance Method Details

#apply_manifest(manifest, opts = {}, &block) ⇒ Object

Runs ‘puppet apply’ on default host, piping manifest through stdin

See Also:



433
434
435
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 433

def apply_manifest(manifest, opts = {}, &block)
  apply_manifest_on(default, manifest, opts, &block)
end

#apply_manifest_on(host, manifest, opts = {}, &block) ⇒ Object

no resource changes during its execution.

Parameters:

  • block (Block) —

    This method will yield to a block of code passed by the caller; this can be used for additional validation, etc.

  • opts (Hash) (defaults to: {}) —

    a customizable set of options

Options Hash (opts):

  • :expect_failures (Boolean) — default: false —

    This option enables detailed exit codes and causes a test failure if ‘puppet –apply` indicates there were no failure during its execution.

  • :future_parser (Boolean) — default: false —

    This option enables the future parser option that is available from Puppet verion 3.2 By default it will use the ‘current’ parser.

  • :noop (Boolean) — default: false —

    If this option exists, the the “–noop” command line parameter will be passed to the ‘puppet apply’ command.

  • :modulepath (String) —

    The search path for modules, as a list of directories separated by the system path separator character. (The POSIX path separator is ‘:’, and the Windows path separator is ‘;’.)

  • :debug (String) — default: false —

    If this option exists, the “–debug” command line parameter will be passed to the ‘puppet apply’ command.



355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 355

def apply_manifest_on(host, manifest, opts = {}, &block)
  block_on host do | host |
    on_options = {}
    on_options[:acceptable_exit_codes] = Array(opts[:acceptable_exit_codes])

    puppet_apply_opts = {}
    if opts[:debug]
      puppet_apply_opts[:debug] = nil
    else
      puppet_apply_opts[:verbose] = nil
    end
    puppet_apply_opts[:parseonly] = nil if opts[:parseonly]
    puppet_apply_opts[:trace] = nil if opts[:trace]
    puppet_apply_opts[:parser] = 'future' if opts[:future_parser]
    puppet_apply_opts[:modulepath] = opts[:modulepath] if opts[:modulepath]
    puppet_apply_opts[:noop] = nil if opts[:noop]

    # From puppet help:
    # "... an exit code of '2' means there were changes, an exit code of
    # '4' means there were failures during the transaction, and an exit
    # code of '6' means there were both changes and failures."
    if [opts[:catch_changes],opts[:catch_failures],opts[:expect_failures],opts[:expect_changes]].compact.length > 1
      raise(ArgumentError,
            'Cannot specify more than one of `catch_failures`, ' +
            '`catch_changes`, `expect_failures`, or `expect_changes` ' +
            'for a single manifest')
    end

    if opts[:catch_changes]
      puppet_apply_opts['detailed-exitcodes'] = nil

      # We're after idempotency so allow exit code 0 only.
      on_options[:acceptable_exit_codes] |= [0]
    elsif opts[:catch_failures]
      puppet_apply_opts['detailed-exitcodes'] = nil

      # We're after only complete success so allow exit codes 0 and 2 only.
      on_options[:acceptable_exit_codes] |= [0, 2]
    elsif opts[:expect_failures]
      puppet_apply_opts['detailed-exitcodes'] = nil

      # We're after failures specifically so allow exit codes 1, 4, and 6 only.
      on_options[:acceptable_exit_codes] |= [1, 4, 6]
    elsif opts[:expect_changes]
      puppet_apply_opts['detailed-exitcodes'] = nil

      # We're after changes specifically so allow exit code 2 only.
      on_options[:acceptable_exit_codes] |= [2]
    else
      # Either use the provided acceptable_exit_codes or default to [0]
      on_options[:acceptable_exit_codes] |= [0]
    end

    # Not really thrilled with this implementation, might want to improve it
    # later. Basically, there is a magic trick in the constructor of
    # PuppetCommand which allows you to pass in a Hash for the last value in
    # the *args Array; if you do so, it will be treated specially. So, here
    # we check to see if our caller passed us a hash of environment variables
    # that they want to set for the puppet command. If so, we set the final
    # value of *args to a new hash with just one entry (the value of which
    # is our environment variables hash)
    if opts.has_key?(:environment)
      puppet_apply_opts['ENV'] = opts[:environment]
    end

    file_path = host.tmpfile('apply_manifest.pp')
    create_remote_file(host, file_path, manifest + "\n")

    if host[:default_apply_opts].respond_to? :merge
      puppet_apply_opts = host[:default_apply_opts].merge( puppet_apply_opts )
    end

    on host, puppet('apply', file_path, puppet_apply_opts), on_options, &block
  end
end

#create_tmpdir_for_user(host, name = '/tmp/beaker', user = nil) ⇒ String

Create a temp directory on remote host with a user. Default user is puppet master user.

the ownership of a temp directory. directory. Default value is ‘/tmp/beaker’ directory. If no username is specified, use ‘puppet master –configprint user` to obtain username from master. Raise RuntimeError if this puppet command returns a non-zero exit code.

Parameters:

  • host (Host) —

    A single remote host on which to create and adjust

  • name (String) (defaults to: '/tmp/beaker') —

    A remote path prefix for the new temp

  • user (String) (defaults to: nil) —

    The name of user that should own the temp

Returns:

  • (String) —

    Returns the name of the newly-created dir.



682
683
684
685
686
687
688
689
690
691
692
693
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 682

def create_tmpdir_for_user(host, name='/tmp/beaker', user=nil)
  if not user
    result = on host, puppet("master --configprint user")
    if not result.exit_code == 0
      raise "`puppet master --configprint` failed, check that puppet is installed on #{host} or explicitly pass in a user name."
    end
    user = result.stdout.strip
  end

  create_tmpdir_on(host, name, user)

end

#puppet_group(host) ⇒ Object

Note:

This method assumes puppet is installed on the host.

Return the name of the puppet group.

Parameters:

  • host (Host) —

    One object that acts like a Beaker::Host



41
42
43
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 41

def puppet_group(host)
  return host.puppet('master')['group']
end

#puppet_user(host) ⇒ Object

Note:

This method assumes puppet is installed on the host.

Return the name of the puppet user.

Parameters:

  • host (Host) —

    One object that acts like a Beaker::Host



31
32
33
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 31

def puppet_user(host)
  return host.puppet('master')['user']
end

#run_agent_on(host, arg = '--no-daemonize --verbose --onetime --test', options = {}, &block) ⇒ Object

Deprecated. 


438
439
440
441
442
443
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 438

def run_agent_on(host, arg='--no-daemonize --verbose --onetime --test',
                 options={}, &block)
  block_on host do | host |
    on host, puppet_agent(arg), options, &block
  end
end

#sign_certificate ⇒ Object

prompt the master to sign certs then check to confirm the cert for the default host is signed

See Also:



665
666
667
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 665

def sign_certificate
  sign_certificate_for(default)
end

#sign_certificate_for(host) ⇒ Object

Ensure the host has requested a cert, then sign it

Parameters:

  • host (Host, Array<Host>, String, Symbol) —

    One or more hosts to act upon, or a role (String or Symbol) that identifies one or more hosts.

Returns:

  • nil

Raises:



637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 637

def sign_certificate_for(host)
  block_on host do | host |
    if [master, dashboard, database].include? host

      on host, puppet( 'agent -t' ), :acceptable_exit_codes => [0,1,2]
      on master, puppet( "cert --allow-dns-alt-names sign #{host}" ), :acceptable_exit_codes => [0,24]

    else

      hostname = Regexp.escape host.node_name

      last_sleep = 0
      next_sleep = 1
      (0..10).each do |i|
        fail_test("Failed to sign cert for #{hostname}") if i == 10

        on master, puppet("cert --sign --all --allow-dns-alt-names"), :acceptable_exit_codes => [0,24]
        break if on(master, puppet("cert --list --all")).stdout =~ /\+ "?#{hostname}"?/
        sleep next_sleep
        (last_sleep, next_sleep) = next_sleep, last_sleep+next_sleep
      end

    end
  end
end

#sleep_until_nc_started(host) ⇒ Object 



578
579
580
581
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 578

def sleep_until_nc_started(host)
  curl_with_retries("start nodeclassifier (ssl)",
                    host, "https://#{host.node_name}:4433", [35, 60])
end

#sleep_until_puppetdb_started(host) ⇒ Object 



567
568
569
570
571
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 567

def sleep_until_puppetdb_started(host)
  curl_with_retries("start puppetdb", host, "http://localhost:8080", 0, 120)
  curl_with_retries("start puppetdb (ssl)",
                    host, "https://#{host.node_name}:8081", [35, 60])
end

#sleep_until_puppetserver_started(host) ⇒ Object 



573
574
575
576
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 573

def sleep_until_puppetserver_started(host)
  curl_with_retries("start puppetserver (ssl)",
                    host, "https://#{host.node_name}:8140", [35, 60])
end

#stop_agent ⇒ Object

stops the puppet agent running on the default host

See Also:



620
621
622
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 620

def stop_agent
  stop_agent_on(default)
end

#stop_agent_on(agent) ⇒ Object

stops the puppet agent running on the host

Parameters:

  • agent (Host, Array<Host>, String, Symbol) —

    One or more hosts to act upon, or a role (String or Symbol) that identifies one or more hosts.



585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 585

def stop_agent_on(agent)
  block_on agent do | host |
    vardir = agent.puppet['vardir']
    agent_running = true
    while agent_running
      agent_running = agent.file_exist?("#{vardir}/state/agent_catalog_run.lock")
      if agent_running
        sleep 2
      end
    end

    # The agent service is `pe-puppet` everywhere EXCEPT certain linux distros on PE 2.8
    # In all the case that it is different, this init script will exist. So we can assume
    # that if the script doesn't exist, we should just use `pe-puppet`
    agent_service = 'pe-puppet-agent'
    agent_service = 'pe-puppet' unless agent.file_exist?('/etc/init.d/pe-puppet-agent')

    # Under a number of stupid circumstances, we can't stop the
    # agent using puppet.  This is usually because of issues with
    # the init script or system on that particular configuration.
    avoid_puppet_at_all_costs = false
    avoid_puppet_at_all_costs ||= agent['platform'] =~ /el-4/
    avoid_puppet_at_all_costs ||= agent['pe_ver'] && version_is_less(agent['pe_ver'], '3.2') && agent['platform'] =~ /sles/

    if avoid_puppet_at_all_costs
      # When upgrading, puppet is already stopped. On EL4, this causes an exit code of '1'
      on agent, "/etc/init.d/#{agent_service} stop", :acceptable_exit_codes => [0, 1]
    else
      on agent, puppet_resource('service', agent_service, 'ensure=stopped')
    end
  end
end

#stub_forge(forge_host = nil) ⇒ Object

This wraps the method ‘stub_hosts` and makes the stub specific to the forge alias.

See Also:



561
562
563
564
565
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 561

def stub_forge(forge_host = nil)
  #use global options hash
  forge_host ||= options[:forge_host]
  stub_forge_on(default, forge_host)
end

#stub_forge_on(machine, forge_host = nil) ⇒ Object

This wraps the method ‘stub_hosts_on` and makes the stub specific to the forge alias.

forge api v1 canonical source is forge.puppetlabs.com forge api v3 canonical source is forgeapi.puppetlabs.com

Parameters:

  • machine (String) —

    the host to perform the stub on

  • forge_host (String) (defaults to: nil) —

    The URL to use as the forge alias, will default to using :forge_host in the global options hash



522
523
524
525
526
527
528
529
530
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 522

def stub_forge_on(machine, forge_host = nil)
  #use global options hash
  forge_host ||= options[:forge_host]
  @forge_ip ||= Resolv.getaddress(forge_host)
  block_on machine do | host |
    stub_hosts_on(host, 'forge.puppetlabs.com' => @forge_ip)
    stub_hosts_on(host, 'forgeapi.puppetlabs.com' => @forge_ip)
  end
end

#stub_hosts(ip_spec) ⇒ Object

This method accepts a block and using the puppet resource ‘host’ will setup host aliases before and after that block on the default host

Examples:

Stub puppetlabs.com on the default host to 127.0.0.1

stub_hosts('puppetlabs.com' => '127.0.0.1')

See Also:



509
510
511
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 509

def stub_hosts(ip_spec)
  stub_hosts_on(default, ip_spec)
end

#stub_hosts_on(machine, ip_spec) ⇒ Object

This method using the puppet resource ‘host’ will setup host aliases and register the remove of host aliases via Beaker::TestCase#teardown

A teardown step is also added to make sure unstubbing of the host is removed always.

Examples:

Stub puppetlabs.com on the master to 127.0.0.1

stub_hosts_on(master, 'puppetlabs.com' => '127.0.0.1')

Parameters:

  • machine (Host, Array<Host>, String, Symbol) —

    One or more hosts to act upon, or a role (String or Symbol) that identifies one or more hosts.

  • ip_spec (Hash{String=>String}) —

    a hash containing the host to ip mappings



457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 457

def stub_hosts_on(machine, ip_spec)
  block_on machine do | host |
    ip_spec.each do |address, ip|
      logger.notify("Stubbing address #{address} to IP #{ip} on machine #{host}")
      on( host, puppet('resource', 'host', address, 'ensure=present', "ip=#{ip}") )
    end

    teardown do
      ip_spec.each do |address, ip|
        logger.notify("Unstubbing address #{address} to IP #{ip} on machine #{host}")
        on( host, puppet('resource', 'host', address, 'ensure=absent') )
      end
    end
  end
end

#wait_for_host_in_dashboard(host) ⇒ Object

wait for a given host to appear in the dashboard



625
626
627
628
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 625

def wait_for_host_in_dashboard(host)
  hostname = host.node_name
  retry_on(dashboard, "! curl --tlsv1 -k -I https://#{dashboard}/nodes/#{hostname} | grep '404 Not Found'")
end

#with_forge_stubbed(forge_host = nil, &block) ⇒ Object

This wraps ‘with_forge_stubbed_on` and provides it the default host

See Also:



553
554
555
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 553

def with_forge_stubbed( forge_host = nil, &block )
  with_forge_stubbed_on( default, forge_host, &block )
end

#with_forge_stubbed_on(host, forge_host = nil, &block) ⇒ Object

This wraps the method ‘with_host_stubbed_on` and makes the stub specific to the forge alias.

forge api v1 canonical source is forge.puppetlabs.com forge api v3 canonical source is forgeapi.puppetlabs.com

Parameters:

  • host (String) —

    the host to perform the stub on

  • forge_host (String) (defaults to: nil) —

    The URL to use as the forge alias, will default to using :forge_host in the global options hash



541
542
543
544
545
546
547
548
549
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 541

def with_forge_stubbed_on( host, forge_host = nil, &block )
  #use global options hash
  forge_host ||= options[:forge_host]
  @forge_ip ||= Resolv.getaddress(forge_host)
  with_host_stubbed_on( host,
                        {'forge.puppetlabs.com'  => @forge_ip,
                       'forgeapi.puppetlabs.com' => @forge_ip},
                        &block                                    )
end

#with_host_stubbed_on(host, ip_spec, &block) ⇒ Object

This method accepts a block and using the puppet resource ‘host’ will setup host aliases before and after that block.

Examples:

Stub puppetlabs.com on the master to 127.0.0.1

with_host_stubbed_on(master, 'forgeapi.puppetlabs.com' => '127.0.0.1') do
  puppet( "module install puppetlabs-stdlib" )
end

Parameters:

  • host (Host, Array<Host>, String, Symbol) —

    One or more hosts to act upon, or a role (String or Symbol) that identifies one or more hosts.

  • ip_spec (Hash{String=>String}) —

    a hash containing the host to ip mappings



484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 484

def with_host_stubbed_on(host, ip_spec, &block)
  begin
    block_on host do |host|
      ip_spec.each_pair do |address, ip|
        logger.notify("Stubbing address #{address} to IP #{ip} on machine #{host}")
        on( host, puppet('resource', 'host', address, 'ensure=present', "ip=#{ip}") )
      end
    end

    block.call

  ensure
    ip_spec.each do |address, ip|
      logger.notify("Unstubbing address #{address} to IP #{ip} on machine #{host}")
      on( host, puppet('resource', 'host', address, 'ensure=absent') )
    end
  end
end

#with_puppet_running(conf_opts, testdir = host.tmpdir(File.basename(@path)), &block) ⇒ Object

Test Puppet running in a certain run mode with specific options, on the default host

See Also:



190
191
192
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 190

def with_puppet_running conf_opts, testdir = host.tmpdir(File.basename(@path)), &block
  with_puppet_running_on(default, conf_opts, testdir, &block)
end

#with_puppet_running_on(host, conf_opts, testdir = host.tmpdir(File.basename(@path)), &block) ⇒ Object

Test Puppet running in a certain run mode with specific options. This ensures the following steps are performed:

  1. The pre-test Puppet configuration is backed up

  2. A new Puppet configuraton file is layed down

  3. Puppet is started or restarted in the specified run mode

  4. Ensure Puppet has started correctly

  5. Further tests are yielded to

  6. Revert Puppet to the pre-test state

  7. Testing artifacts are saved in a folder named for the test

Examples:

A simple use case to ensure a master is running

with_puppet_running_on( master ) do
    ...tests that require a master...
end

Fully utilizing the possiblities of config options

with_puppet_running_on( master,
                        :main => {:logdest => '/var/blah'},
                        :master => {:masterlog => '/elswhere'},
                        :agent => {:server => 'localhost'} ) do

  ...tests to be ran...
end

Parameters:

  • host (Host) —

    One object that act like Host

  • conf_opts (Hash{Symbol=>String}) —

    Represents puppet settings. Sections of the puppet.conf may be specified, if no section is specified the a puppet.conf file will be written with the options put in a section named after [mode]

  • testdir (File) (defaults to: host.tmpdir(File.basename(@path))) —

    The temporary directory which will hold backup configuration, and other test artifacts.

  • block (Block) —

    The point of this method, yields so tests may be ran. After the block is finished puppet will revert to a previous state.

Options Hash (conf_opts):

  • :__commandline_args__ (String) —

    A special setting for command_line arguments such as –debug or –logdest, which cannot be set in puppet.conf. For example:

    :commandline_args => ‘–logdest /tmp/a.log’

    These will only be applied when starting a FOSS master, as a pe master is just bounced.

  • :__service_args__ (Hash) —

    A special setting of options for controlling how the puppet master service is handled. The only setting currently is :bypass_service_script, which if set true will force stopping and starting a webrick master using the start_puppet_from_source_* methods, even if it seems the host has passenger. This is needed in FOSS tests to initialize SSL.

Raises:

  • (ArgumentError) 


101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
 
# File 'lib/beaker/dsl/helpers/puppet_helpers.rb', line 101

def with_puppet_running_on host, conf_opts, testdir = host.tmpdir(File.basename(@path)), &block
  raise(ArgumentError, "with_puppet_running_on's conf_opts must be a Hash. You provided a #{conf_opts.class}: '#{conf_opts}'") if !conf_opts.kind_of?(Hash)
  cmdline_args = conf_opts[:__commandline_args__]
  service_args = conf_opts[:__service_args__] || {}
  conf_opts = conf_opts.reject { |k,v| [:__commandline_args__, :__service_args__].include?(k) }

  curl_retries = host['master-start-curl-retries'] || options['master-start-curl-retries']
  logger.debug "Setting curl retries to #{curl_retries}"

  if options[:is_puppetserver]
    confdir = host.puppet('master')['confdir']
    vardir = host.puppet('master')['vardir']

    if cmdline_args
      split_args = cmdline_args.split()

      split_args.each do |arg|
        case arg
        when /--confdir=(.*)/
          confdir = $1
        when /--vardir=(.*)/
          vardir = $1
        end
      end
    end

    puppetserver_opts = { "jruby-puppet" => {
      "master-conf-dir" => confdir,
      "master-var-dir" => vardir,
    }}

    puppetserver_conf = File.join("#{host['puppetserver-confdir']}", "puppetserver.conf")
    modify_tk_config(host, puppetserver_conf, puppetserver_opts)
  end
  begin
    backup_file = backup_the_file(host, host.puppet('master')['confdir'], testdir, 'puppet.conf')
    lay_down_new_puppet_conf host, conf_opts, testdir

    if host.use_service_scripts? && !service_args[:bypass_service_script]
      bounce_service( host, host['puppetservice'], curl_retries )
    else
      puppet_master_started = start_puppet_from_source_on!( host, cmdline_args )
    end

    yield self if block_given?

  rescue Beaker::DSL::Assertions, Minitest::Assertion => early_assertion
    fail_test(early_assertion)
  rescue Exception => early_exception
    original_exception = RuntimeError.new("PuppetAcceptance::DSL::Helpers.with_puppet_running_on failed (check backtrace for location) because: #{early_exception}\n#{early_exception.backtrace.join("\n")}\n")
    raise(original_exception)

  ensure
    begin

      if host.use_service_scripts? && !service_args[:bypass_service_script]
        restore_puppet_conf_from_backup( host, backup_file )
        bounce_service( host, host['puppetservice'], curl_retries )
      else
        if puppet_master_started
          stop_puppet_from_source_on( host )
        else
          dump_puppet_log(host)
        end
        restore_puppet_conf_from_backup( host, backup_file )
      end

    rescue Exception => teardown_exception
      begin
        if !host.is_pe?
          dump_puppet_log(host)
        end
      rescue Exception => dumping_exception
        logger.error("Raised during attempt to dump puppet logs: #{dumping_exception}")
      end

      if original_exception
        logger.error("Raised during attempt to teardown with_puppet_running_on: #{teardown_exception}\n---\n")
        raise original_exception
      else
        raise teardown_exception
      end
    end
  end
end