Module: Beaker::DSL::InstallUtils

Included in:
Beaker::DSL
Defined in:
lib/beaker/dsl/install_utils.rb

Overview

This module contains methods to help cloning, extracting git info, ordering of Puppet packages, and installing ruby projects that contain an ‘install.rb` script.

Constant Summary collapse

SourcePath =

The default install path

"/opt/puppet-git-repos"
GitURI =

A regex to know if the uri passed is pointing to a git repo

%r{^(git|https?|file)://|^git@|^gitmirror@}
GitHubSig =

Github’s ssh signature for cloning via ssh

'github.com,207.97.227.239 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ=='

Instance Method Summary collapse

Instance Method Details

#deploy_frictionless_to_master(host) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Classify the master so that it can deploy frictionless packages for a given host.

Parameters:

  • host (Host) —

    The host to install pacakges for



244
245
246
247
248
249
250
 
# File 'lib/beaker/dsl/install_utils.rb', line 244

def deploy_frictionless_to_master(host)
  klass = host['platform'].gsub(/-/, '_').gsub(/\./,'')
  klass = "pe_repo::platform::#{klass}"
  on dashboard, "cd /opt/puppet/share/puppet-dashboard && /opt/puppet/bin/bundle exec rake nodeclass:add[#{klass},skip]"
  on dashboard, "cd /opt/puppet/share/puppet-dashboard && /opt/puppet/bin/bundle exec rake node:addclass[#{master},#{klass}]"
  on master, "puppet agent -t", :acceptable_exit_codes => [0,2]
end

#do_install(hosts, options = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Perform a Puppet Enterprise upgrade or install

Examples:

do_install(hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win =>  version_win})

Parameters:

  • hosts (Array<Host>) —

    The hosts to install or upgrade PE on

  • options (Hash{Symbol=>Symbol, String}) (defaults to: {}) —

    The options

Options Hash (options):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Otherwise uses individual Windows hosts pe_ver)

  • :installer (String) — default: 'puppet-enterprise-installer' —

    The name of the installer to use for upgrading/installing

  • :type (Symbol) — default: :install —

    One of :upgrade or :install



270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
 
# File 'lib/beaker/dsl/install_utils.rb', line 270

def do_install hosts, options = {}
  options[:installer] = options[:installer] || 'puppet-enterprise-installer' 
  options[:type] = options[:type] || :install 
  hostcert='uname | grep -i sunos > /dev/null && hostname || hostname -s'
  master_certname = on(master, hostcert).stdout.strip
  pre30database = version_is_less(options[:pe_ver] || database['pe_ver'], '3.0')
  pre30master = version_is_less(options[:pe_ver] || master['pe_ver'], '3.0')

  # Set PE distribution for all the hosts, create working dir
  use_all_tar = ENV['PE_USE_ALL_TAR'] == 'true'
  hosts.each do |host|
    if host['platform'] !~ /windows/
      platform = use_all_tar ? 'all' : host['platform']
      version = options[:pe_ver] || host['pe_ver']
      host['dist'] = "puppet-enterprise-#{version}-#{platform}"
    end
    host['working_dir'] = "/tmp/" + Time.new.strftime("%Y-%m-%d_%H.%M.%S") #unique working dirs make me happy
    on host, "mkdir #{host['working_dir']}"
  end

  fetch_puppet(hosts, options)

  hosts.each do |host|
    # Database host was added in 3.0. Skip it if installing an older version
    next if host == database and host != master and host != dashboard and pre30database
    if host['platform'] =~ /windows/
      on host, "#{installer_cmd(host, options)} PUPPET_MASTER_SERVER=#{master} PUPPET_AGENT_CERTNAME=#{host}"
    else
      # We only need answers if we're using the classic installer
      version = options[:pe_ver] || host['pe_ver']
      if (! host['roles'].include? 'frictionless') || version_is_less(version, '3.2.0')
        answers = Beaker::Answers.answers(options[:pe_ver] || host['pe_ver'], hosts, master_certname, options)
        create_remote_file host, "#{host['working_dir']}/answers", Beaker::Answers.answer_string(host, answers)
      else
        # If We're *not* running the classic installer, we want
        # to make sure the master has packages for us.
        deploy_frictionless_to_master(host)
      end

      on host, installer_cmd(host, options)
    end
  end


  # If we're installing a database version less than 3.0, ignore the database host
  install_hosts = hosts.dup
  install_hosts.delete(database) if pre30database and database != master and database != dashboard

  # On each agent, we ensure the certificate is signed then shut down the agent
  install_hosts.each do |host|
    sign_certificate_for(host)
    stop_agent_on(host)
  end

  # Wait for PuppetDB to be totally up and running (post 3.0 version of pe only)
  sleep_until_puppetdb_started(database) unless pre30database

  # Run the agent once to ensure everything is in the dashboard
  install_hosts.each do |host|
    on host, puppet_agent('-t'), :acceptable_exit_codes => [0,2]

    # Workaround for PE-1105 when deploying 3.0.0
    # The installer did not respect our database host answers in 3.0.0,
    # and would cause puppetdb to be bounced by the agent run. By sleeping
    # again here, we ensure that if that bounce happens during an upgrade
    # test we won't fail early in the install process.
    if host['pe_ver'] == '3.0.0' and host == database
      sleep_until_puppetdb_started(database)
    end
  end 

  install_hosts.each do |host|
    wait_for_host_in_dashboard(host)
  end

  if pre30master
    task = 'nodegroup:add_all_nodes group=default'
  else
    task = 'defaultgroup:ensure_default_group'
  end
  on dashboard, "/opt/puppet/bin/rake -sf /opt/puppet/share/puppet-dashboard/Rakefile #{task} RAILS_ENV=production"

  # Now that all hosts are in the dashbaord, run puppet one more
  # time to configure mcollective
  on install_hosts, puppet_agent('-t'), :acceptable_exit_codes => [0,2]
end

#extract_repo_info_from(uri) ⇒ Hash{Symbol=>String}

Returns a hash containing the project name, repository path, and revision (defaults to HEAD)

Examples:

Usage

project = extract_repo_info_from '[email protected]:puppetlabs/SuperSecretSauce#what_is_justin_doing'

puts project[:name]
#=> 'SuperSecretSauce'

puts project[:rev]
#=> 'what_is_justin_doing'

Parameters:

  • uri (String) —

    A uri in the format of <git uri>#<revision> the ‘git://`, `http://`, `https://`, and ssh (if cloning as the remote git user) protocols are valid for <git uri>

Returns:

  • (Hash{Symbol=>String}) —

    Returns a hash containing the project name, repository path, and revision (defaults to HEAD)



39
40
41
42
43
44
45
46
 
# File 'lib/beaker/dsl/install_utils.rb', line 39

def extract_repo_info_from uri
  project = {}
  repo, rev = uri.split('#', 2)
  project[:name] = Pathname.new(repo).basename('.git').to_s
  project[:path] = repo
  project[:rev]  = rev || 'HEAD'
  return project
end

#fetch_puppet(hosts, options) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the PE package to download/upload per-host, download/upload that package onto the host and unpack it.

Parameters:

  • hosts (Array<Host>) —

    The hosts to download/upload and unpack PE onto

  • options (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (options):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Otherwise uses individual Windows hosts pe_ver)



186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
 
# File 'lib/beaker/dsl/install_utils.rb', line 186

def fetch_puppet(hosts, options)
  hosts.each do |host|
    # We install Puppet from the master for frictionless installs, so we don't need to *fetch* anything
    next if host['roles'].include? 'frictionless' and ! version_is_less(options[:pe_ver] || host['pe_ver'], '3.2.0')

    windows = host['platform'] =~ /windows/
    path = options[:pe_dir] || host['pe_dir']
    local = File.directory?(path)
    filename = ""
    extension = ""
    if windows
      version = options[:pe_ver_win] || host['pe_ver']
      filename = "puppet-enterprise-#{version}"
      extension = ".msi"
    else
      filename = "#{host['dist']}"
      extension = ""
      if local
        extension = File.exists?("#{path}/#{filename}.tar.gz") ? ".tar.gz" : ".tar"
      else
        extension = link_exists?("#{path}/#{filename}.tar.gz") ? ".tar.gz" : ".tar"
      end
    end
    if local
      if not File.exists?("#{path}/#{filename}#{extension}")
        raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
      end
      scp_to host, "#{path}/#{filename}#{extension}", "#{host['working_dir']}/#{filename}#{extension}"
      if extension =~ /gz/
        on host, "cd #{host['working_dir']}; gunzip #{filename}#{extension}"
      end
      if extension =~ /tar/
        on host, "cd #{host['working_dir']}; tar -xvf #{filename}.tar"
      end
    else
      if not link_exists?("#{path}/#{filename}#{extension}")
        raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist"
      end
      gunzip = ""
      untar = ""
      save_locally = ""
      if extension =~ /gz/
        gunzip = "| gunzip"
      end
      if extension =~ /tar/
        untar = "| tar -xvf -"
      end
      if extension =~ /msi/
        save_locally = "-O"
      end
      on host, "cd #{host['working_dir']}; curl #{save_locally} #{path}/#{filename}#{extension} #{gunzip} #{untar}"
    end
  end
end

#find_git_repo_versions(host, path, repository) ⇒ Hash

Note:

This requires the helper methods:

Returns Executes git describe on [host] and returns a Hash with the key of [repository] and value of the output from git describe.

Examples:

Getting multiple project versions

versions = [puppet_repo, facter_repo, hiera_repo].inject({}) do |vers, repo_info|
  vers.merge(find_git_repo_versions(host, '/opt/git-puppet-repos', repo_info) )
end

Parameters:

  • host (Host) —

    An object implementing Hosts‘s interface.

  • path (String) —

    The path on the remote [host] to the repository

  • repository (Hash{Symbol=>String}) —

    A hash representing repo info like that emitted by #extract_repo_info_from

Returns:

  • (Hash) —

    Executes git describe on [host] and returns a Hash with the key of [repository] and value of the output from git describe.



82
83
84
85
86
87
88
89
90
91
 
# File 'lib/beaker/dsl/install_utils.rb', line 82

def find_git_repo_versions host, path, repository
  version = {}
  step "Grab version for #{repository[:name]}" do
    on host, "cd #{path}/#{repository[:name]} && " +
              "git describe || true" do
      version[repository[:name]] = stdout.chomp
    end
  end
  version
end

#install_from_git(host, path, repository) ⇒ Object

See Also:



95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
 
# File 'lib/beaker/dsl/install_utils.rb', line 95

def install_from_git host, path, repository
  name   = repository[:name]
  repo   = repository[:path]
  rev    = repository[:rev]
  target = "#{path}/#{name}"

  step "Clone #{repo} if needed" do
    on host, "test -d #{path} || mkdir -p #{path}"
    on host, "test -d #{target} || git clone #{repo} #{target}"
  end

  step "Update #{name} and check out revision #{rev}" do
    commands = ["cd #{target}",
                "remote rm origin",
                "remote add origin #{repo}",
                "fetch origin",
                "clean -fdx",
                "checkout -f #{rev}"]
    on host, commands.join(" && git ")
  end

  step "Install #{name} on the system" do
    # The solaris ruby IPS package has bindir set to /usr/ruby/1.8/bin.
    # However, this is not the path to which we want to deliver our
    # binaries. So if we are using solaris, we have to pass the bin and
    # sbin directories to the install.rb
    install_opts = ''
    install_opts = '--bindir=/usr/bin --sbindir=/usr/sbin' if
      host['platform'].include? 'solaris'

      on host,  "cd #{target} && " +
                "if [ -f install.rb ]; then " +
                "ruby ./install.rb #{install_opts}; " +
                "else true; fi"
  end
end

#install_pe ⇒ Object

Note:

Either pe_ver and pe_dir should be set in the ENV or each host should have pe_ver and pe_dir set individually. Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz) for Unix like systems and puppet-enterprise-VERSION.msi for Windows systems.

Install PE based upon host configuration and options

Examples:

install_pe


452
453
454
455
456
457
458
459
460
461
462
463
464
 
# File 'lib/beaker/dsl/install_utils.rb', line 452

def install_pe 
  #process the version files if necessary
  hosts.each do |host|
    if host['platform'] =~ /windows/
      host['pe_ver'] = host['pe_ver'] || 
        Beaker::Options::PEVersionScraper.load_pe_version(host[:pe_dir] || options[:pe_dir], options[:pe_version_file_win])
    else
      host['pe_ver'] = host['pe_ver'] || 
        Beaker::Options::PEVersionScraper.load_pe_version(host[:pe_dir] || options[:pe_dir], options[:pe_version_file])
    end
  end
  do_install sorted_hosts
end

#install_puppet ⇒ Object

Note:

This will attempt to add a repository for apt.puppetlabs.com on Debian or Ubuntu machines, or yum.puppetlabs.com on EL or Fedora machines, then install the package ‘puppet’

Install POSS based upon host configuration and options

Examples:

install_puppet

Returns:

  • nil



418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
 
# File 'lib/beaker/dsl/install_utils.rb', line 418

def install_puppet
  hosts.each do |host|
    if host['platform'] =~ /el-(5|6)/
      relver = $1
      on host, "rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-#{relver}.noarch.rpm"
      on host, 'yum install -y puppet'
    elsif host['platform'] =~ /fedora-(\d+)/
      relver = $1
      on host, "rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-fedora-#{relver}.noarch.rpm"
      on host, 'yum install -y puppet'
    elsif host['platform'] =~ /(ubuntu|debian)/
      if ! host.check_for_package 'curl'
        on host, 'apt-get install -y curl'
      end
      on host, 'curl -O http://apt.puppetlabs.com/puppetlabs-release-$(lsb_release -c -s).deb'
      on host, 'dpkg -i puppetlabs-release-$(lsb_release -c -s).deb'
      on host, 'apt-get -y -f -m update'
      on host, 'apt-get install -y puppet'
    else
      raise "install_puppet() called for unsupported platform '#{host['platform']}' on '#{host.name}'"
    end
  end
  nil
end

#installer_cmd(host, options) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create the PE install command string based upon the host and options settings

Examples:

on host, "#{installer_cmd(host, options)} -a #{host['working_dir']}/answers"

Parameters:

  • host (Host) —

    The host that PE is to be installed on

  • options (Hash{Symbol=>String}) —

    The options

Options Hash (options):

  • :installer (String) —

    The name of the installer to use for upgrading/installing

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Othersie uses individual Windows hosts pe_ver)

  • String (String :pe_ver Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)) —

    :pe_ver Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)



143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/beaker/dsl/install_utils.rb', line 143

def installer_cmd(host, options)
  version = options[:pe_ver] || host['pe_ver']
  if host['platform'] =~ /windows/
    version = options[:pe_ver_win] || host['pe_ver']
    "cd #{host['working_dir']} && msiexec.exe /qn /i puppet-enterprise-#{version}.msi"
  # Frictionless install didn't exist pre-3.2.0, so in that case we fall
  # through and do a regular install.
  elsif host['roles'].include? 'frictionless' and ! version_is_less(version, '3.2.0')
    "cd #{host['working_dir']} && curl -kO https://#{master}:8140/packages/#{version}/install.bash && bash install.bash"
  else
    "cd #{host['working_dir']}/#{host['dist']} && ./#{options[:installer]} -a #{host['working_dir']}/answers"
  end
end

#link_exists?(link) ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine is a given URL is accessible

Examples:

extension = link_exists?("#{URL}.tar.gz") ? ".tar.gz" : ".tar"

Parameters:

  • link (String) —

    The URL to examine

Returns:

  • (Boolean) —

    true if the URL has a ‘200’ HTTP response code, false otherwise



163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/beaker/dsl/install_utils.rb', line 163

def link_exists?(link)
  require "net/http"
  require "net/https"
  require "open-uri"
  url = URI.parse(link)
  http = Net::HTTP.new(url.host, url.port)
  http.use_ssl = (url.scheme == 'https')
  http.start do |http|
    return http.head(url.request_uri).code == "200"
  end
end

#upgrade_pe(path) ⇒ Object

Note:

Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz) for Unix like systems and puppet-enterprise-VERSION.msi for Windows systems.

Upgrade PE based upon host configuration and options

Examples:

upgrade_pe("http://neptune.puppetlabs.lan/3.0/ci-ready/")

Parameters:

  • path (String) —

    A path (either local directory or a URL to a listing of PE builds). Will contain a LATEST file indicating the latest build to install.



475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
 
# File 'lib/beaker/dsl/install_utils.rb', line 475

def upgrade_pe path 
  version = Options::PEVersionScraper.load_pe_version(path, options[:pe_version_file])
  version_win = Options::PEVersionScraper.load_pe_version(path, options[:pe_version_file_win])
  pre_30 = version_is_less(version, '3.0')
  if pre_30
    do_install(sorted_hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win => version_win, :installer => 'puppet-enterprise-upgrader'})
  else
    do_install(sorted_hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win =>  version_win})
  end
  #at this point we've completed a successful upgrade, update the host pe_ver to reflect that
  hosts.each do |host|
    if host['platform'] =~ /windows/
      host['pe_ver'] = version_win
    else
      host['pe_ver'] = version
    end
  end
end