Module: Beaker::DSL::InstallUtils

Included in:
Beaker::DSL
Defined in:
lib/beaker/dsl/install_utils.rb

Overview

This module contains methods to help cloning, extracting git info, ordering of Puppet packages, and installing ruby projects that contain an ‘install.rb` script.

Constant Summary collapse

SourcePath =

The default install path

"/opt/puppet-git-repos"
GitURI =

A regex to know if the uri passed is pointing to a git repo

%r{^(git|https?|file)://|^git@}
GitHubSig =

Github’s ssh signature for cloning via ssh

'github.com,207.97.227.239 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ=='

Instance Method Summary collapse

Instance Method Details

#do_install(hosts, options = {}) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Perform a Puppet Enterprise upgrade or install

Examples:

do_install(hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win =>  version_win})

Parameters:

  • hosts (Array<Host>) —

    The hosts to install or upgrade PE on

  • options (Hash{Symbol=>Symbol, String}) (defaults to: {}) —

    The options

Options Hash (options):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Otherwise uses individual Windows hosts pe_ver)

  • :installer (String) — default: 'puppet-enterprise-installer' —

    The name of the installer to use for upgrading/installing

  • :type (Symbol) — default: :install —

    One of :upgrade or :install



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
 
# File 'lib/beaker/dsl/install_utils.rb', line 237

def do_install hosts, options = {}
  options[:installer] = options[:installer] || 'puppet-enterprise-installer' 
  options[:type] = options[:type] || :install 
  hostcert='uname | grep -i sunos > /dev/null && hostname || hostname -s'
  master_certname = on(master, hostcert).stdout.strip
  pre30database = version_is_less(options[:pe_ver] || database['pe_ver'], '3.0')
  pre30master = version_is_less(options[:pe_ver] || master['pe_ver'], '3.0')

  # Set PE distribution for all the hosts, create working dir
  use_all_tar = ENV['PE_USE_ALL_TAR'] == 'true'
  hosts.each do |host|
    if host['platform'] !~ /windows/
      platform = use_all_tar ? 'all' : host['platform']
      version = options[:pe_ver] || host['pe_ver']
      host['dist'] = "puppet-enterprise-#{version}-#{platform}"
    end
    host['working_dir'] = "/tmp/" + Time.new.strftime("%Y-%m-%d_%H.%M.%S") #unique working dirs make me happy
    on host, "mkdir #{host['working_dir']}"
  end

  fetch_puppet(hosts, options)

  hosts.each do |host|
    # Database host was added in 3.0. Skip it if installing an older version
    next if host == database and host != master and host != dashboard and pre30database
    if host['platform'] =~ /windows/
      on host, "#{installer_cmd(host, options)} PUPPET_MASTER_SERVER=#{master} PUPPET_AGENT_CERTNAME=#{host}"
    else
      answers = Beaker::Answers.answers(options[:pe_ver] || host['pe_ver'], hosts, master_certname, options)
      create_remote_file host, "#{host['working_dir']}/answers", Beaker::Answers.answer_string(host, answers)

      on host, "#{installer_cmd(host, options)} -a #{host['working_dir']}/answers"
    end
  end


  # If we're installing a database version less than 3.0, ignore the database host
  install_hosts = hosts.dup
  install_hosts.delete(database) if pre30database and database != master and database != dashboard

  # On each agent, we ensure the certificate is signed then shut down the agent
  install_hosts.each do |host|
    sign_certificate_for(host)
    stop_agent_on(host)
  end

  # Wait for PuppetDB to be totally up and running (post 3.0 version of pe only)
  sleep_until_puppetdb_started(database) unless pre30database

  # Run the agent once to ensure everything is in the dashboard
  install_hosts.each do |host|
    on host, puppet_agent('-t'), :acceptable_exit_codes => [0,2]

    # Workaround for PE-1105 when deploying 3.0.0
    # The installer did not respect our database host answers in 3.0.0,
    # and would cause puppetdb to be bounced by the agent run. By sleeping
    # again here, we ensure that if that bounce happens during an upgrade
    # test we won't fail early in the install process.
    if host['pe_ver'] == '3.0.0' and host == database
      sleep_until_puppetdb_started(database)
    end
  end 

  install_hosts.each do |host|
    wait_for_host_in_dashboard(host)
  end

  if pre30master
    task = 'nodegroup:add_all_nodes group=default'
  else
    task = 'defaultgroup:ensure_default_group'
  end
  on dashboard, "/opt/puppet/bin/rake -sf /opt/puppet/share/puppet-dashboard/Rakefile #{task} RAILS_ENV=production"

  # Now that all hosts are in the dashbaord, run puppet one more
  # time to configure mcollective
  on install_hosts, puppet_agent('-t'), :acceptable_exit_codes => [0,2]
end

#extract_repo_info_from(uri) ⇒ Hash{Symbol=>String}

Returns a hash containing the project name, repository path, and revision (defaults to HEAD)

Examples:

Usage

project = extract_repo_info_from '[email protected]:puppetlabs/SuperSecretSauce#what_is_justin_doing'

puts project[:name]
#=> 'SuperSecretSauce'

puts project[:rev]
#=> 'what_is_justin_doing'

Parameters:

  • uri (String) —

    A uri in the format of <git uri>#<revision> the ‘git://`, `http://`, `https://`, and ssh (if cloning as the remote git user) protocols are valid for <git uri>

Returns:

  • (Hash{Symbol=>String}) —

    Returns a hash containing the project name, repository path, and revision (defaults to HEAD)



39
40
41
42
43
44
45
46
 
# File 'lib/beaker/dsl/install_utils.rb', line 39

def extract_repo_info_from uri
  project = {}
  repo, rev = uri.split('#', 2)
  project[:name] = Pathname.new(repo).basename('.git').to_s
  project[:path] = repo
  project[:rev]  = rev || 'HEAD'
  return project
end

#fetch_puppet(hosts, options) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine the PE package to download/upload per-host, download/upload that package onto the host and unpack it.

Parameters:

  • hosts (Array<Host>) —

    The hosts to download/upload and unpack PE onto

  • options (Hash{Symbol=>Symbol, String}) —

    The options

Options Hash (options):

  • :pe_dir (String) —

    Default directory or URL to pull PE package from (Otherwise uses individual hosts pe_dir)

  • :pe_ver (String) —

    Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Otherwise uses individual Windows hosts pe_ver)



179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
 
# File 'lib/beaker/dsl/install_utils.rb', line 179

def fetch_puppet(hosts, options)
  hosts.each do |host|
    windows = host['platform'] =~ /windows/
    path = options[:pe_dir] || host['pe_dir']
    local = File.directory?(path)
    filename = ""
    extension = ""
    if windows
      version = options[:pe_ver_win] || host['pe_ver']
      filename = "puppet-enterprise-#{version}"
      extension = ".msi"
    else
      filename = "#{host['dist']}"
      extension = ""
      if local
        extension = File.exists?("#{path}/#{filename}.tar.gz") ? ".tar.gz" : ".tar"
      else
        extension = link_exists?("#{path}/#{filename}.tar.gz") ? ".tar.gz" : ".tar"
      end
    end
    if local
       if not File.exists?("#{path}/#{filename}#{extension}")
         raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist" 
       end
       scp_to host, "#{path}/#{filename}#{extension}", "#{host['working_dir']}/#{filename}#{extension}"
    else
       if not link_exists?("#{path}/#{filename}#{extension}")
         raise "attempting installation on #{host}, #{path}/#{filename}#{extension} does not exist" 
       end
       on host, "cd #{host['working_dir']}; curl #{path}/#{filename}#{extension} -o #{filename}#{extension}"
    end
    if extension =~ /gz/
      on host, "cd #{host['working_dir']}; gunzip #{filename}#{extension}"
    end
    if extension =~ /tar/
      on host, "cd #{host['working_dir']}; tar -xvf #{filename}.tar"
    end
  end
end

#find_git_repo_versions(host, path, repository) ⇒ Hash

Note:

This requires the helper methods:

Returns Executes git describe on [host] and returns a Hash with the key of [repository] and value of the output from git describe.

Examples:

Getting multiple project versions

versions = [puppet_repo, facter_repo, hiera_repo].inject({}) do |vers, repo_info|
  vers.merge(find_git_repo_versions(host, '/opt/git-puppet-repos', repo_info) )
end

Parameters:

  • host (Host) —

    An object implementing Hosts‘s interface.

  • path (String) —

    The path on the remote [host] to the repository

  • repository (Hash{Symbol=>String}) —

    A hash representing repo info like that emitted by #extract_repo_info_from

Returns:

  • (Hash) —

    Executes git describe on [host] and returns a Hash with the key of [repository] and value of the output from git describe.



82
83
84
85
86
87
88
89
90
91
 
# File 'lib/beaker/dsl/install_utils.rb', line 82

def find_git_repo_versions host, path, repository
  version = {}
  step "Grab version for #{repository[:name]}" do
    on host, "cd #{path}/#{repository[:name]} && " +
              "git describe || true" do
      version[repository[:name]] = stdout.chomp
    end
  end
  version
end

#install_from_git(host, path, repository) ⇒ Object

See Also:



95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
 
# File 'lib/beaker/dsl/install_utils.rb', line 95

def install_from_git host, path, repository
  name   = repository[:name]
  repo   = repository[:path]
  rev    = repository[:rev]
  target = "#{path}/#{name}"

  step "Clone #{repo} if needed" do
    on host, "test -d #{path} || mkdir -p #{path}"
    on host, "test -d #{target} || git clone #{repo} #{target}"
  end

  step "Update #{name} and check out revision #{rev}" do
    commands = ["cd #{target}",
                "remote rm origin",
                "remote add origin #{repo}",
                "fetch origin",
                "clean -fdx",
                "checkout -f #{rev}"]
    on host, commands.join(" && git ")
  end

  step "Install #{name} on the system" do
    # The solaris ruby IPS package has bindir set to /usr/ruby/1.8/bin.
    # However, this is not the path to which we want to deliver our
    # binaries. So if we are using solaris, we have to pass the bin and
    # sbin directories to the install.rb
    install_opts = ''
    install_opts = '--bindir=/usr/bin --sbindir=/usr/sbin' if
      host['platform'].include? 'solaris'

      on host,  "cd #{target} && " +
                "if [ -f install.rb ]; then " +
                "ruby ./install.rb #{install_opts}; " +
                "else true; fi"
  end
end

#install_pe ⇒ Object

Note:

Either pe_ver and pe_dir should be set in the ENV or each host should have pe_ver and pe_dir set individually. Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz) for Unix like systems and puppet-enterprise-VERSION.msi for Windows systems.

Install PE based upon host configuration and options

Examples:

install_pe


376
377
378
379
380
381
382
383
384
385
386
387
388
 
# File 'lib/beaker/dsl/install_utils.rb', line 376

def install_pe 
  #process the version files if necessary
  hosts.each do |host|
    if host['platform'] =~ /windows/
      host['pe_ver'] = host['pe_ver'] || 
        Beaker::Options::PEVersionScraper.load_pe_version(host[:pe_dir] || options[:pe_dir], options[:pe_version_file_win])
    else
      host['pe_ver'] = host['pe_ver'] || 
        Beaker::Options::PEVersionScraper.load_pe_version(host[:pe_dir] || options[:pe_dir], options[:pe_version_file])
    end
  end
  do_install sorted_hosts
end

#installer_cmd(host, options) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create the PE install command string based upon the host and options settings

Examples:

on host, "#{installer_cmd(host, options)} -a #{host['working_dir']}/answers"

Parameters:

  • host (Host) —

    The host that PE is to be installed on

  • options (Hash{Symbol=>String}) —

    The options

Options Hash (options):

  • :installer (String) —

    The name of the installer to use for upgrading/installing

  • :pe_ver_win (String) —

    Default PE version to install or upgrade to on Windows hosts (Othersie uses individual Windows hosts pe_ver)

  • String (String :pe_ver Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)) —

    :pe_ver Default PE version to install or upgrade to (Otherwise uses individual hosts pe_ver)



143
144
145
146
147
148
149
150
151
 
# File 'lib/beaker/dsl/install_utils.rb', line 143

def installer_cmd(host, options)
  if host['platform'] =~ /windows/
    version = options[:pe_ver_win] || host['pe_ver']
    "cd #{host['working_dir']} && msiexec.exe /qn /i puppet-enterprise-#{version}.msi"
  else
    version = options[:pe_ver] || host['pe_ver']
    "cd #{host['working_dir']}/#{host['dist']} && ./#{options[:installer]}"
  end
end

#link_exists?(link) ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Determine is a given URL is accessible

Examples:

extension = link_exists?("#{URL}.tar.gz") ? ".tar.gz" : ".tar"

Parameters:

  • link (String) —

    The URL to examine

Returns:

  • (Boolean) —

    true if the URL has a ‘200’ HTTP response code, false otherwise



159
160
161
162
163
164
165
166
 
# File 'lib/beaker/dsl/install_utils.rb', line 159

def link_exists?(link)
  require "net/http"
  require "open-uri"
  url = URI.parse(link)
  Net::HTTP.start(url.host, url.port) do |http|
    return http.head(url.request_uri).code == "200"
  end
end

#upgrade_pe(path) ⇒ Object

Note:

Install file names are assumed to be of the format puppet-enterprise-VERSION-PLATFORM.(tar)|(tar.gz) for Unix like systems and puppet-enterprise-VERSION.msi for Windows systems.

Upgrade PE based upon host configuration and options

Examples:

upgrade_pe("http://neptune.puppetlabs.lan/3.0/ci-ready/")

Parameters:

  • path (String) —

    A path (either local directory or a URL to a listing of PE builds). Will contain a LATEST file indicating the latest build to install.



399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
 
# File 'lib/beaker/dsl/install_utils.rb', line 399

def upgrade_pe path 
  version = Options::PEVersionScraper.load_pe_version(path, options[:pe_version_file])
  version_win = Options::PEVersionScraper.load_pe_version(path, options[:pe_version_file_win])
  pre_30 = version_is_less(version, '3.0')
  if pre_30
    do_install(sorted_hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win => version_win, :installer => 'puppet-enterprise-upgrader'})
  else
    do_install(sorted_hosts, {:type => :upgrade, :pe_dir => path, :pe_ver => version, :pe_ver_win =>  version_win})
  end
  #at this point we've completed a successful upgrade, update the host pe_ver to reflect that
  hosts.each do |host|
    if host['platform'] =~ /windows/
      host['pe_ver'] = version_win
    else
      host['pe_ver'] = version
    end
  end
end