Module: Beaker::HostPrebuiltSteps
Overview
Provides convienience methods for commonly run actions on hosts
Constant Summary collapse
- NTPSERVER =
'pool.ntp.org'- SLEEPWAIT =
5- TRIES =
5- ETC_HOSTS_PATH =
"/etc/hosts"- ETC_HOSTS_PATH_SOLARIS =
"/etc/inet/hosts"- ROOT_KEYS_SCRIPT =
"https://raw.githubusercontent.com/puppetlabs/puppetlabs-sshkeys/master/templates/scripts/manage_root_authorized_keys"- ROOT_DOTFILES_SYNC_CMD =
Prefer the mechanism a booted host uses (the tmpfiles rule); fall back to copying the templates where systemd-tmpfiles is not available.
'if [ -f /usr/lib/tmpfiles.d/rootfiles.conf ] && command -v systemd-tmpfiles >/dev/null 2>&1; then ' \ 'systemd-tmpfiles --create rootfiles.conf; ' \ 'elif [ -d /usr/share/rootfiles ]; then ' \ 'for f in /usr/share/rootfiles/.[!.]*; do [ -e "/root/${f##*/}" ] || cp -p "$f" /root/; done; ' \ 'fi'
- ROOT_KEYS_SYNC_CMD =
"curl -k -o - -L #{ROOT_KEYS_SCRIPT} | %s"- ROOT_KEYS_SYNC_CMD_AIX =
"curl --tlsv1 -o - -L #{ROOT_KEYS_SCRIPT} | %s"
Instance Method Summary collapse
-
#additive_hash_merge(h1, h2) ⇒ Hash
Merge the two provided hashes so that an array of values is created from collisions.
-
#apt_get_update(hosts) ⇒ Object
Run 'apt-get update' on the provided host or hosts.
-
#check_and_install_packages_if_needed(host, package_list) ⇒ Object
Installs the given packages if they aren't already on a host.
-
#construct_env(host, opts) ⇒ Hash
Create the hash of default environment from host (:host_env), global options hash (:host_env) and default PE/Foss puppet variables.
-
#copy_file_to_remote(host, file_path, file_content) ⇒ Object
Create a file on host or hosts at the provided file path with the provided file contents.
-
#copy_ssh_to_root(host, opts) ⇒ Object
Make it possible to log in as root by copying the current users ssh keys to the root account.
-
#disable_se_linux(host, opts) ⇒ Object
Disable SELinux on centos, does nothing on other platforms.
-
#disable_updates(hosts, opts) ⇒ Object
Update /etc/hosts to make updates.puppetlabs.com (aka the dujour server) resolve to 127.0.01, so that we don't pollute the server with test data.
-
#enable_root_login(host, opts) ⇒ Object
Update sshd_config on debian, ubuntu, centos, el, redhat and fedora boxes to allow for root login.
-
#ensure_root_dotfiles(host) ⇒ Object
Make sure root's shell dotfiles exist on EL 8+ and Fedora hosts.
-
#get_domain_name(host) ⇒ Object
Determine the domain name of the provided host from its /etc/resolv.conf.
-
#hack_etc_hosts(hosts, _opts) ⇒ Object
Update /etc/hosts to make it possible for each provided host to reach each other host by name.
-
#host_packages(host) ⇒ Array<String>
Return a list of packages that should be present.
-
#install_one_of_packages(host, packages) ⇒ Object
Installs one of alternative packages (first available).
-
#package_proxy(host, opts) ⇒ Object
Setup files for enabling requests to pass to a proxy server This works for the APT package manager on debian and ubuntu and YUM package manager on el, centos, fedora and redhat.
-
#set_env(host, opts) ⇒ Object
Add a host specific set of env vars to each provided host's ~/.ssh/environment.
-
#set_etc_hosts(host, etc_hosts) ⇒ Object
Append the provided string to the /etc/hosts file of the provided host.
-
#sync_root_keys(host, opts) ⇒ Object
Install a set of authorized keys using ROOT_KEYS_SCRIPT.
-
#timesync(host, opts) ⇒ Object
Run timesync on the provided hosts.
-
#validate_host(host, opts) ⇒ Object
Validate that hosts are prepared to be used as SUTs, if packages are missing attempt to install them.
Methods included from DSL::Patterns
Instance Method Details
#additive_hash_merge(h1, h2) ⇒ Hash
Merge the two provided hashes so that an array of values is created from collisions
432 433 434 435 436 437 438 439 440 441 442 443 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 432 def additive_hash_merge h1, h2 merged_hash = {} normalized_h2 = h2.each_with_object({}) { |(k, v), h| h[k.to_s.upcase] = v; } h1.each_pair do |key, _val| normalized_key = key.to_s.upcase if normalized_h2.has_key?(normalized_key) merged_hash[key] = [h1[key], normalized_h2[normalized_key]] merged_hash[key] = merged_hash[key].uniq # remove dupes end end merged_hash end |
#apt_get_update(hosts) ⇒ Object
Run 'apt-get update' on the provided host or hosts. If the platform of the provided host is not ubuntu or debian: do nothing.
192 193 194 195 196 197 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 192 def apt_get_update hosts block_on hosts do |host| # -qq: Only output errors to stdout host.exec(Command.new("apt-get update -qq")) if /ubuntu|debian/.match?(host[:platform]) end end |
#check_and_install_packages_if_needed(host, package_list) ⇒ Object
Installs the given packages if they aren't already on a host
136 137 138 139 140 141 142 143 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 136 def check_and_install_packages_if_needed host, package_list package_list.each do |string| alternatives = string.split('|') next if alternatives.any? { |pkg| host.check_for_package pkg } install_one_of_packages host, alternatives end end |
#construct_env(host, opts) ⇒ Hash
Create the hash of default environment from host (:host_env), global options hash (:host_env) and default PE/Foss puppet variables
449 450 451 452 453 454 455 456 457 458 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 449 def construct_env host, opts env = additive_hash_merge(host[:host_env], opts[:host_env]) env.each_key do |key| separator = host['pathseparator'] separator = ':' if key == 'PATH' && (not host.is_powershell?) env[key] = env[key].join(separator) end env end |
#copy_file_to_remote(host, file_path, file_content) ⇒ Object
Create a file on host or hosts at the provided file path with the provided file contents.
203 204 205 206 207 208 209 210 211 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 203 def copy_file_to_remote(host, file_path, file_content) block_on host do |host| Tempfile.open 'beaker' do |tempfile| File.open(tempfile.path, 'w') { |file| file.puts file_content } host.do_scp_to(tempfile.path, file_path, @options) end end end |
#copy_ssh_to_root(host, opts) ⇒ Object
Make it possible to log in as root by copying the current users ssh keys to the root account
267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 267 def copy_ssh_to_root host, opts logger = opts[:logger] block_on host do |host| logger.debug "Give root a copy of current user's keys, on #{host.name}" if host['platform'].include?('windows') and host.is_cygwin? host.exec(Command.new('cp -r .ssh /cygdrive/c/Users/Administrator/.')) host.exec(Command.new('chown -R Administrator /cygdrive/c/Users/Administrator/.ssh')) elsif host['platform'].include?('windows') and not host.is_cygwin? # from https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/xcopy.mspx?mfr=true: # /i : If Source is a directory or contains wildcards and Destination # does not exist, xcopy assumes destination specifies a directory # name and creates a new directory. Then, xcopy copies all specified # files into the new directory. By default, xcopy prompts you to # specify whether Destination is a file or a directory. # # /y : Suppresses prompting to confirm that you want to overwrite an # existing destination file. host.exec(Command.new("if exist .ssh (xcopy .ssh C:\\Users\\Administrator\\.ssh /s /e /y /i)")) elsif host['platform'].include?('osx') host.exec(Command.new('sudo cp -r .ssh /var/root/.'), { :pty => true }) elsif /(free|open)bsd/.match?(host['platform']) || host['platform'].include?('solaris-11') host.exec(Command.new('sudo cp -r .ssh /root/.'), { :pty => true }) elsif host['platform'].include?('solaris-10') host.exec(Command.new('sudo cp -r .ssh /.'), { :pty => true }) else host.exec(Command.new('sudo su -c "cp -r .ssh /root/."'), { :pty => true }) end host.exec(Command.new('sudo fixfiles restore /root')) if host.selinux_enabled? end end |
#disable_se_linux(host, opts) ⇒ Object
Disable SELinux on centos, does nothing on other platforms
386 387 388 389 390 391 392 393 394 395 396 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 386 def disable_se_linux host, opts logger = opts[:logger] block_on host do |host| if /centos|el-|redhat|fedora/.match?(host['platform']) logger.debug("Disabling se_linux on #{host.name}") host.exec(Command.new("sudo su -c \"setenforce 0\""), { :pty => true }) else logger.warn("Attempting to disable SELinux on non-supported platform: #{host.name}: #{host['platform']}") end end end |
#disable_updates(hosts, opts) ⇒ Object
Update /etc/hosts to make updates.puppetlabs.com (aka the dujour server) resolve to 127.0.01, so that we don't pollute the server with test data. See SERVER-1000, BKR-182, BKR-237, DJ-10 for additional details.
323 324 325 326 327 328 329 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 323 def disable_updates hosts, opts logger = opts[:logger] hosts.each do |host| logger.notify "Disabling updates.puppetlabs.com by modifying hosts file to resolve updates to 127.0.0.1 on #{host}" set_etc_hosts(host, "127.0.0.1\tupdates.puppetlabs.com\n") end end |
#enable_root_login(host, opts) ⇒ Object
Update sshd_config on debian, ubuntu, centos, el, redhat and fedora boxes to allow for root login
Does nothing on other platfoms.
338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 338 def enable_root_login host, opts logger = opts[:logger] block_on host do |host| logger.debug "Update sshd_config to allow root login" if host['platform'].include?('osx') # If osx > 10.10 use '/private/etc/ssh/sshd_config', else use '/etc/sshd_config' ssh_config_file = '/private/etc/ssh/sshd_config' ssh_config_file = '/etc/sshd_config' if /^osx-10\.(9|10)/i.match?(host['platform']) host.exec(Command.new("sudo sed -i '' 's/#PermitRootLogin no/PermitRootLogin Yes/g' #{ssh_config_file}")) host.exec(Command.new("sudo sed -i '' 's/#PermitRootLogin yes/PermitRootLogin Yes/g' #{ssh_config_file}")) elsif host['platform'].include?('freebsd') host.exec(Command.new("sudo sed -i -e 's/#PermitRootLogin no/PermitRootLogin yes/g' /etc/ssh/sshd_config"), { :pty => true }) elsif host['platform'].include?('openbsd') host.exec(Command.new("sudo perl -pi -e 's/^PermitRootLogin no/PermitRootLogin yes/' /etc/ssh/sshd_config"), { :pty => true }) elsif host['platform'].include?('solaris-10') host.exec(Command.new("sudo gsed -i -e 's/#PermitRootLogin no/PermitRootLogin yes/g' /etc/ssh/sshd_config"), { :pty => true }) elsif host['platform'].include?('solaris-11') host.exec(Command.new("if grep \"root::::type=role\" /etc/user_attr; then sudo rolemod -K type=normal root; else echo \"root user already type=normal\"; fi"), { :pty => true }) host.exec(Command.new("sudo gsed -i -e 's/PermitRootLogin no/PermitRootLogin yes/g' /etc/ssh/sshd_config"), { :pty => true }) elsif host.is_powershell? logger.warn("Attempting to enable root login non-supported platform: #{host.name}: #{host['platform']}") elsif host.is_cygwin? host.exec(Command.new("sed -ri 's/^#?PermitRootLogin /PermitRootLogin yes/' /etc/sshd_config"), { :pty => true }) else host.exec(Command.new("sudo su -c \"sed -ri 's/^#?PermitRootLogin no|^#?PermitRootLogin yes/PermitRootLogin yes/' /etc/ssh/sshd_config\""), { :pty => true }) end # restart sshd if /debian|ubuntu/.match?(host['platform']) host.exec(Command.new("sudo su -c \"service ssh restart\""), { :pty => true }) elsif /(el|centos|redhat|oracle|scientific)-[0-6]\b/.match?(host['platform']) host.exec(Command.new("sudo -E /sbin/service sshd reload"), { :pty => true }) elsif /amazon|arch|centos|el|redhat|fedora/.match?(host['platform']) host.exec(Command.new("sudo -E systemctl restart sshd.service"), { :pty => true }) elsif /(free|open)bsd/.match?(host['platform']) host.exec(Command.new("sudo /etc/rc.d/sshd restart")) elsif host['platform'].include?('solaris') host.exec(Command.new("sudo -E svcadm restart network/ssh"), { :pty => true }) else logger.warn("Attempting to update ssh on non-supported platform: #{host.name}: #{host['platform']}") end end end |
#ensure_root_dotfiles(host) ⇒ Object
Make sure root's shell dotfiles exist on EL 8+ and Fedora hosts.
Since rootfiles 8.1-32 (EL 9+) and 9.0 (Fedora 43+), /root/.bashrc
and friends are no longer shipped as regular files: they are %ghost
entries that systemd-tmpfiles copies out of /usr/share/rootfiles at
boot, per /usr/lib/tmpfiles.d/rootfiles.conf. Nothing else runs that
rule -- the systemd package carries no tmpfiles file trigger -- so in a
container, which is never booted, installing the package leaves /root
without the dotfiles.
That matters because a non-interactive bash started by sshd only
sources /etc/bashrc (and from there /etc/profile.d/*.sh) through
~/.bashrc. Without it, PATH additions made in profile.d, such as
/etc/profile.d/puppet-agent.sh, never reach commands run over ssh.
Apply the tmpfiles rule the way boot would, and fall back to copying
the templates when systemd-tmpfiles is not installed. Either path only
creates files that are missing, so this is a no-op on a booted host,
on hosts whose rootfiles still ships the files directly (no rule and no
/usr/share/rootfiles), and for any customised file already in /root.
110 111 112 113 114 115 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 110 def ensure_root_dotfiles(host) platform = host['platform'] return unless (platform.variant == 'el' && platform.version.to_i >= 8) || platform.variant == 'fedora' host.exec(Command.new(ROOT_DOTFILES_SYNC_CMD), :accept_all_exit_codes => true) end |
#get_domain_name(host) ⇒ Object
Determine the domain name of the provided host from its /etc/resolv.conf
215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 215 def get_domain_name(host) domain = nil search = nil resolv_conf = if host['platform'].include?('windows') if host.is_cygwin? host.exec(Command.new("cat /cygdrive/c/Windows/System32/drivers/etc/hosts")).stdout else host.exec(Command.new('type C:\Windows\System32\drivers\etc\hosts')).stdout end else host.exec(Command.new("cat /etc/resolv.conf")).stdout end resolv_conf.each_line do |line| if (match = /^\s*domain\s+(\S+)/.match(line)) domain = match[1] elsif (match = /^\s*search\s+(\S+)/.match(line)) search = match[1] end end return_value ||= domain return_value ||= search return unless return_value return_value.gsub(/\.$/, '') end |
#hack_etc_hosts(hosts, _opts) ⇒ Object
Update /etc/hosts to make it possible for each provided host to reach each other host by name. Assumes that each provided host has host set; in the instance where a provider sets host to an address which facilitates access to the host externally, but the actual host addresses differ from this, we check first for the presence of a host key first, and use that if present.
307 308 309 310 311 312 313 314 315 316 317 318 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 307 def hack_etc_hosts hosts, _opts etc_hosts = "127.0.0.1\tlocalhost localhost.localdomain\n" hosts.each do |host| ip = host['vm_ip'] || host['ip'].to_s hostname = host[:vmhostname] || host.name domain = get_domain_name(host) etc_hosts += "#{ip}\t#{hostname}.#{domain} #{hostname}\n" end hosts.each do |host| set_etc_hosts(host, etc_hosts) end end |
#host_packages(host) ⇒ Array<String>
Return a list of packages that should be present.
121 122 123 124 125 126 127 128 129 130 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 121 def host_packages(host) packages = host['platform'].base_packages if host.is_cygwin? raise RuntimeError, "cygwin is not installed on #{host}" if !host.cygwin_installed? packages << 'curl' end packages += host['platform'].timesync_packages if host[:timesync] packages end |
#install_one_of_packages(host, packages) ⇒ Object
Installs one of alternative packages (first available)
149 150 151 152 153 154 155 156 157 158 159 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 149 def install_one_of_packages host, packages error = nil packages.each do |pkg| begin return host.install_package pkg rescue Beaker::Host::CommandFailure => e error = e end end raise error end |
#package_proxy(host, opts) ⇒ Object
Setup files for enabling requests to pass to a proxy server This works for the APT package manager on debian and ubuntu and YUM package manager on el, centos, fedora and redhat.
404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 404 def package_proxy host, opts logger = opts[:logger] block_on host do |host| logger.debug("enabling proxy support on #{host.name}") case host['platform'] when /ubuntu/, /debian/ host.exec(Command.new("echo 'Acquire::http::Proxy \"#{opts[:package_proxy]}/\";' >> /etc/apt/apt.conf.d/10proxy")) when /amazon/, /^el-/, /centos/, /fedora/, /redhat/ host.exec(Command.new("echo 'proxy=#{opts[:package_proxy]}/' >> /etc/yum.conf")) when /solaris-11/ host.exec(Command.new("/usr/bin/pkg unset-publisher solaris || :")) host.exec(Command.new("/usr/bin/pkg set-publisher -g %s solaris" % opts[:package_proxy])) else logger.debug("Attempting to enable package manager proxy support on non-supported platform: #{host.name}: #{host['platform']}") end end end |
#set_env(host, opts) ⇒ Object
Add a host specific set of env vars to each provided host's ~/.ssh/environment
463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 463 def set_env host, opts logger = opts[:logger] block_on host do |host| skip_msg = host.skip_set_env? unless skip_msg.nil? logger.debug(skip_msg) next end env = construct_env(host, opts) logger.debug("setting local environment on #{host.name}") env['CYGWIN'] = 'nodosfilewarning' if host['platform'].include?('windows') && host.is_cygwin? host.ssh_permit_user_environment host.ssh_set_user_environment(env) # close the host to re-establish the connection with the new sshd settings host.close # print out the working env if host.is_powershell? host.exec(Command.new("SET")) else host.exec(Command.new("cat #{host[:ssh_env_file]}")) end end end |
#set_etc_hosts(host, etc_hosts) ⇒ Object
Append the provided string to the /etc/hosts file of the provided host
245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 245 def set_etc_hosts(host, etc_hosts) if host['platform'].include?('freebsd') host.echo_to_file(etc_hosts, '/etc/hosts') elsif ((host['platform'].include?('windows')) and not host.is_cygwin?) host.exec(Command.new("echo '#{etc_hosts}' >> C:\\Windows\\System32\\drivers\\etc\\hosts")) else host.exec(Command.new("echo '#{etc_hosts}' >> /etc/hosts")) end # AIX must be configured to prefer local DNS over external return unless host['platform'].include?('aix') aix_netsvc = '/etc/netsvc.conf' aix_local_resolv = 'hosts = local, bind' return if host.exec(Command.new("grep '#{aix_local_resolv}' #{aix_netsvc}"), :accept_all_exit_codes => true).exit_code == 0 host.exec(Command.new("echo '#{aix_local_resolv}' >> #{aix_netsvc}")) end |
#sync_root_keys(host, opts) ⇒ Object
Install a set of authorized keys using ROOT_KEYS_SCRIPT. This is a convenience method to allow for easy login to hosts after they have been provisioned with Beaker.
167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 167 def sync_root_keys host, opts # JJM This step runs on every system under test right now. We're anticipating # issues on Windows and maybe Solaris. We will likely need to filter this step # but we're deliberately taking the approach of "assume it will work, fix it # when reality dictates otherwise" logger = opts[:logger] block_on host do |host| logger.notify "Sync root authorized_keys from github on #{host.name}" # Allow all exit code, as this operation is unlikely to cause problems if it fails. if host['platform'].include?('solaris') host.exec(Command.new(ROOT_KEYS_SYNC_CMD % "bash"), :accept_all_exit_codes => true) elsif host['platform'].include?('aix') host.exec(Command.new(ROOT_KEYS_SYNC_CMD_AIX % "env PATH=/usr/gnu/bin:$PATH bash"), :accept_all_exit_codes => true) else host.exec(Command.new(ROOT_KEYS_SYNC_CMD % "env PATH=\"/usr/gnu/bin:$PATH\" bash"), :accept_all_exit_codes => true) end end rescue => e report_and_raise(logger, e, "sync_root_keys") end |
#timesync(host, opts) ⇒ Object
Run timesync on the provided hosts
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 30 def timesync host, opts logger = opts[:logger] ntp_server = opts[:ntp_server] ? opts[:ntp_server] : NTPSERVER block_on host do |host| logger.notify "Update system time sync for '#{host.name}'" if host['platform'].include? 'windows' # The exit code of 5 is for Windows 2008 systems where the w32tm /register command # is not actually necessary. host.exec(Command.new("w32tm /register"), :acceptable_exit_codes => [0, 5]) host.exec(Command.new("net start w32time"), :acceptable_exit_codes => [0, 2]) host.exec(Command.new("w32tm /config /manualpeerlist:#{ntp_server} /syncfromflags:manual /update")) host.exec(Command.new("w32tm /resync")) logger.notify "NTP date succeeded on #{host}" else if host['platform'].uses_chrony? ntp_command = "chronyc add server #{ntp_server} prefer trust;chronyc makestep;chronyc burst 1/2" elsif /opensuse-|sles-/.match?(host['platform']) ntp_command = "sntp #{ntp_server}" else ntp_command = "ntpdate -u -t 20 #{ntp_server}" end success = false try = 0 until try >= TRIES try += 1 if host.exec(Command.new(ntp_command), :accept_all_exit_codes => true).exit_code == 0 success = true break end sleep SLEEPWAIT end raise "NTP date was not successful after #{try} tries" unless success logger.notify "NTP date succeeded on #{host} after #{try} tries" end end nil rescue => e report_and_raise(logger, e, "timesync (--ntp)") end |
#validate_host(host, opts) ⇒ Object
Validate that hosts are prepared to be used as SUTs, if packages are missing attempt to install them.
78 79 80 81 82 83 84 85 86 |
# File 'lib/beaker/host_prebuilt_steps.rb', line 78 def validate_host host, opts logger = opts[:logger] block_on host do |host| check_and_install_packages_if_needed(host, host_packages(host)) ensure_root_dotfiles(host) end rescue => e report_and_raise(logger, e, "validate") end |