Class: Beaker::OpenStack
- Inherits:
-
Hypervisor
- Object
- Hypervisor
- Beaker::OpenStack
- Defined in:
- lib/beaker/hypervisor/openstack.rb
Overview
Beaker support for OpenStack This code is EXPERIMENTAL! Please file any issues/concerns at github.com/puppetlabs/beaker/issues
Constant Summary collapse
- SLEEPWAIT =
5
Constants inherited from Hypervisor
Constants included from HostPrebuiltSteps
HostPrebuiltSteps::APT_CFG, HostPrebuiltSteps::ARCHLINUX_PACKAGES, HostPrebuiltSteps::CUMULUS_PACKAGES, HostPrebuiltSteps::DEBIAN_PACKAGES, HostPrebuiltSteps::ETC_HOSTS_PATH, HostPrebuiltSteps::ETC_HOSTS_PATH_SOLARIS, HostPrebuiltSteps::FREEBSD_PACKAGES, HostPrebuiltSteps::IPS_PKG_REPO, HostPrebuiltSteps::NTPSERVER, HostPrebuiltSteps::OPENBSD_PACKAGES, HostPrebuiltSteps::PSWINDOWS_PACKAGES, HostPrebuiltSteps::ROOT_KEYS_SCRIPT, HostPrebuiltSteps::ROOT_KEYS_SYNC_CMD, HostPrebuiltSteps::ROOT_KEYS_SYNC_CMD_AIX, HostPrebuiltSteps::SLES10_PACKAGES, HostPrebuiltSteps::SLES_PACKAGES, HostPrebuiltSteps::SOLARIS10_PACKAGES, HostPrebuiltSteps::SOLARIS11_PACKAGES, HostPrebuiltSteps::TRIES, HostPrebuiltSteps::UNIX_PACKAGES, HostPrebuiltSteps::WINDOWS_PACKAGES
Instance Method Summary collapse
-
#cleanup ⇒ Object
Destroy any OpenStack instances.
-
#cleanup_storage(vm) ⇒ Object
Detach and delete guest volumes.
-
#create_or_associate_keypair(host, keyname) ⇒ Object
private
Get key_name from options or generate a new rsa key and add it to OpenStack keypairs.
-
#enable_root(host) ⇒ Object
enable root on a single host (the current one presumably) but only if the username isn’t ‘root’.
-
#enable_root_on_hosts ⇒ void
private
Enables root access for a host when username is not root This method ripped from the aws_sdk implementation and is probably wrong because it iterates on a collection when there’s no guarantee the collection has all been brought up in openstack yet and will thus explode.
-
#flavor(f) ⇒ String
Provided a flavor name return the OpenStack id for that flavor.
-
#get_ip ⇒ Object
Get a floating IP address to associate with the instance, try to allocate a new one from the specified pool if none are available.
-
#get_volume_api_version ⇒ Object
Get the API version.
-
#get_volumes(host) ⇒ Object
Get a hash of volumes from the host.
-
#image(i) ⇒ String
Provided an image name return the OpenStack id for that image.
-
#initialize(openstack_hosts, options) ⇒ OpenStack
constructor
Create a new instance of the OpenStack hypervisor object.
-
#network(n) ⇒ String
Provided a network name return the OpenStack id for that network.
-
#provision ⇒ Object
Create new instances in OpenStack.
-
#provision_storage(host, vm) ⇒ Object
Create and attach dynamic volumes.
-
#security_groups(sgs) ⇒ Array
Provided an array of security groups return that array if all security groups are present.
-
#volume_client_create ⇒ Fog::OpenStack::Volume
Create a volume client on request.
Methods inherited from Hypervisor
#configure, create, #generate_host_name, #proxy_package_manager, #validate
Methods included from HostPrebuiltSteps
#add_el_extras, #additive_hash_merge, #apt_get_update, #check_and_install_packages_if_needed, #construct_env, #copy_file_to_remote, #copy_ssh_to_root, #disable_iptables, #disable_se_linux, #disable_updates, #enable_root_login, #get_domain_name, #hack_etc_hosts, #install_one_of_packages, #package_proxy, #proxy_config, #set_env, #set_etc_hosts, #sync_root_keys, #timesync, #validate_host
Methods included from DSL::Patterns
Constructor Details
#initialize(openstack_hosts, options) ⇒ OpenStack
Create a new instance of the OpenStack hypervisor object
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
# File 'lib/beaker/hypervisor/openstack.rb', line 25 def initialize(openstack_hosts, ) require 'fog' @options = @logger = [:logger] @hosts = openstack_hosts @vms = [] raise 'You must specify an Openstack API key (:openstack_api_key) for OpenStack instances!' unless @options[:openstack_api_key] raise 'You must specify an Openstack username (:openstack_username) for OpenStack instances!' unless @options[:openstack_username] raise 'You must specify an Openstack auth URL (:openstack_auth_url) for OpenStack instances!' unless @options[:openstack_auth_url] raise 'You must specify an Openstack tenant (:openstack_tenant) for OpenStack instances!' unless @options[:openstack_tenant] raise 'You must specify an Openstack network (:openstack_network) for OpenStack instances!' unless @options[:openstack_network] # Common keystone authentication credentials @credentials = { :provider => :openstack, :openstack_auth_url => @options[:openstack_auth_url], :openstack_api_key => @options[:openstack_api_key], :openstack_username => @options[:openstack_username], :openstack_tenant => @options[:openstack_tenant], :openstack_region => @options[:openstack_region], } # Keystone version 3 requires users and projects to be scoped if @credentials[:openstack_auth_url].include?('/v3/') @credentials[:openstack_user_domain] = @options[:openstack_user_domain] || 'Default' @credentials[:openstack_project_domain] = @options[:openstack_project_domain] || 'Default' end @compute_client ||= Fog::Compute.new(@credentials) if not @compute_client raise "Unable to create OpenStack Compute instance (api key: #{@options[:openstack_api_key]}, username: #{@options[:openstack_username]}, auth_url: #{@options[:openstack_auth_url]}, tenant: #{@options[:openstack_tenant]})" end @network_client ||= Fog::Network.new(@credentials) if not @network_client raise "Unable to create OpenStack Network instance (api_key: #{@options[:openstack_api_key]}, username: #{@options[:openstack_username]}, auth_url: #{@options[:openstack_auth_url]}, tenant: #{@options[:openstack_tenant]})" end # Validate openstack_volume_support setting value, reset to boolean if passed via ENV value string @options[:openstack_volume_support] = true if @options[:openstack_volume_support].to_s.match(/\btrue\b/i) @options[:openstack_volume_support] = false if @options[:openstack_volume_support].to_s.match(/\bfalse\b/i) [true,false].include? @options[:openstack_volume_support] or raise "Invalid openstack_volume_support setting, current: @options[:openstack_volume_support]" end |
Instance Method Details
#cleanup ⇒ Object
Destroy any OpenStack instances
282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 |
# File 'lib/beaker/hypervisor/openstack.rb', line 282 def cleanup @logger.notify "Cleaning up OpenStack" @vms.each do |vm| cleanup_storage(vm) if @options[:openstack_volume_support] @logger.debug "Release floating IPs for OpenStack host #{vm.name}" floating_ips = vm.all_addresses # fetch and release its floating IPs floating_ips.each do |address| @compute_client.disassociate_address(vm.id, address['ip']) @compute_client.release_address(address['id']) end @logger.debug "Destroying OpenStack host #{vm.name}" vm.destroy if @options[:openstack_keyname].nil? @logger.debug "Deleting random keypair" @compute_client.delete_key_pair vm.key_name end end end |
#cleanup_storage(vm) ⇒ Object
Detach and delete guest volumes
189 190 191 192 193 194 195 196 |
# File 'lib/beaker/hypervisor/openstack.rb', line 189 def cleanup_storage vm vm.volumes.each do |vol| @logger.debug "Deleting volume #{vol.name} for OpenStack host #{vm.name}" vm.detach_volume(vol.id) vol.wait_for { ready? } vol.destroy end end |
#create_or_associate_keypair(host, keyname) ⇒ Object
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
Get key_name from options or generate a new rsa key and add it to OpenStack keypairs
329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 |
# File 'lib/beaker/hypervisor/openstack.rb', line 329 def create_or_associate_keypair(host, keyname) if @options[:openstack_keyname] @logger.debug "Adding optional key_name #{@options[:openstack_keyname]} to #{host.name} (#{host[:vmhostname]})" keyname = @options[:openstack_keyname] else @logger.debug "Generate a new rsa key" # There is apparently an error that can occur when generating RSA keys, probably # due to some timing issue, probably similar to the issue described here: # https://github.com/negativecode/vines/issues/34 # In order to mitigate this error, we will simply try again up to three times, and # then fail if we continue to error out. begin retries ||= 0 key = OpenSSL::PKey::RSA.new 2048 rescue OpenSSL::PKey::RSAError => e retries += 1 if retries > 2 @logger.notify "error generating RSA key #{retries} times, exiting" raise e end retry end type = key.ssh_type data = [ key.to_blob ].pack('m0') @logger.debug "Creating Openstack keypair '#{keyname}' for public key '#{type} #{data}'" @compute_client.create_key_pair keyname, "#{type} #{data}" host['ssh'][:key_data] = [ key.to_pem ] end host[:keyname] = keyname end |
#enable_root(host) ⇒ Object
enable root on a single host (the current one presumably) but only if the username isn’t ‘root’
315 316 317 318 319 320 321 322 |
# File 'lib/beaker/hypervisor/openstack.rb', line 315 def enable_root(host) if host['user'] != 'root' copy_ssh_to_root(host, @options) enable_root_login(host, @options) host['user'] = 'root' host.close end end |
#enable_root_on_hosts ⇒ void
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
This method returns an undefined value.
Enables root access for a host when username is not root This method ripped from the aws_sdk implementation and is probably wrong because it iterates on a collection when there’s no guarantee the collection has all been brought up in openstack yet and will thus explode
307 308 309 310 311 |
# File 'lib/beaker/hypervisor/openstack.rb', line 307 def enable_root_on_hosts @hosts.each do |host| enable_root(host) end end |
#flavor(f) ⇒ String
Provided a flavor name return the OpenStack id for that flavor
76 77 78 79 |
# File 'lib/beaker/hypervisor/openstack.rb', line 76 def flavor f @logger.debug "OpenStack: Looking up flavor '#{f}'" @compute_client.flavors.find { |x| x.name == f } || raise("Couldn't find flavor: #{f}") end |
#get_ip ⇒ Object
Get a floating IP address to associate with the instance, try to allocate a new one from the specified pool if none are available
200 201 202 203 204 205 206 207 208 209 210 211 212 |
# File 'lib/beaker/hypervisor/openstack.rb', line 200 def get_ip begin @logger.debug "Creating IP" ip = @compute_client.addresses.create rescue Fog::Compute::OpenStack::NotFound # If there are no more floating IP addresses, allocate a # new one and try again. @compute_client.allocate_address(@options[:floating_ip_pool]) ip = @compute_client.addresses.find { |ip| ip.instance_id.nil? } end raise 'Could not find or allocate an address' if not ip ip end |
#get_volume_api_version ⇒ Object
Get the API version
129 130 131 132 133 134 135 136 |
# File 'lib/beaker/hypervisor/openstack.rb', line 129 def get_volume_api_version case @volume_client when Fog::Volume::OpenStack::V1 1 else -1 end end |
#get_volumes(host) ⇒ Object
Get a hash of volumes from the host
123 124 125 126 |
# File 'lib/beaker/hypervisor/openstack.rb', line 123 def get_volumes host return host['volumes'] if host['volumes'] {} end |
#image(i) ⇒ String
Provided an image name return the OpenStack id for that image
84 85 86 87 |
# File 'lib/beaker/hypervisor/openstack.rb', line 84 def image i @logger.debug "OpenStack: Looking up image '#{i}'" @compute_client.images.find { |x| x.name == i } || raise("Couldn't find image: #{i}") end |
#network(n) ⇒ String
Provided a network name return the OpenStack id for that network
92 93 94 95 |
# File 'lib/beaker/hypervisor/openstack.rb', line 92 def network n @logger.debug "OpenStack: Looking up network '#{n}'" @network_client.networks.find { |x| x.name == n } || raise("Couldn't find network: #{n}") end |
#provision ⇒ Object
Create new instances in OpenStack
215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 |
# File 'lib/beaker/hypervisor/openstack.rb', line 215 def provision @logger.notify "Provisioning OpenStack" @hosts.each do |host| ip = get_ip hostname = ip.ip.gsub('.','-') host[:vmhostname] = hostname + '.rfc1918.puppetlabs.net' create_or_associate_keypair(host, hostname) @logger.debug "Provisioning #{host.name} (#{host[:vmhostname]})" = { :flavor_ref => flavor(host[:flavor]).id, :image_ref => image(host[:image]).id, :nics => [ {'net_id' => network(@options[:openstack_network]).id } ], :name => host[:vmhostname], :hostname => host[:vmhostname], :user_data => host[:user_data] || "#cloud-config\nmanage_etc_hosts: true\n", :key_name => host[:keyname], } [:security_groups] = security_groups(@options[:security_group]) unless @options[:security_group].nil? vm = @compute_client.servers.create() #wait for the new instance to start up try = 1 attempts = @options[:timeout].to_i / SLEEPWAIT while try <= attempts begin vm.wait_for(5) { ready? } break rescue Fog::Errors::TimeoutError => e if try >= attempts @logger.debug "Failed to connect to new OpenStack instance #{host.name} (#{host[:vmhostname]})" raise e end @logger.debug "Timeout connecting to instance #{host.name} (#{host[:vmhostname]}), trying again..." end sleep SLEEPWAIT try += 1 end # Associate a public IP to the server ip.server = vm host[:ip] = ip.ip @logger.debug "OpenStack host #{host.name} (#{host[:vmhostname]}) assigned ip: #{host[:ip]}" #set metadata vm..update({:jenkins_build_url => @options[:jenkins_build_url].to_s, :department => @options[:department].to_s, :project => @options[:project].to_s }) @vms << vm # Wait for the host to accept ssh logins host.wait_for_port(22) #enable root if user is not root enable_root(host) provision_storage(host, vm) if @options[:openstack_volume_support] @logger.notify "OpenStack Volume Support Disabled, can't provision volumes" if not @options[:openstack_volume_support] end hack_etc_hosts @hosts, @options end |
#provision_storage(host, vm) ⇒ Object
Create and attach dynamic volumes
Creates an array of volumes and attaches them to the current host. The host bus type is determined by the image type, so by default devices appear as /dev/vdb, /dev/vdc etc. Setting the glance properties hw_disk_bus=scsi, hw_scsi_model=virtio-scsi will present them as /dev/sdb, /dev/sdc (or 2:0:0:1, 2:0:0:2 in SCSI addresses)
148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 |
# File 'lib/beaker/hypervisor/openstack.rb', line 148 def provision_storage host, vm volumes = get_volumes(host) if !volumes.empty? # Lazily create the volume client if needed volume_client_create volumes.keys.each_with_index do |volume, index| @logger.debug "Creating volume #{volume} for OpenStack host #{host.name}" # The node defintion file defines volume sizes in MB (due to precedent # with the vagrant virtualbox implementation) however OpenStack requires # this translating into GB openstack_size = volumes[volume]['size'].to_i / 1000 # Set up the volume creation arguments args = { :size => openstack_size, :description => "Beaker volume: host=#{host.name} volume=#{volume}", } # Between version 1 and subsequent versions the API was updated to # rename 'display_name' to just 'name' for better consistency if get_volume_api_version == 1 args[:display_name] = volume else args[:name] = volume end # Create the volume and wait for it to become available vol = @volume_client.volumes.create(**args) vol.wait_for { ready? } # Fog needs a device name to attach as, so invent one. The guest # doesn't pay any attention to this device = "/dev/vd#{('b'.ord + index).chr}" vm.attach_volume(vol.id, device) end end end |
#security_groups(sgs) ⇒ Array
Provided an array of security groups return that array if all security groups are present
101 102 103 104 105 106 107 |
# File 'lib/beaker/hypervisor/openstack.rb', line 101 def security_groups sgs for sg in sgs @logger.debug "Openstack: Looking up security group '#{sg}'" @compute_client.security_groups.find { |x| x.name == sg } || raise("Couldn't find security group: #{sg}") sgs end end |
#volume_client_create ⇒ Fog::OpenStack::Volume
Create a volume client on request
111 112 113 114 115 116 117 118 119 120 |
# File 'lib/beaker/hypervisor/openstack.rb', line 111 def volume_client_create @volume_client ||= Fog::Volume.new(@credentials) unless @volume_client raise "Unable to create OpenStack Volume instance"\ " (api_key: #{@options[:openstack_api_key]},"\ " username: #{@options[:openstack_username]},"\ " auth_url: #{@options[:openstack_auth_url]},"\ " tenant: #{@options[:openstack_tenant]})" end end |