Class: Beaker::GoogleComputeHelper
- Inherits:
-
Object
- Object
- Beaker::GoogleComputeHelper
- Defined in:
- lib/beaker/hypervisor/google_compute_helper.rb
Overview
Beaker helper module for doing API level Google Compute Engine interaction.
Defined Under Namespace
Classes: GoogleComputeError
Constant Summary collapse
- SLEEPWAIT =
5- AUTH_URL =
'https://www.googleapis.com/auth/compute'- API_VERSION =
'v1'- BASE_URL =
"https://www.googleapis.com/compute/#{API_VERSION}/projects/"- DEFAULT_ZONE_NAME =
'us-central1-a'- DEFAULT_MACHINE_TYPE =
'e2-standard-4'- DEFAULT_NETWORK_NAME =
'default'- VALID_PROTOS =
%w[tcp udp icmp esp ah ipip sctp].freeze
- GCP_AUTH_SCOPE =
[ ::Google::Apis::ComputeV1::AUTH_COMPUTE, ::Google::Apis::OsloginV1::AUTH_CLOUD_PLATFORM_READ_ONLY, ].freeze
Instance Method Summary collapse
-
#add_firewall_port(name, port, proto) ⇒ Google::Apis::ComputeV1::Operation
Add an allowed port to the firewall.
-
#add_firewall_source_range(name, range) ⇒ Google::Apis::ComputeV1::Operation
Add a source range to the firewall.
-
#add_firewall_source_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a source_tag to an existing firewall.
-
#add_firewall_target_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a taget_tag to an existing firewall.
-
#add_instance_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a tag to a Google Compute Instance.
-
#authenticate ⇒ Google::Auth::UserRefreshCredentials|Google::Auth::ServiceAccountCredentials
Creates an authentication object to use in the various Google APIs.
-
#client(version) ⇒ Object
Set the user-agent information for the application.
-
#create_disk(name, size, img = nil) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute disk.
-
#create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: []) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute firewall.
-
#create_instance(name, img, machine_type, disk_size, hostname) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute instance.
-
#default_network ⇒ String
Determines the default Google Compute network based upon defaults and options.
-
#default_network_from_subnet(subnetwork_name) ⇒ String
Infer the network that a given subnetwork is attached to.
-
#default_region ⇒ String
Get the region name from the provided zone.
-
#default_subnetwork ⇒ String
Determine the subnetwork to use for instances.
-
#default_zone ⇒ String
Determines the default Google Compute zone based upon options and defaults.
-
#delete_disk(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute disk.
-
#delete_firewall(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute firewall.
-
#delete_instance(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute instance.
-
#get_firewall(name) ⇒ Google::Apis::ComputeV1::Firewall
Get the named firewall.
-
#get_image(project, name) ⇒ Google::Apis::ComputeV1::Image
Find the correct image object for a given project and name.
-
#get_instance(name) ⇒ Google::Apis::ComputeV1::Instance
Get the named instace from Google Compute Image.
-
#get_latest_image_from_family(image_project, family) ⇒ Google::Apis::ComputeV1::Image
Find the latest non-deprecated image in the given project and family.
-
#get_machine_type(type_name = DEFAULT_MACHINE_TYPE) ⇒ Google::Apis::ComputeV1::MachineType
Determines the Google Compute machineType object based upon the selected gce_machine_type option.
-
#get_network(network_name = default_network) ⇒ Google::Apis::ComputeV1::Network
Determines the Google Compute network object in use for the current connection.
-
#initialize(options) ⇒ GoogleComputeHelper
constructor
Create a new instance of the Google Compute Engine helper object.
-
#list_disks ⇒ Array[Google::Apis::ComputeV1::Disk]
Determines a list of existing Google Compute disks.
-
#list_firewalls ⇒ Array[Google::Apis::ComputeV1::Firewall]
Determines a list of existing Google Compute firewalls.
-
#list_instances ⇒ Array[Google::Apis::ComputeV1::Instance]
Determines a list of existing Google Compute instances.
-
#set_metadata_on_instance(name, data) ⇒ Google::Apis::ComputeV1::Operation
Set key/value metadata pairs to a Google Compute instance.
-
#ssh_username ⇒ String
Find the username for ssh to use with this connection.
Constructor Details
#initialize(options) ⇒ GoogleComputeHelper
Create a new instance of the Google Compute Engine helper object
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 38 def initialize() = @logger = [:logger] client(Beaker::Version::STRING) # ::Google::Apis.logger = ::Logger.new(::STDERR) # ::Google::Apis.logger.level = ::Logger::DEBUG # ::Google::Apis.logger.level = ::Logger::WARN [:gce_project] = ENV['BEAKER_gce_project'] if ENV['BEAKER_gce_project'] [:gce_zone] = ENV.fetch('BEAKER_gce_zone', DEFAULT_ZONE_NAME) [:gce_network] = ENV.fetch('BEAKER_gce_network', DEFAULT_NETWORK_NAME) [:gce_subnetwork] = ENV.fetch('BEAKER_gce_subnetwork', nil) @configure_ports = ENV.fetch('BEAKER_gce_ports', '').strip # Split the ports based on commas, removing any empty values [:gce_ports] = @configure_ports.split(/\s*?,\s*/).reject(&:empty?) raise 'You must specify a gce_project for Google Compute Engine instances!' unless [:gce_project] [:gce_ports].each do |port| parts = port.split('/', 2) raise "Invalid format for port #{port}. Should be 'port/proto'" unless parts.length == 2 proto = parts[1] unless VALID_PROTOS.include? proto raise "Invalid value '#{proto}' for protocol in '#{port}'. Must be one of '#{VALID_PROTOS.join("', '")}'" end end = authenticate @compute = ::Google::Apis::ComputeV1::ComputeService.new @compute. = # Find the appropriate username to log into created instances @cloudoslogin = ::Google::Apis::OsloginV1::CloudOSLoginService.new @cloudoslogin. = end |
Instance Method Details
#add_firewall_port(name, port, proto) ⇒ Google::Apis::ComputeV1::Operation
Add an allowed port to the firewall
414 415 416 417 418 419 420 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 414 def add_firewall_port(name, port, proto) firewall = get_firewall(name) firewall.allowed = [] if firewall.allowed.nil? firewall.allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: proto, ports: [port]) operation = @compute.patch_firewall([:gce_project], name, firewall) @compute.wait_global_operation([:gce_project], operation.name) end |
#add_firewall_source_range(name, range) ⇒ Google::Apis::ComputeV1::Operation
Add a source range to the firewall.
392 393 394 395 396 397 398 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 392 def add_firewall_source_range(name, range) firewall = get_firewall(name) firewall.source_ranges = [] if firewall.source_ranges.nil? firewall.source_ranges << range operation = @compute.patch_firewall([:gce_project], name, firewall) @compute.wait_global_operation([:gce_project], operation.name) end |
#add_firewall_source_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a source_tag to an existing firewall
454 455 456 457 458 459 460 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 454 def add_firewall_source_tag(name, tag) firewall = @compute.get_firewall([:gce_project], name) firewall. = [] if firewall..nil? firewall. << tag operation = @compute.patch_firewall([:gce_project], name, firewall) @compute.wait_global_operation([:gce_project], operation.name) end |
#add_firewall_target_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a taget_tag to an existing firewall
434 435 436 437 438 439 440 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 434 def add_firewall_target_tag(name, tag) firewall = @compute.get_firewall([:gce_project], name) firewall. = [] if firewall..nil? firewall. << tag operation = @compute.patch_firewall([:gce_project], name, firewall) @compute.wait_global_operation([:gce_project], operation.name) end |
#add_instance_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a tag to a Google Compute Instance
577 578 579 580 581 582 583 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 577 def add_instance_tag(name, tag) instance = get_instance(name) = instance. .items << tag operation = @compute.([:gce_project], [:gce_zone], name, ) @compute.wait_zone_operation([:gce_project], [:gce_zone], operation.name) end |
#authenticate ⇒ Google::Auth::UserRefreshCredentials|Google::Auth::ServiceAccountCredentials
Creates an authentication object to use in the various Google APIs
This method currently supports using application credentials via the GOOGLE_APPLICATION_CREDENTIALS environment variable, and application default credentials.
189 190 191 192 193 194 195 196 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 189 def authenticate if ENV['GOOGLE_APPLICATION_CREDENTIALS'] ::Google::Auth::ServiceAccountCredentials.from_env(scope: GCP_AUTH_SCOPE) else # Fall back to default application auth ::Google::Auth.get_application_default(GCP_AUTH_SCOPE) end end |
#client(version) ⇒ Object
Set the user-agent information for the application.
175 176 177 178 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 175 def client(version) ::Google::Apis::ClientOptions.default.application_name = 'beaker-google' ::Google::Apis::ClientOptions.default.application_version = version end |
#create_disk(name, size, img = nil) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute disk
475 476 477 478 479 480 481 482 483 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 475 def create_disk(name, size, img = nil) new_disk = ::Google::Apis::ComputeV1::Disk.new( name: name, size_gb: size, source_image: img, ) operation = @compute.insert_disk([:gce_project], [:gce_zone], new_disk) @compute.wait_zone_operation([:gce_project], [:gce_zone], operation.name) end |
#create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: []) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute firewall
the firewall
specified, but not both.
specified, but not both.
‘source_ranges’ or ‘source_tags’ is specified, GCP adds a default ‘source_range’ of ‘0.0.0.0/0’ (allow all)
or ‘source_tags’ is specified, GCP adds a default ‘source_range’ of ‘0.0.0.0/0’ (allow all)
‘target_ranges’ or ‘target_tags’ is specified, the firewall applies to all hosts in the VPC
‘target_tags’ is specified, the firewall applies to all hosts in the VPC
330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 330 def create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: []) allowed = [] allow.each do |port| parts = port.split('/', 2) allowed << if parts[1] == 'tcp' || parts[1] == 'udp' || parts[1] == 'sctp' ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1], ports: [parts[0]]) else ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1]) end end denied = [] deny.each do |port| parts = port.split('/', 2) denied << if parts[1] == 'tcp' || parts[1] == 'udp' || parts[1] == 'sctp' ::Google::Apis::ComputeV1::Firewall::Denied.new(ip_protocol: parts[1], ports: [parts[0]]) else ::Google::Apis::ComputeV1::Firewall::Denied.new(ip_protocol: parts[1]) end end firewall_object = ::Google::Apis::ComputeV1::Firewall.new( name: name, direction: 'INGRESS', network: network.self_link, allowed: allowed, denied: denied, source_ranges: source_ranges, source_tags: , target_ranges: target_ranges, target_tags: , ) operation = @compute.insert_firewall([:gce_project], firewall_object) @compute.wait_global_operation([:gce_project], operation.name) end |
#create_instance(name, img, machine_type, disk_size, hostname) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute instance
504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 504 def create_instance(name, img, machine_type, disk_size, hostname) initialize_params = ::Google::Apis::ComputeV1::AttachedDiskInitializeParams.new( disk_size_gb: disk_size, source_image: img.self_link, ) disk_params = ::Google::Apis::ComputeV1::AttachedDisk.new( boot: true, auto_delete: true, initialize_params: initialize_params, ) # attached_network = ::Google::Apis::ComputeV1::networkInterfaces.new() = ::Google::Apis::ComputeV1::Tags.new( items: [name], ) network_interface = ::Google::Apis::ComputeV1::NetworkInterface.new( network: get_network(default_network).self_link, subnetwork: @compute.get_subnetwork([:gce_project], default_region, default_subnetwork).self_link, # Create an AccessConfig to add a NAT IP to the host. # TODO: Make this configurable access_configs: [ ::Google::Apis::ComputeV1::AccessConfig.new( network_tier: 'STANDARD', ), ], ) instance_opts = { machine_type: machine_type.self_link, name: name, disks: [disk_params], network_interfaces: [network_interface], tags: , } # use custom hostname if specified if hostname && ENV.fetch('BEAKER_set_gce_hostname', false) # The google api requires an FQDN for the custom hostname valid_hostname = hostname.include?('.') ? hostname : "#{hostname}.beaker.test" instance_opts[:hostname] = valid_hostname end new_instance = ::Google::Apis::ComputeV1::Instance.new(**instance_opts) operation = @compute.insert_instance([:gce_project], [:gce_zone], new_instance) @compute.wait_zone_operation([:gce_project], [:gce_zone], operation.name) end |
#default_network ⇒ String
Determines the default Google Compute network based upon defaults and options
103 104 105 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 103 def default_network [:gce_network] end |
#default_network_from_subnet(subnetwork_name) ⇒ String
Infer the network that a given subnetwork is attached to
142 143 144 145 146 147 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 142 def default_network_from_subnet(subnetwork_name) subnetwork = @compute.get_subnetwork([:gce_project], default_region, subnetwork_name) m = %r{.*/networks/(?<network_name>.*)\Z}.match subnetwork.network nil if m.nil? m['network_name'] end |
#default_region ⇒ String
Get the region name from the provided zone.
Assume that the region is the name of the zone without the final - and zone letter
94 95 96 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 94 def default_region [:gce_zone].split('-')[0..1].join('-') end |
#default_subnetwork ⇒ String
Determine the subnetwork to use for instances
If the network is the ‘default’ network, get the ‘default’ subnetwork for the region. If no subnet is provided by the user, pick the first one out of the user-provided network
156 157 158 159 160 161 162 163 164 165 166 167 168 169 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 156 def default_subnetwork network_name = [:gce_network] if network_name == 'default' [:gce_subnetwork] ||= @compute.get_subnetwork([:gce_project], default_region, 'default').name elsif [:gce_subnetwork].nil? # No subnet set, get the first subnet in our current region for the network subnetwork = @compute.get_network([:gce_project], network_name).subnetworks[0] m = %r{.*/subnetworks/(?<subnetwork_name>.*)\Z}.match subnetwork raise "Unable to find a subnetwork in provided network #{network_name}" if m.nil? [:gce_subnetwork] = m['subnetwork_name'] end [:gce_subnetwork] end |
#default_zone ⇒ String
Determines the default Google Compute zone based upon options and defaults
83 84 85 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 83 def default_zone [:gce_zone] end |
#delete_disk(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute disk
633 634 635 636 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 633 def delete_disk(name) operation = @compute.delete_disk([:gce_project], default_zone, name) @compute.wait_zone_operation([:gce_project], [:gce_zone], operation.name) end |
#delete_firewall(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute firewall
648 649 650 651 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 648 def delete_firewall(name) operation = @compute.delete_firewall([:gce_project], name) @compute.wait_global_operation([:gce_project], operation.name) end |
#delete_instance(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute instance
618 619 620 621 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 618 def delete_instance(name) operation = @compute.delete_instance([:gce_project], default_zone, name) @compute.wait_zone_operation([:gce_project], [:gce_zone], operation.name) end |
#get_firewall(name) ⇒ Google::Apis::ComputeV1::Firewall
Get the named firewall
376 377 378 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 376 def get_firewall(name) @compute.get_firewall([:gce_project], name) end |
#get_image(project, name) ⇒ Google::Apis::ComputeV1::Image
Find the correct image object for a given project and name
211 212 213 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 211 def get_image(project, name) @compute.get_image(project, name) end |
#get_instance(name) ⇒ Google::Apis::ComputeV1::Instance
Get the named instace from Google Compute Image
561 562 563 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 561 def get_instance(name) @compute.get_instance([:gce_project], [:gce_zone], name) end |
#get_latest_image_from_family(image_project, family) ⇒ Google::Apis::ComputeV1::Image
Find the latest non-deprecated image in the given project and family
227 228 229 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 227 def get_latest_image_from_family(image_project, family) @compute.get_image_from_family(image_project, family) end |
#get_machine_type(type_name = DEFAULT_MACHINE_TYPE) ⇒ Google::Apis::ComputeV1::MachineType
Determines the Google Compute machineType object based upon the selected gce_machine_type option
242 243 244 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 242 def get_machine_type(type_name = DEFAULT_MACHINE_TYPE) @compute.get_machine_type([:gce_project], default_zone, type_name) end |
#get_network(network_name = default_network) ⇒ Google::Apis::ComputeV1::Network
Determines the Google Compute network object in use for the current connection
254 255 256 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 254 def get_network(network_name = default_network) @compute.get_network([:gce_project], network_name) end |
#list_disks ⇒ Array[Google::Apis::ComputeV1::Disk]
Determines a list of existing Google Compute disks
compared to Time.now to determine how many further code execution attempts remain
282 283 284 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 282 def list_disks @compute.list_disks([:gce_project], default_zone).items end |
#list_firewalls ⇒ Array[Google::Apis::ComputeV1::Firewall]
Determines a list of existing Google Compute firewalls
294 295 296 297 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 294 def list_firewalls @compute.list_firewalls([:gce_project], filter: 'name != default-allow-internal AND name != default-ssh').items end |
#list_instances ⇒ Array[Google::Apis::ComputeV1::Instance]
Determines a list of existing Google Compute instances
266 267 268 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 266 def list_instances @compute.list_instances([:gce_project], default_zone).items end |
#set_metadata_on_instance(name, data) ⇒ Google::Apis::ComputeV1::Operation
Set key/value metadata pairs to a Google Compute instance
This function replaces any existing items in the metadata hash!
item should have a ‘key’ and ‘value’ key.
600 601 602 603 604 605 606 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 600 def (name, data) instance = @compute.get_instance([:gce_project], [:gce_zone], name) mdata = instance..dup mdata.items = data operation = @compute.([:gce_project], [:gce_zone], name, mdata) @compute.wait_zone_operation([:gce_project], [:gce_zone], operation.name) end |
#ssh_username ⇒ String
Find the username for ssh to use with this connection
115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 115 def ssh_username = @compute. # This is a bit of a hack based on what I found in a user (default application credentials) # and a service account. There might be a better way of doing this. case .class.to_s when 'Google::Auth::UserRefreshCredentials' .refresh! userid = ::Google::Auth::IDTokens.verify_oidc(.id_token)['email'] when 'Google::Auth::ServiceAccountCredentials' userid = .issuer else raise 'Unknown type of credential' end userid = "users/#{userid}" unless userid.start_with? 'users/' @cloudoslogin.get_user_login_profile(userid).posix_accounts[0].username end |