Class: Beaker::GoogleComputeHelper
- Inherits:
-
Object
- Object
- Beaker::GoogleComputeHelper
- Defined in:
- lib/beaker/hypervisor/google_compute_helper.rb
Overview
Beaker helper module for doing API level Google Compute Engine interaction.
Defined Under Namespace
Classes: GoogleComputeError
Constant Summary collapse
- SLEEPWAIT =
5
- AUTH_URL =
'https://www.googleapis.com/auth/compute'
- API_VERSION =
'v1'
- BASE_URL =
"https://www.googleapis.com/compute/#{API_VERSION}/projects/"
- DEFAULT_ZONE_NAME =
'us-central1-a'
- DEFAULT_MACHINE_TYPE =
'e2-standard-4'
- DEFAULT_NETWORK_NAME =
'default'
- VALID_PROTOS =
['tcp', 'udp', 'icmp', 'esp', 'ah', 'ipip', 'sctp']
- GCP_AUTH_SCOPE =
[ Google::Apis::ComputeV1::AUTH_COMPUTE, Google::Apis::OsloginV1::AUTH_CLOUD_PLATFORM_READ_ONLY, ].freeze
Instance Method Summary collapse
-
#add_firewall_port(name, port, proto) ⇒ Google::Apis::ComputeV1::Operation
Add an allowed port to the firewall.
-
#add_firewall_source_range(name, range) ⇒ Google::Apis::ComputeV1::Operation
Add a source range to the firewall.
-
#add_firewall_source_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a source_tag to an existing firewall.
-
#add_firewall_target_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a taget_tag to an existing firewall.
-
#add_instance_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a tag to a Google Compute Instance.
-
#authenticate ⇒ Google::Auth::UserRefreshCredentials|Google::Auth::ServiceAccountCredentials
Creates an authentication object to use in the various Google APIs.
-
#create_disk(name, size, img = nil) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute disk.
-
#create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: []) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute firewall.
-
#create_instance(name, img, machine_type, disk_size, hostname) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute instance.
-
#default_network ⇒ String
Determines the default Google Compute network based upon defaults and options.
-
#default_network_from_subnet(subnetwork_name) ⇒ String
Infer the network that a given subnetwork is attached to.
-
#default_region ⇒ String
Get the region name from the provided zone.
-
#default_subnetwork ⇒ String
Determine the subnetwork to use for instances.
-
#default_zone ⇒ String
Determines the default Google Compute zone based upon options and defaults.
-
#delete_disk(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute disk.
-
#delete_firewall(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute firewall.
-
#delete_instance(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute instance.
-
#get_firewall(name) ⇒ Google::Apis::ComputeV1::Firewall
Get the named firewall.
-
#get_image(project, name) ⇒ Google::Apis::ComputeV1::Image
Find the correct image object for a given project and name.
-
#get_instance(name) ⇒ Google::Apis::ComputeV1::Instance
Get the named instace from Google Compute Image.
-
#get_latest_image_from_family(image_project, family) ⇒ Google::Apis::ComputeV1::Image
Find the latest non-deprecated image in the given project and family.
-
#get_machine_type(type_name = DEFAULT_MACHINE_TYPE) ⇒ Google::Apis::ComputeV1::MachineType
Determines the Google Compute machineType object based upon the selected gce_machine_type option.
-
#get_network(network_name = default_network) ⇒ Google::Apis::ComputeV1::Network
Determines the Google Compute network object in use for the current connection.
-
#initialize(options) ⇒ GoogleComputeHelper
constructor
Create a new instance of the Google Compute Engine helper object.
-
#list_disks ⇒ Array[Google::Apis::ComputeV1::Disk]
Determines a list of existing Google Compute disks.
-
#list_firewalls ⇒ Array[Google::Apis::ComputeV1::Firewall]
Determines a list of existing Google Compute firewalls.
-
#list_instances ⇒ Array[Google::Apis::ComputeV1::Instance]
Determines a list of existing Google Compute instances.
-
#set_client(version) ⇒ Object
Set the user-agent information for the application.
-
#set_metadata_on_instance(name, data) ⇒ Google::Apis::ComputeV1::Operation
Set key/value metadata pairs to a Google Compute instance.
-
#ssh_username ⇒ String
Find the username for ssh to use with this connection.
Constructor Details
#initialize(options) ⇒ GoogleComputeHelper
Create a new instance of the Google Compute Engine helper object
37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 37 def initialize() @options = @logger = [:logger] set_client(Beaker::Version::STRING) # ::Google::Apis.logger = ::Logger.new(::STDERR) # ::Google::Apis.logger.level = ::Logger::DEBUG # ::Google::Apis.logger.level = ::Logger::WARN @options[:gce_project] = ENV['BEAKER_gce_project'] if ENV['BEAKER_gce_project'] @options[:gce_zone] = ENV.fetch('BEAKER_gce_zone', DEFAULT_ZONE_NAME) @options[:gce_network] = ENV.fetch('BEAKER_gce_network', DEFAULT_NETWORK_NAME) @options[:gce_subnetwork] = ENV.fetch('BEAKER_gce_subnetwork', nil) @configure_ports = ENV.fetch('BEAKER_gce_ports', "").strip # Split the ports based on commas, removing any empty values @options[:gce_ports] = @configure_ports.split(/\s*?,\s*/).reject { |s| s.empty? } raise 'You must specify a gce_project for Google Compute Engine instances!' unless @options[:gce_project] @options[:gce_ports].each do |port| parts = port.split('/', 2) raise "Invalid format for port #{port}. Should be 'port/proto'" unless parts.length == 2 proto = parts[1] raise "Invalid value '#{proto}' for protocol in '#{port}'. Must be one of '#{VALID_PROTOS.join("', '")}'" unless VALID_PROTOS.include? proto end = authenticate @compute = ::Google::Apis::ComputeV1::ComputeService.new @compute. = # Find the appropriate username to log into created instances @cloudoslogin = Google::Apis::OsloginV1::CloudOSLoginService.new @cloudoslogin. = end |
Instance Method Details
#add_firewall_port(name, port, proto) ⇒ Google::Apis::ComputeV1::Operation
Add an allowed port to the firewall
407 408 409 410 411 412 413 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 407 def add_firewall_port(name, port, proto) firewall = get_firewall(name) firewall.allowed = [] if firewall.allowed.nil? firewall.allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: proto, ports: [port]) operation = @compute.patch_firewall(@options[:gce_project], name, firewall) @compute.wait_global_operation(@options[:gce_project], operation.name) end |
#add_firewall_source_range(name, range) ⇒ Google::Apis::ComputeV1::Operation
Add a source range to the firewall.
385 386 387 388 389 390 391 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 385 def add_firewall_source_range(name, range) firewall = get_firewall(name) firewall.source_ranges = [] if firewall.source_ranges.nil? firewall.source_ranges << range operation = @compute.patch_firewall(@options[:gce_project], name, firewall) @compute.wait_global_operation(@options[:gce_project], operation.name) end |
#add_firewall_source_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a source_tag to an existing firewall
447 448 449 450 451 452 453 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 447 def add_firewall_source_tag(name, tag) firewall = @compute.get_firewall(@options[:gce_project], name) firewall. = [] if firewall..nil? firewall. << tag operation = @compute.patch_firewall(@options[:gce_project], name, firewall) @compute.wait_global_operation(@options[:gce_project], operation.name) end |
#add_firewall_target_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a taget_tag to an existing firewall
427 428 429 430 431 432 433 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 427 def add_firewall_target_tag(name, tag) firewall = @compute.get_firewall(@options[:gce_project], name) firewall. = [] if firewall..nil? firewall. << tag operation = @compute.patch_firewall(@options[:gce_project], name, firewall) @compute.wait_global_operation(@options[:gce_project], operation.name) end |
#add_instance_tag(name, tag) ⇒ Google::Apis::ComputeV1::Operation
Add a tag to a Google Compute Instance
570 571 572 573 574 575 576 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 570 def add_instance_tag(name, tag) instance = get_instance(name) = instance. .items << tag operation = @compute.(@options[:gce_project], @options[:gce_zone], name, ) @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name) end |
#authenticate ⇒ Google::Auth::UserRefreshCredentials|Google::Auth::ServiceAccountCredentials
Creates an authentication object to use in the various Google APIs
This method currently supports using application credentials via the GOOGLE_APPLICATION_CREDENTIALS environment variable, and application default credentials.
185 186 187 188 189 190 191 192 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 185 def authenticate if ENV['GOOGLE_APPLICATION_CREDENTIALS'] ::Google::Auth::ServiceAccountCredentials.from_env(scope: GCP_AUTH_SCOPE) else # Fall back to default application auth ::Google::Auth.get_application_default(GCP_AUTH_SCOPE) end end |
#create_disk(name, size, img = nil) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute disk
468 469 470 471 472 473 474 475 476 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 468 def create_disk(name, size, img = nil) new_disk = ::Google::Apis::ComputeV1::Disk.new( name: name, size_gb: size, source_image: img, ) operation = @compute.insert_disk(@options[:gce_project], @options[:gce_zone], new_disk) @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name) end |
#create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: []) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute firewall
the firewall
or ‘source_tags’ is specified, GCP adds a default ‘source_range’ of ‘0.0.0.0/0’ (allow all)
or ‘source_tags’ is specified, GCP adds a default ‘source_range’ of ‘0.0.0.0/0’ (allow all)
or ‘target_tags’ is specified, the firewall applies to all hosts in the VPC
or ‘target_tags’ is specified, the firewall applies to all hosts in the VPC
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 324 def create_firewall(name, network, allow: [], deny: [], source_ranges: [], source_tags: [], target_ranges: [], target_tags: []) allowed = [] allow.each do |port| parts = port.split('/', 2) if parts[1] == 'tcp' || parts[1] == 'udp' || parts[1] == 'sctp' then allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1], ports: [parts[0]]) else allowed << ::Google::Apis::ComputeV1::Firewall::Allowed.new(ip_protocol: parts[1]) end end denied = [] deny.each do |port| parts = port.split('/', 2) if parts[1] == 'tcp' || parts[1] == 'udp' || parts[1] == 'sctp' then denied << ::Google::Apis::ComputeV1::Firewall::Denied.new(ip_protocol: parts[1], ports: [parts[0]]) else denied << ::Google::Apis::ComputeV1::Firewall::Denied.new(ip_protocol: parts[1]) end end firewall_object = ::Google::Apis::ComputeV1::Firewall.new( name: name, direction: 'INGRESS', network: network.self_link, allowed: allowed, denied: denied, source_ranges: source_ranges, source_tags: , target_ranges: target_ranges, target_tags: , ) operation = @compute.insert_firewall(@options[:gce_project], firewall_object) @compute.wait_global_operation(@options[:gce_project], operation.name) end |
#create_instance(name, img, machine_type, disk_size, hostname) ⇒ Google::Apis::ComputeV1::Operation
Create a Google Compute instance
497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 497 def create_instance(name, img, machine_type, disk_size, hostname) initialize_params = ::Google::Apis::ComputeV1::AttachedDiskInitializeParams.new( disk_size_gb: disk_size, source_image: img.self_link, ) disk_params = ::Google::Apis::ComputeV1::AttachedDisk.new( boot: true, auto_delete: true, initialize_params: initialize_params, ) # attached_network = ::Google::Apis::ComputeV1::networkInterfaces.new() = ::Google::Apis::ComputeV1::Tags.new( items: [name], ) network_interface = ::Google::Apis::ComputeV1::NetworkInterface.new( network: get_network(default_network).self_link, subnetwork: @compute.get_subnetwork(@options[:gce_project], default_region, default_subnetwork).self_link, # Create an AccessConfig to add a NAT IP to the host. # TODO: Make this configurable access_configs: [ ::Google::Apis::ComputeV1::AccessConfig.new( network_tier: 'STANDARD', ), ], ) instance_opts = { :machine_type => machine_type.self_link, :name => name, :disks => [disk_params], :network_interfaces => [network_interface], :tags => , } # use custom hostname if specified if hostname && ENV.fetch('BEAKER_set_gce_hostname', false) # The google api requires an FQDN for the custom hostname hostname.include?('.') ? valid_hostname = hostname : valid_hostname = hostname + '.beaker.test' instance_opts[:hostname] = valid_hostname end new_instance = ::Google::Apis::ComputeV1::Instance.new(instance_opts) operation = @compute.insert_instance(@options[:gce_project], @options[:gce_zone], new_instance) @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name) end |
#default_network ⇒ String
Determines the default Google Compute network based upon defaults and options
99 100 101 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 99 def default_network @options[:gce_network] end |
#default_network_from_subnet(subnetwork_name) ⇒ String
Infer the network that a given subnetwork is attached to
138 139 140 141 142 143 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 138 def default_network_from_subnet(subnetwork_name) subnetwork = @compute.get_subnetwork(@options[:gce_project], default_region, subnetwork_name) m = %r{.*/networks/(?<network_name>.*)\Z}.match subnetwork.network nil if m.nil? m['network_name'] end |
#default_region ⇒ String
Get the region name from the provided zone.
Assume that the region is the name of the zone without the final - and zone letter
90 91 92 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 90 def default_region @options[:gce_zone].split('-')[0..1].join('-') end |
#default_subnetwork ⇒ String
Determine the subnetwork to use for instances
If the network is the ‘default’ network, get the ‘default’ subnetwork for the region. If no subnet is provided by the user, pick the first one out of the user-provided network
152 153 154 155 156 157 158 159 160 161 162 163 164 165 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 152 def default_subnetwork network_name = @options[:gce_network] if network_name == 'default' @options[:gce_subnetwork] ||= @compute.get_subnetwork(@options[:gce_project], default_region, 'default').name elsif @options[:gce_subnetwork].nil? # No subnet set, get the first subnet in our current region for the network subnetwork = @compute.get_network(@options[:gce_project], network_name).subnetworks[0] m = %r{.*/subnetworks/(?<subnetwork_name>.*)\Z}.match subnetwork raise "Unable to find a subnetwork in provided network #{network_name}" if m.nil? @options[:gce_subnetwork] = m['subnetwork_name'] end @options[:gce_subnetwork] end |
#default_zone ⇒ String
Determines the default Google Compute zone based upon options and defaults
79 80 81 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 79 def default_zone @options[:gce_zone] end |
#delete_disk(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute disk
626 627 628 629 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 626 def delete_disk(name) operation = @compute.delete_disk(@options[:gce_project], default_zone, name) @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name) end |
#delete_firewall(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute firewall
641 642 643 644 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 641 def delete_firewall(name) operation = @compute.delete_firewall(@options[:gce_project], name) @compute.wait_global_operation(@options[:gce_project], operation.name) end |
#delete_instance(name) ⇒ Google::Apis::ComputeV1::Operation
Delete a Google Compute instance
611 612 613 614 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 611 def delete_instance(name) operation = @compute.delete_instance(@options[:gce_project], default_zone, name) @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name) end |
#get_firewall(name) ⇒ Google::Apis::ComputeV1::Firewall
Get the named firewall
369 370 371 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 369 def get_firewall(name) firewall = @compute.get_firewall(@options[:gce_project], name) end |
#get_image(project, name) ⇒ Google::Apis::ComputeV1::Image
Find the correct image object for a given project and name
207 208 209 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 207 def get_image(project, name) @compute.get_image(project, name) end |
#get_instance(name) ⇒ Google::Apis::ComputeV1::Instance
Get the named instace from Google Compute Image
554 555 556 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 554 def get_instance(name) @compute.get_instance(@options[:gce_project], @options[:gce_zone], name) end |
#get_latest_image_from_family(image_project, family) ⇒ Google::Apis::ComputeV1::Image
Find the latest non-deprecated image in the given project and family
223 224 225 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 223 def get_latest_image_from_family(image_project, family) @compute.get_image_from_family(image_project, family) end |
#get_machine_type(type_name = DEFAULT_MACHINE_TYPE) ⇒ Google::Apis::ComputeV1::MachineType
Determines the Google Compute machineType object based upon the selected gce_machine_type option
238 239 240 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 238 def get_machine_type(type_name = DEFAULT_MACHINE_TYPE) @compute.get_machine_type(@options[:gce_project], default_zone, type_name) end |
#get_network(network_name = default_network) ⇒ Google::Apis::ComputeV1::Network
Determines the Google Compute network object in use for the current connection
250 251 252 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 250 def get_network(network_name = default_network) @compute.get_network(@options[:gce_project], network_name) end |
#list_disks ⇒ Array[Google::Apis::ComputeV1::Disk]
Determines a list of existing Google Compute disks
compared to Time.now to determine how many further code execution attempts remain
278 279 280 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 278 def list_disks @compute.list_disks(@options[:gce_project], default_zone).items end |
#list_firewalls ⇒ Array[Google::Apis::ComputeV1::Firewall]
Determines a list of existing Google Compute firewalls
290 291 292 293 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 290 def list_firewalls @compute.list_firewalls(@options[:gce_project], filter: 'name != default-allow-internal AND name != default-ssh').items end |
#list_instances ⇒ Array[Google::Apis::ComputeV1::Instance]
Determines a list of existing Google Compute instances
262 263 264 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 262 def list_instances @compute.list_instances(@options[:gce_project], default_zone).items end |
#set_client(version) ⇒ Object
Set the user-agent information for the application.
171 172 173 174 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 171 def set_client(version) ::Google::Apis::ClientOptions.default.application_name = 'beaker-google' ::Google::Apis::ClientOptions.default.application_version = version end |
#set_metadata_on_instance(name, data) ⇒ Google::Apis::ComputeV1::Operation
Set key/value metadata pairs to a Google Compute instance
This function replaces any existing items in the metadata hash!
item should have a ‘key’ and ‘value’ key.
593 594 595 596 597 598 599 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 593 def (name, data) instance = @compute.get_instance(@options[:gce_project], @options[:gce_zone], name) mdata = instance..dup mdata.items = data operation = @compute.(@options[:gce_project], @options[:gce_zone], name, mdata) @compute.wait_zone_operation(@options[:gce_project], @options[:gce_zone], operation.name) end |
#ssh_username ⇒ String
Find the username for ssh to use with this connection
111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 |
# File 'lib/beaker/hypervisor/google_compute_helper.rb', line 111 def ssh_username = @compute. # This is a bit of a hack based on what I found in a user (default application credentials) # and a service account. There might be a better way of doing this. case .class.to_s when 'Google::Auth::UserRefreshCredentials' .refresh! userid = ::Google::Auth::IDTokens.verify_oidc(.id_token)['email'] when 'Google::Auth::ServiceAccountCredentials' userid = .issuer else raise 'Unknown type of credential' end userid = "users/#{userid}" unless userid.start_with? 'users/' @cloudoslogin.get_user_login_profile(userid).posix_accounts[0].username end |