Class: Beaker::AwsSdk

Inherits:
Hypervisor show all
Defined in:
lib/beaker/hypervisor/aws_sdk.rb

Overview

This is an alternate EC2 driver that implements direct API access using Amazon’s AWS-SDK library: / SDK For Ruby

It is built for full control, to reduce any other layers beyond the pure vendor API.

Constant Summary collapse

ZOMBIE =

anything older than 3 hours is considered a zombie

3
PING_SECURITY_GROUP_NAME = 
'beaker-ping'

Constants inherited from Hypervisor

Hypervisor::CHARMAP

Constants included from HostPrebuiltSteps

HostPrebuiltSteps::APT_CFG, HostPrebuiltSteps::CUMULUS_PACKAGES, HostPrebuiltSteps::DEBIAN_PACKAGES, HostPrebuiltSteps::ETC_HOSTS_PATH, HostPrebuiltSteps::ETC_HOSTS_PATH_SOLARIS, HostPrebuiltSteps::FREEBSD_PACKAGES, HostPrebuiltSteps::IPS_PKG_REPO, HostPrebuiltSteps::NTPSERVER, HostPrebuiltSteps::OPENBSD_PACKAGES, HostPrebuiltSteps::PSWINDOWS_PACKAGES, HostPrebuiltSteps::ROOT_KEYS_SCRIPT, HostPrebuiltSteps::ROOT_KEYS_SYNC_CMD, HostPrebuiltSteps::ROOT_KEYS_SYNC_CMD_AIX, HostPrebuiltSteps::SLEEPWAIT, HostPrebuiltSteps::SLES10_PACKAGES, HostPrebuiltSteps::SLES_PACKAGES, HostPrebuiltSteps::SOLARIS10_PACKAGES, HostPrebuiltSteps::TRIES, HostPrebuiltSteps::UNIX_PACKAGES, HostPrebuiltSteps::WINDOWS_PACKAGES

Instance Method Summary collapse

Methods inherited from Hypervisor

#configure, create, #generate_host_name, #proxy_package_manager, #validate

Methods included from HostPrebuiltSteps

#add_el_extras, #additive_hash_merge, #apt_get_update, #check_and_install_packages_if_needed, #construct_env, #copy_file_to_remote, #copy_ssh_to_root, #disable_iptables, #disable_se_linux, #disable_updates, #enable_root_login, #epel_info_for, #get_domain_name, #get_ip, #hack_etc_hosts, #package_proxy, #proxy_config, #set_env, #set_etc_hosts, #sync_root_keys, #timesync, #validate_host

Methods included from DSL::Patterns

#block_on

Constructor Details

#initialize(hosts, options) ⇒ AwsSdk

Initialize AwsSdk hypervisor driver

Parameters:

  • hosts (Array<Beaker::Host>) —

    Array of Beaker::Host objects

  • options (Hash<String, String>) —

    Options hash



20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 20

def initialize(hosts, options)
  @hosts = hosts
  @options = options
  @logger = options[:logger]

  # Get AWS credentials
  creds = load_credentials()

  config = {
    :access_key_id => creds[:access_key],
    :secret_access_key => creds[:secret_key],
    :logger => Logger.new($stdout),
    :log_level => :debug,
    :log_formatter => AWS::Core::LogFormatter.colored,
    :max_retries => 12,
  }
  AWS.config(config)

  @ec2 = AWS::EC2.new()
end

Instance Method Details

#add_tags ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Add metadata tags to all instances



494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 494

def add_tags
  @hosts.each do |host|
    instance = host['instance']

    # Define tags for the instance
    @logger.notify("aws-sdk: Add tags for #{host.name}")
    instance.add_tag("jenkins_build_url", :value => @options[:jenkins_build_url])
    instance.add_tag("Name", :value => host.name)
    instance.add_tag("department", :value => @options[:department])
    instance.add_tag("project", :value => @options[:project])
    instance.add_tag("created_by", :value => @options[:created_by])

    host[:host_tags].each do |name, val|
      instance.add_tag(name.to_s, :value => val)
    end
  end

  nil
end

#backoff_sleep(tries) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Calculates and waits a back-off period based on the number of tries

Logs each backupoff time and retry value to the console.

Parameters:

  • tries (Number) —

    number of tries to calculate back-off period



657
658
659
660
661
662
663
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 657

def backoff_sleep(tries)
  # Exponential with some randomization
  sleep_time = 2 ** tries
  @logger.notify("aws-sdk: Sleeping #{sleep_time} seconds for attempt #{tries}.")
  sleep sleep_time
  nil
end

#cleanup ⇒ void

This method returns an undefined value.

Cleanup all earlier provisioned hosts on EC2 using the AWS::EC2 library

It goes without saying, but a #cleanup does nothing without a #provision method call first.



92
93
94
95
96
97
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 92

def cleanup
  # Provisioning should have set the host 'instance' values.
  kill_instances(@hosts.map{|h| h['instance']}.select{|x| !x.nil?})
  delete_key_pair_all_regions()
  nil
end

#configure_hosts ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

f5 hosts are skipped since this isn’t a valid step there

This method returns an undefined value.

Configure /etc/hosts for each node



551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 551

def configure_hosts
  non_netdev_hosts = @hosts.select{ |h| !(h['platform'] =~ /f5-|netscaler/) }
  non_netdev_hosts.each do |host|
    host_entries = non_netdev_hosts.map do |h|
      h == host ? etc_hosts_entry(h, :private_ip) : etc_hosts_entry(h)
    end
    host_entries.unshift "127.0.0.1\tlocalhost localhost.localdomain\n"
    set_etc_hosts(host, host_entries.join(''))
  end
  nil
end

#create_group(rv, ports) ⇒ AWS::EC2::SecurityGroup

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create a new security group

Accepts a region or VPC for group creation.

Parameters:

  • rv (AWS::EC2::Region, AWS::EC2::VPC) —

    the AWS region or vpc control object

  • ports (Array<Number>) —

    an array of port numbers

Returns:

  • (AWS::EC2::SecurityGroup) —

    created security group



879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 879

def create_group(rv, ports)
  name = group_id(ports)
  @logger.notify("aws-sdk: Creating group #{name} for ports #{ports.to_s}")
  group = rv.security_groups.create(name,
                                    :description => "Custom Beaker security group for #{ports.to_a}")

  unless ports.is_a? Set
    ports = Set.new(ports)
  end

  ports.each do |port|
    group.authorize_ingress(:tcp, port)
  end

  group
end

#create_instance(host, ami_spec, subnet_id) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Create an EC2 instance for host, tag it, and return it.



233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 233

def create_instance(host, ami_spec, subnet_id)
  amitype = host['vmname'] || host['platform']
  amisize = host['amisize'] || 'm1.small'
  vpc_id = host['vpc_id'] || @options['vpc_id'] || nil

  if vpc_id and !subnet_id
    raise RuntimeError, "A subnet_id must be provided with a vpc_id"
  end

  # Use snapshot provided for this host
  image_type = host['snapshot']
  if not image_type
    raise RuntimeError, "No snapshot/image_type provided for EC2 provisioning"
  end
  ami = ami_spec[amitype]
  ami_region = ami[:region]

  # Main region object for ec2 operations
  region = @ec2.regions[ami_region]

  # If we haven't defined a vpc_id then we use the default vpc for the provided region
  if !vpc_id
    @logger.notify("aws-sdk: filtering available vpcs in region by 'isDefault")
    filtered_vpcs = region.client.describe_vpcs(:filters => [{:name => 'isDefault', :values => ['true']}])
    if !filtered_vpcs[:vpc_set].empty?
      vpc_id = filtered_vpcs[:vpc_set].first[:vpc_id]
    else #there's no default vpc, use nil
      vpc_id = nil
    end
  end

  # Grab the vpc object based upon provided id
  vpc = vpc_id ? region.vpcs[vpc_id] : nil

  # Grab image object
  image_id = ami[:image][image_type.to_sym]
  @logger.notify("aws-sdk: Checking image #{image_id} exists and getting its root device")
  image = region.images[image_id]
  if image.nil? and not image.exists?
    raise RuntimeError, "Image not found: #{image_id}"
  end

  @logger.notify("Image Storage Type: #{image.root_device_type}")

  # Transform the images block_device_mappings output into a format
  # ready for a create.
  block_device_mappings = []
  if image.root_device_type == :ebs
    orig_bdm = image.block_device_mappings()
    @logger.notify("aws-sdk: Image block_device_mappings: #{orig_bdm.to_hash}")
    orig_bdm.each do |device_name, rest|
      block_device_mappings << {
        :device_name => device_name,
        :ebs => {
          # Change the default size of the root volume.
          :volume_size => host['volume_size'] || rest[:volume_size],
          # This is required to override the images default for
          # delete_on_termination, forcing all volumes to be deleted once the
          # instance is terminated.
          :delete_on_termination => true,
        }
      }
    end
  end

  security_group = ensure_group(vpc || region, Beaker::EC2Helper.amiports(host))
  #check if ping is enabled
  ping_security_group = ensure_ping_group(vpc || region)

  msg = "aws-sdk: launching %p on %p using %p/%p%s" %
        [host.name, amitype, amisize, image_type,
         subnet_id ? ("in %p" % subnet_id) : '']
  @logger.notify(msg)
  config = {
    :count => 1,
    :image_id => image_id,
    :monitoring_enabled => true,
    :key_pair => ensure_key_pair(region),
    :security_groups => [security_group, ping_security_group],
    :instance_type => amisize,
    :disable_api_termination => false,
    :instance_initiated_shutdown_behavior => "terminate",
    :subnet => subnet_id,
  }
  config[:block_device_mappings] = block_device_mappings if image.root_device_type == :ebs
  region.instances.create(config)
end

#create_new_key_pair(region, pair_name) ⇒ AWS::EC2::KeyPair

Create a new key pair for a given Beaker run

Parameters:

  • region (AWS::EC2::Region) —

    the region the key pair will be imported into

  • pair_name (String) —

    the name of the key to be created

Returns:

  • (AWS::EC2::KeyPair) —

    key pair created



790
791
792
793
794
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 790

def create_new_key_pair(region, pair_name)
  @logger.debug("aws-sdk: generating new key pair: #{pair_name}")
  ssh_string = public_key()
  region.key_pairs.import(pair_name, ssh_string)
end

#create_ping_group(rv) ⇒ AWS::EC2::SecurityGroup

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Create a new ping enabled security group

Accepts a region or VPC for group creation.

Parameters:

  • rv (AWS::EC2::Region, AWS::EC2::VPC) —

    the AWS region or vpc control object

Returns:

  • (AWS::EC2::SecurityGroup) —

    created security group



861
862
863
864
865
866
867
868
869
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 861

def create_ping_group(rv)
  @logger.notify("aws-sdk: Creating group #{PING_SECURITY_GROUP_NAME}")
  group = rv.security_groups.create(PING_SECURITY_GROUP_NAME,
                                    :description => "Custom Beaker security group to enable ping")

  group.allow_ping

  group
end

#delete_key_pair(region, pair_name) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Deletes a given key pair

Parameters:

  • region (AWS::EC2::Region) —

    the region the key belongs to

  • pair_name (String) —

    the name of the key to be deleted



776
777
778
779
780
781
782
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 776

def delete_key_pair(region, pair_name)
  kp = region.key_pairs[pair_name]
  if kp.exists?
    @logger.debug("aws-sdk: delete key pair in region: #{region.name}")
    kp.delete()
  end
end

#delete_key_pair_all_regions(keypair_name_filter = nil) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Deletes key pairs from all regions

Parameters:

  • keypair_name_filter (String) (defaults to: nil) —

    if given, will get all keypairs that match a simple String#start_with? filter. If no filter is given, the basic key name returned by #key_name will be used.

Returns:

  • nil



733
734
735
736
737
738
739
740
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 733

def delete_key_pair_all_regions(keypair_name_filter=nil)
  region_keypairs_hash = my_key_pairs(keypair_name_filter)
  region_keypairs_hash.each_pair do |region, keypair_name_array|
    keypair_name_array.each do |keypair_name|
      delete_key_pair(region, keypair_name)
    end
  end
end

#enable_root(host) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Enables root access for a host when username is not root



577
578
579
580
581
582
583
584
585
586
587
588
589
590
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 577

def enable_root(host)
  if host['user'] != 'root'
    if host['platform'] =~ /f5-/
      enable_root_f5(host)
    elsif host['platform'] =~ /netscaler/
      enable_root_netscaler(host)
    else
      copy_ssh_to_root(host, @options)
      (host, @options)
      host['user'] = 'root'
    end
    host.close
  end
end

#enable_root_f5(host) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

This method does not support other platforms

Enables root access for a host on an f5 platform

Returns:

  • nil



597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 597

def enable_root_f5(host)
  for tries in 1..10
    begin
      #This command is problematic as the F5 is not always done loading
      if host.exec(Command.new("modify sys db systemauth.disablerootlogin value false"), :acceptable_exit_codes => [0,1]).exit_code == 0 \
          and host.exec(Command.new("modify sys global-settings gui-setup disabled"), :acceptable_exit_codes => [0,1]).exit_code == 0 \
          and host.exec(Command.new("save sys config"), :acceptable_exit_codes => [0,1]).exit_code == 0
        backoff_sleep(tries)
        break
      elsif tries == 10
        raise "Instance was unable to be configured"
      end
    rescue Beaker::Host::CommandFailure => e
      @logger.debug("Instance not yet configured (#{e})")
    end
    backoff_sleep(tries)
  end
  host['user'] = 'root'
  host.close
  sha256 = Digest::SHA256.new
  password = sha256.hexdigest((1..50).map{(rand(86)+40).chr}.join.gsub(/\\/,'\&\&'))
  host['ssh'] = {:password => password}
  host.exec(Command.new("echo -e '#{password}\\n#{password}' | tmsh modify auth password admin"))
  @logger.notify("f5: Configured admin password to be #{password}")
end

#enable_root_netscaler(host) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

This method does not support other platforms

Enables root access for a host on an netscaler platform

Returns:

  • nil



628
629
630
631
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 628

def enable_root_netscaler(host)
  host['ssh'] = {:password => host['instance'].id}
  @logger.notify("netscaler: nsroot password is #{host['instance'].id}")
end

#enable_root_on_hosts ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Enables root for instances with custom username like ubuntu-amis



567
568
569
570
571
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 567

def enable_root_on_hosts
  @hosts.each do |host|
    enable_root(host)
  end
end

#ensure_group(vpc, ports) ⇒ AWS::EC2::SecurityGroup

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return an existing group, or create new one

Accepts a VPC as input for checking & creation.

Parameters:

  • vpc (AWS::EC2::VPC) —

    the AWS vpc control object

  • ports (Array<Number>) —

    an array of port numbers

Returns:

  • (AWS::EC2::SecurityGroup) —

    created security group



841
842
843
844
845
846
847
848
849
850
851
852
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 841

def ensure_group(vpc, ports)
  @logger.notify("aws-sdk: Ensure security group exists for ports #{ports.to_s}, create if not")
  name = group_id(ports)

  group = vpc.security_groups.filter('group-name', name).first

  if group.nil?
    group = create_group(vpc, ports)
  end

  group
end

#ensure_key_pair(region) ⇒ AWS::EC2::KeyPair

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Creates the KeyPair for this test run

Parameters:

  • region (AWS::EC2::Region) —

    region to create the key pair in

Returns:

  • (AWS::EC2::KeyPair) —

    created key_pair



719
720
721
722
723
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 719

def ensure_key_pair(region)
  pair_name = key_name()
  delete_key_pair(region, pair_name)
  create_new_key_pair(region, pair_name)
end

#ensure_ping_group(vpc) ⇒ AWS::EC2::SecurityGroup

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return an existing group, or create new one

Accepts a VPC as input for checking & creation.

Parameters:

  • vpc (AWS::EC2::VPC) —

    the AWS vpc control object

Returns:

  • (AWS::EC2::SecurityGroup) —

    created security group



821
822
823
824
825
826
827
828
829
830
831
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 821

def ensure_ping_group(vpc)
  @logger.notify("aws-sdk: Ensure security group exists that enables ping, create if not")

  group = vpc.security_groups.filter('group-name', PING_SECURITY_GROUP_NAME).first

  if group.nil?
    group = create_ping_group(vpc)
  end

  group
end

#etc_hosts_entry(host, interface = :ip) ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return a valid /etc/hosts line for a given host

Parameters:

  • host (Beaker::Host) —

    Beaker::Host object for generating /etc/hosts entry

  • interface (Symbol) (defaults to: :ip) —

    Symbol identifies which ip should be used for host

Returns:

  • (String) —

    formatted hosts entry for host



538
539
540
541
542
543
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 538

def etc_hosts_entry(host, interface = :ip)
  name = host.name
  domain = get_domain_name(host)
  ip = host[interface.to_s]
  "#{ip}\t#{name} #{name}.#{domain} #{host['dns_name']}\n"
end

#group_id(ports) ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return a reproducable security group identifier based on input ports

Parameters:

  • ports (Array<Number>) —

    array of port numbers

Returns:

  • (String) —

    group identifier



801
802
803
804
805
806
807
808
809
810
811
812
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 801

def group_id(ports)
  if ports.nil? or ports.empty?
    raise ArgumentError, "Ports list cannot be nil or empty"
  end

  unless ports.is_a? Set
    ports = Set.new(ports)
  end

  # Lolwut, #hash is inconsistent between ruby processes
  "Beaker-#{Zlib.crc32(ports.inspect)}"
end

#instance_by_id(id) ⇒ AWS::EC2::Instance

Provided an id return an instance object. Instance object will respond to methods described here: AWS Instance Object.

Parameters:

  • id (String) —

    The id of the instance to return

Returns:

  • (AWS::EC2::Instance) —

    An AWS::EC2 instance object



127
128
129
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 127

def instance_by_id(id)
  @ec2.instances[id]
end

#instances ⇒ AWS::EC2::InstanceCollection

Return all instances currently on ec2.

Returns:

  • (AWS::EC2::InstanceCollection) —

    An array of AWS::EC2 instance objects

See Also:



134
135
136
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 134

def instances
  @ec2.instances
end

#key_name ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Generate a reusable key name from the local hosts hostname

Returns:

  • (String) —

    safe key name for current host



702
703
704
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 702

def key_name
  "#{key_name_prefix}-#{@options[:aws_keyname_modifier]}-#{@options[:timestamp].strftime("%F_%H_%M_%S_%N")}"
end

#key_name_prefix ⇒ String

Note:

This is the part of the key that will stay static between Beaker runs on the same host.

Generate a key prefix for key pair names

Returns:

  • (String) —

    Beaker key pair name based on sanitized hostname



693
694
695
696
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 693

def key_name_prefix
  safe_hostname = Socket.gethostname.gsub('.', '-')
  "Beaker-#{local_user}-#{safe_hostname}"
end

#kill_instances(instances) ⇒ void

This method returns an undefined value.

Kill all instances.

Parameters:

  • instances (Enumerable<EC2::Instance>) 


76
77
78
79
80
81
82
83
84
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 76

def kill_instances(instances)
  instances.each do |instance|
    if !instance.nil? and instance.exists?
      @logger.notify("aws-sdk: killing EC2 instance #{instance.id}")
      instance.terminate
    end
  end
  nil
end

#kill_zombie_volumes ⇒ Object

Destroy any volumes marked ‘available’, INCLUDING THOSE YOU DON’T OWN! Use with care.



203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 203

def kill_zombie_volumes
  # Occasionaly, tearing down ec2 instances leaves orphaned EBS volumes behind -- these stack up quickly.
  # This simply looks for EBS volumes that are not in use
  # Note: don't use volumes.each here as that funtion doesn't allow proper rescue from error states
  @logger.notify("aws-sdk: Kill Zombie Volumes!")
  volume_count = 0
  @ec2.regions.each do |region|
    @logger.debug "Reviewing: #{region.name}"
    volumes = @ec2.regions[region.name].volumes.map { |vol| vol.id }
    volumes.each do |vol|
      begin
        vol = @ec2.regions[region.name].volumes[vol]
        if ( vol.status.to_s =~ /available/ )
          @logger.debug "Tear down available volume: #{vol.id}"
          vol.delete()
          volume_count += 1
        end
      rescue AWS::EC2::Errors::InvalidVolume::NotFound => e
        @logger.debug "Failed to remove volume: #{vol.id}, #{e}"
      end
    end
  end
  @logger.notify "Freed #{volume_count} volume(s)"

end

#kill_zombies(max_age = ZOMBIE, key = key_name) ⇒ Object

Shutdown and destroy ec2 instances idenfitied by key that have been alive longer than ZOMBIE hours.

Parameters:

  • max_age (Integer) (defaults to: ZOMBIE) —

    The age in hours that a machine needs to be older than to be considered a zombie

  • key (String) (defaults to: key_name) —

    The key_name to match for



173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 173

def kill_zombies(max_age = ZOMBIE, key = key_name)
  @logger.notify("aws-sdk: Kill Zombies! (keyname: #{key}, age: #{max_age} hrs)")
  #examine all available regions
  kill_count = 0
  time_now = Time.now.getgm #ec2 uses GM time
  @ec2.regions.each do |region|
    @logger.debug "Reviewing: #{region.name}"
    # Note: don't use instances.each here as that funtion doesn't allow proper rescue from error states
    instances = @ec2.regions[region.name].instances
    instances.each do |instance|
      begin
        if (instance.key_name =~ /#{key}/)
          @logger.debug "Examining #{instance.id} (keyname: #{instance.key_name}, launch time: #{instance.launch_time}, status: #{instance.status})"
          if ((time_now - instance.launch_time) >  max_age*60*60) and instance.status.to_s !~ /terminated/
            @logger.debug "Kill! #{instance.id}: #{instance.key_name} (Current status: #{instance.status})"
            instance.terminate()
            kill_count += 1
          end
        end
      rescue AWS::Core::Resource::NotFound, AWS::EC2::Errors => e
        @logger.debug "Failed to remove instance: #{instance.id}, #{e}"
      end
    end
  end
  delete_key_pair_all_regions(key_name_prefix)

  @logger.notify "#{key}: Killed #{kill_count} instance(s)"
end

#launch_all_nodes ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Create EC2 instances for all hosts, tag them, and wait until they’re running. When a host provides a subnet_id, create the instance in that subnet, otherwise prefer a CONFIG subnet_id. If neither are set but there is a CONFIG subnet_ids list, attempt to create the host in each specified subnet, which might fail due to capacity constraints, for example. Specifying both a CONFIG subnet_id and subnet_ids will provoke an error.



372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 372

def launch_all_nodes
  @logger.notify("aws-sdk: launch all hosts in configuration")
  ami_spec = YAML.load_file(@options[:ec2_yaml])["AMI"]
  global_subnet_id = @options['subnet_id']
  global_subnets = @options['subnet_ids']
  if global_subnet_id and global_subnets
    raise RuntimeError, 'Config specifies both subnet_id and subnet_ids'
  end
  no_subnet_hosts = []
  specific_subnet_hosts = []
  some_subnet_hosts = []
  @hosts.each do |host|
    if global_subnet_id or host['subnet_id']
      specific_subnet_hosts.push(host)
    elsif global_subnets
      some_subnet_hosts.push(host)
    else
      no_subnet_hosts.push(host)
    end
  end
  instances = [] # Each element is {:instance => i, :host => h}
  begin
    @logger.notify("aws-sdk: launch instances not particular about subnet")
    launch_nodes_on_some_subnet(some_subnet_hosts, global_subnets, ami_spec,
                                instances)
    @logger.notify("aws-sdk: launch instances requiring a specific subnet")
    specific_subnet_hosts.each do |host|
      subnet_id = host['subnet_id'] || global_subnet_id
      instance = create_instance(host, ami_spec, subnet_id)
      instances.push({:instance => instance, :host => host})
    end
    @logger.notify("aws-sdk: launch instances requiring no subnet")
    no_subnet_hosts.each do |host|
      instance = create_instance(host, ami_spec, nil)
      instances.push({:instance => instance, :host => host})
    end
    wait_for_status(:running, instances)
  rescue Exception => ex
    @logger.notify("aws-sdk: exception - #{ex}")
    kill_instances(instances.map{|x| x[:instance]})
    raise ex
  end
  # At this point, all instances should be running since wait
  # either returns on success or throws an exception.
  if instances.empty?
    raise RuntimeError, "Didn't manage to launch any EC2 instances"
  end
  # Assign the now known running instances to their hosts.
  instances.each {|x| x[:host]['instance'] = x[:instance]}
  nil
end

#launch_nodes_on_some_subnet(hosts, subnets, ami_spec, instances_created) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

For each host, create an EC2 instance in one of the specified subnets and push it onto instances_created. Each subnet will be tried at most once for each host, and more than one subnet may be tried if capacity constraints are encountered. Each Hash in instances_created will contain an :instance and :host value.

Parameters:

  • hosts (Enumerable<Host>)
  • subnets (Enumerable<String>)
  • ami_spec (Hash)
  • instances_created —

    Enumerable<HashSymbol=>EC2::Instance,Host>



333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 333

def launch_nodes_on_some_subnet(hosts, subnets, ami_spec, instances_created)
  # Shuffle the subnets so we don't always hit the same one
  # first, and cycle though the subnets independently of the
  # host, so we stick with one that's working.  Try each subnet
  # once per-host.
  if subnets.nil? or subnets.empty?
    return
  end
  subnet_i = 0
  shuffnets = subnets.shuffle
  hosts.each do |host|
    instance = nil
    shuffnets.length.times do
      begin
        subnet_id = shuffnets[subnet_i]
        instance = create_instance(host, ami_spec, subnet_id)
        instances_created.push({:instance => instance, :host => host})
        break
      rescue AWS::EC2::Errors::InsufficientInstanceCapacity => ex
        @logger.notify("aws-sdk: hit #{subnet_id} capacity limit; moving on")
        subnet_i = (subnet_i + 1) % shuffnets.length
      end
    end
    if instance.nil?
      raise RuntimeError, "unable to launch host in any requested subnet"
    end
  end
end

#load_credentials ⇒ Hash<Symbol, String>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return a hash containing AWS credentials

Returns:

  • (Hash<Symbol, String>) —

    AWS credentials



900
901
902
903
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 900

def load_credentials
  return load_env_credentials unless load_env_credentials.empty?
  load_fog_credentials(@options[:dot_fog])
end

#load_env_credentials(prefix = 'AWS') ⇒ Hash<Symbol, String>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return AWS credentials loaded from environment variables

Parameters:

  • prefix (String) (defaults to: 'AWS') —

    environment variable prefix

Returns:

  • (Hash<Symbol, String>) —

    ec2 credentials



910
911
912
913
914
915
916
917
918
919
920
921
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 910

def load_env_credentials(prefix='AWS')
  provider = AWS::Core::CredentialProviders::ENVProvider.new prefix

  if provider.set?
    {
      :access_key => provider.access_key_id,
      :secret_key => provider.secret_access_key,
    }
  else
    {}
  end
end

#load_fog_credentials(dot_fog = '.fog') ⇒ Hash<Symbol, String>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Return a hash containing the fog credentials for EC2

Parameters:

  • dot_fog (String) (defaults to: '.fog') —

    dot fog path

Returns:

  • (Hash<Symbol, String>) —

    ec2 credentials



927
928
929
930
931
932
933
934
935
936
937
938
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 927

def load_fog_credentials(dot_fog = '.fog')
  fog = YAML.load_file( dot_fog )
  default = fog[:default]

  raise "You must specify an aws_access_key_id in your .fog file (#{dot_fog}) for ec2 instances!" unless default[:aws_access_key_id]
  raise "You must specify an aws_secret_access_key in your .fog file (#{dot_fog}) for ec2 instances!" unless default[:aws_secret_access_key]

  {
    :access_key => default[:aws_access_key_id],
    :secret_key => default[:aws_secret_access_key],
  }
end

#local_user ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the local user running this tool

Returns:

  • (String) —

    username of local user



710
711
712
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 710

def local_user
  ENV['USER']
end

#log_instances(key = key_name, status = /running/) ⇒ Object

Print instances to the logger. Instances will be from all regions associated with provided key name and limited by regex compared to instance status. Defaults to running instances.

Parameters:

  • key (String) (defaults to: key_name) —

    The key_name to match for

  • status (Regex) (defaults to: /running/) —

    The regular expression to match against the instance’s status



105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 105

def log_instances(key = key_name, status = /running/)
  instances = []
  @ec2.regions.each do |region|
    @logger.debug "Reviewing: #{region.name}"
    @ec2.regions[region.name].instances.each do |instance|
      if (instance.key_name =~ /#{key}/) and (instance.status.to_s =~ status)
        instances << instance
      end
    end
  end
  output = ""
  instances.each do |instance|
    output << "#{instance.id} keyname: #{instance.key_name}, dns name: #{instance.dns_name}, private ip: #{instance.private_ip_address}, ip: #{instance.ip_address}, launch time #{instance.launch_time}, status: #{instance.status}\n"
  end
  @logger.notify("aws-sdk: List instances (keyname: #{key})")
  @logger.notify("#{output}")
end

#my_key_pairs(name_filter = nil) ⇒ Hash{AWS::EC2::Region=>Array[String]}

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Gets the Beaker user’s keypairs by region

Parameters:

  • name_filter (String) (defaults to: nil) —

    if given, will get all keypairs that match a simple String#start_with? filter. If no filter is given, the basic key name returned by #key_name will be used.

Returns:

  • (Hash{AWS::EC2::Region=>Array[String]}) —

    a hash of region instance to an array of the keypair names that match for the filter



751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 751

def my_key_pairs(name_filter=nil)
  keypairs_by_region = {}
  keyname_default = key_name()
  keyname_filtered = "#{name_filter}-*"
  @ec2.regions.each do |region|
    if name_filter
      aws_name_filter = keyname_filtered
    else
      aws_name_filter = keyname_default
    end
    keypair_collection = region.key_pairs.filter('key-name', aws_name_filter)
    keypair_collection.each do |keypair|
      keypairs_by_region[region] ||= []
      keypairs_by_region[region] << keypair.name
    end
  end
  keypairs_by_region
end

#populate_dns ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Populate the hosts IP address from the EC2 dns_name



518
519
520
521
522
523
524
525
526
527
528
529
530
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 518

def populate_dns
  # Obtain the IP addresses and dns_name for each host
  @hosts.each do |host|
    @logger.notify("aws-sdk: Populate DNS for #{host.name}")
    instance = host['instance']
    host['ip'] = instance.ip_address ? instance.ip_address : instance.private_ip_address
    host['private_ip'] = instance.private_ip_address
    host['dns_name'] = instance.dns_name
    @logger.notify("aws-sdk: name: #{host.name} ip: #{host['ip']} private_ip: #{host['private_ip']} dns_name: #{instance.dns_name}")
  end

  nil
end

#provision ⇒ void

This method returns an undefined value.

Provision all hosts on EC2 using the AWS::EC2 API



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 44

def provision
  start_time = Time.now

  # Perform the main launch work
  launch_all_nodes()

  wait_for_status_netdev()

  # Add metadata tags to each instance
  add_tags()

  # Grab the ip addresses and dns from EC2 for each instance to use for ssh
  populate_dns()

  #enable root if user is not root
  enable_root_on_hosts()

  # Set the hostname for each box
  set_hostnames()

  # Configure /etc/hosts on each host
  configure_hosts()

  @logger.notify("aws-sdk: Provisioning complete in #{Time.now - start_time} seconds")

  nil #void
end

#public_key ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Retrieve the public key locally from the executing users ~/.ssh directory

Returns:

  • (String) —

    contents of public key



669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 669

def public_key
  keys = Array(@options[:ssh][:keys])
  keys << '~/.ssh/id_rsa'
  keys << '~/.ssh/id_dsa'
  key_file = nil
  keys.each do |key|
    key_filename = File.expand_path(key + '.pub')
    key_file = key_filename if File.exists?(key_filename)
  end

  if key_file
    @logger.debug("Using public key: #{key_file}")
  else
    raise RuntimeError, "Expected to find a public key, but couldn't in #{keys}"
  end
  File.read(key_file)
end

#security_group_by_id(id) ⇒ AWS::EC2::SecurityGroup

Provided an id return a security group object Security object will respond to methods described here: AWS SecurityGroup Object.

Parameters:

  • id (String) —

    The id of the security group to return

Returns:

  • (AWS::EC2::SecurityGroup) —

    An AWS::EC2 security group object



157
158
159
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 157

def security_group_by_id(id)
  @ec2.security_groups[id]
end

#security_groups ⇒ AWS::EC2::SecurityGroupCollection

Return all security groups currently on ec2.

Returns:

  • (AWS::EC2::SecurityGroupCollection) —

    An array of AWS::EC2 security group objects

See Also:

  • AwsSdk#security_goup_by_id


164
165
166
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 164

def security_groups
  @ec2.security_groups
end

#set_hostnames ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Set the hostname of all instances to be the hostname defined in the beaker configuration.



638
639
640
641
642
643
644
645
646
647
648
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 638

def set_hostnames
  @hosts.each do |host|
    if host['platform'] =~ /el-7/
      # on el-7 hosts, the hostname command doesn't "stick" randomly
      host.exec(Command.new("hostnamectl set-hostname #{host.name}"))
    else
      next if host['platform'] =~ /netscaler/
      host.exec(Command.new("hostname #{host.name}"))
    end
  end
end

#vpc_by_id(id) ⇒ AWS::EC2::VPC

Provided an id return a VPC object. VPC object will respond to methods described here: AWS VPC Object.

Parameters:

  • id (String) —

    The id of the VPC to return

Returns:

  • (AWS::EC2::VPC) —

    An AWS::EC2 vpc object



142
143
144
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 142

def vpc_by_id(id)
  @ec2.vpcs[id]
end

#vpcs ⇒ AWS::EC2::VPCCollection

Return all VPCs currently on ec2.

Returns:

  • (AWS::EC2::VPCCollection) —

    An array of AWS::EC2 vpc objects

See Also:



149
150
151
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 149

def vpcs
  @ec2.vpcs
end

#wait_for_status(status, instances, &block) ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

This method returns an undefined value.

Wait until all instances reach the desired state. Each Hash in instances must contain an :instance and :host value.

Parameters:

  • status (Symbol) —

    EC2 state to wait for, :running :stopped etc.

  • instances —

    Enumerable<HashSymbol=>EC2::Instance,Host>

  • block (Proc) —

    more complex checks can be made by passing a block in. This overrides the status parameter. EC2::Instance objects from the hosts will be yielded to the passed block



435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 435

def wait_for_status(status, instances, &block)
  # Wait for each node to reach status :running
  @logger.notify("aws-sdk: Waiting for all hosts to be #{status}")
  instances.each do |x|
    name = x[:name]
    instance = x[:instance]
    @logger.notify("aws-sdk: Wait for node #{name} to be #{status}")
    # Here we keep waiting for the machine state to reach ':running' with an
    # exponential backoff for each poll.
    # TODO: should probably be a in a shared method somewhere
    for tries in 1..10
      begin
        if block_given?
          test_result = yield instance
        else
          test_result = instance.status == status
        end
        if test_result
          # Always sleep, so the next command won't cause a throttle
          backoff_sleep(tries)
          break
        elsif tries == 10
          raise "Instance never reached state #{status}"
        end
      rescue AWS::EC2::Errors::InvalidInstanceID::NotFound => e
        @logger.debug("Instance #{name} not yet available (#{e})")
      end
      backoff_sleep(tries)
    end
  end
end

#wait_for_status_netdev ⇒ void

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Note:

if any host is an netdev one, these checks will happen once across all of the hosts, and then we’ll exit

This method returns an undefined value.

Handles special checks needed for netdev platforms.



474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
 
# File 'lib/beaker/hypervisor/aws_sdk.rb', line 474

def wait_for_status_netdev()
  @hosts.each do |host|
    if host['platform'] =~ /f5-|netscaler/
      wait_for_status(:running, @hosts)

      wait_for_status(nil, @hosts) do |instance|
        instance_status_collection = instance.client.describe_instance_status({:instance_ids => [instance.id]})
        first_instance = instance_status_collection[:instance_status_set].first
        first_instance[:system_status][:status] == "ok"
      end

      break
    end
  end
end