Class: ChefMetal::Provisioner::FogProvisioner

Inherits:
ChefMetal::Provisioner show all
Includes:
Chef::Mixin::ShellOut
Defined in:
lib/chef_metal/provisioner/fog_provisioner.rb

Overview

Provisions machines in vagrant.

Constant Summary collapse

DEFAULT_OPTIONS = 
{
  :create_timeout => 600,
  :start_timeout => 600,
  :ssh_timeout => 20
}

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(compute_options, id = nil) ⇒ FogProvisioner

Create a new fog provisioner.

## Parameters compute_options - hash of options to be passed to Fog::Compute.new Special options:

- :base_bootstrap_options is merged with bootstrap_options in acquire_machine
  to present the full set of bootstrap options.  Write down any bootstrap_options
  you intend to apply universally here.
- :aws_credentials is an AWS CSV file (created with Download Credentials)
  containing your aws key information.  If you do not specify aws_access_key_id
  and aws_secret_access_key explicitly, the first line from this file
  will be used.  You may pass a Cheffish::AWSCredentials object.
- :create_timeout - the time to wait for the instance to boot to ssh (defaults to 600)
- :start_timeout - the time to wait for the instance to start (defaults to 600)
- :ssh_timeout - the time to wait for ssh to be available if the instance is detected as up (defaults to 20)

id - the ID in the provisioner_url (fog:PROVIDER:ID)



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 44

def initialize(compute_options, id=nil)
  @compute_options = compute_options
  @base_bootstrap_options = compute_options.delete(:base_bootstrap_options) || {}

  case compute_options[:provider]
  when 'AWS'
    aws_credentials = compute_options.delete(:aws_credentials)
    if aws_credentials
      @aws_credentials = aws_credentials
    else
      @aws_credentials = ChefMetal::AWSCredentials.new
      @aws_credentials.load_default
    end
    compute_options[:aws_access_key_id] ||= @aws_credentials.default[:access_key_id]
    compute_options[:aws_secret_access_key] ||= @aws_credentials.default[:secret_access_key]
    # TODO actually find a key with the proper id
    # TODO let the user specify credentials and provider profiles that we can use
    if id && [0] != id
      raise "Default AWS credentials point at AWS account #{[0]}, but inflating from URL #{id}"
    end
  when 'OpenStack'
    openstack_credentials = compute_options.delete(:openstack_credentials)
    if openstack_credentials
      @openstack_credentials = openstack_credentials
    else
      @openstack_credentials = ChefMetal::OpenstackCredentials.new
      @openstack_credentials.load_default
    end

    compute_options[:openstack_username] ||= @openstack_credentials.default[:openstack_username]
    compute_options[:openstack_api_key] ||= @openstack_credentials.default[:openstack_api_key]
    compute_options[:openstack_auth_url] ||= @openstack_credentials.default[:openstack_auth_url]
    compute_options[:openstack_tenant] ||= @openstack_credentials.default[:openstack_tenant]
  end
  @key_pairs = {}
  @base_bootstrap_options_for = {}
end

Instance Attribute Details

#aws_credentials ⇒ Object (readonly)

Returns the value of attribute aws_credentials.



83
84
85
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 83

def aws_credentials
  @aws_credentials
end

#compute_options ⇒ Object (readonly)

Returns the value of attribute compute_options.



82
83
84
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 82

def compute_options
  @compute_options
end

#key_pairs ⇒ Object (readonly)

Returns the value of attribute key_pairs.



85
86
87
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 85

def key_pairs
  @key_pairs
end

#openstack_credentials ⇒ Object (readonly)

Returns the value of attribute openstack_credentials.



84
85
86
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 84

def openstack_credentials
  @openstack_credentials
end

Class Method Details

.inflate(node) ⇒ Object 



22
23
24
25
26
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 22

def self.inflate(node)
  url = node['normal']['provisioner_output']['provisioner_url']
  scheme, provider, id = url.split(':', 3)
  FogProvisioner.new({ :provider => provider }, id)
end

Instance Method Details

#acquire_machine(action_handler, node) ⇒ Object

Acquire a machine, generally by provisioning it. Returns a Machine object pointing at the machine, allowing useful actions like setup, converge, execute, file and directory. The Machine object will have a “node” property which must be saved to the server (if it is any different from the original node object).

## Parameters action_handler - the action_handler object that is calling this method; this

is generally a action_handler, but could be anything that can support the
ChefMetal::ActionHandler interface (i.e., in the case of the test
kitchen metal driver for acquiring and destroying VMs; see the base
class for what needs providing).

node - node object (deserialized json) representing this machine. If

the node has a provisioner_options hash in it, these will be used
instead of options provided by the provisioner.  TODO compare and
fail if different?
node will have node['normal']['provisioner_options'] in it with any options.
It is a hash with this format:

   -- provisioner_url: fog:<relevant_fog_options>
   -- bootstrap_options: hash of options to pass to compute.servers.create
   -- is_windows: true if windows.  TODO detect this from ami?
   -- create_timeout - the time to wait for the instance to boot to ssh (defaults to 600)
   -- start_timeout - the time to wait for the instance to start (defaults to 600)
   -- ssh_timeout - the time to wait for ssh to be available if the instance is detected as up (defaults to 20)

Example bootstrap_options for ec2:
   :image_id =>'ami-311f2b45',
   :flavor_id =>'t1.micro',
   :key_name => 'key-pair-name'

node['normal']['provisioner_output'] will be populated with information
about the created machine.  For vagrant, it is a hash with this
format:

   -- provisioner_url: fog:<relevant_fog_options>
   -- server_id: the ID of the server so it can be found again



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 148

def acquire_machine(action_handler, node)
  # Set up the modified node data
  provisioner_output = node['normal']['provisioner_output'] || {
    'provisioner_url' => provisioner_url,
    'provisioner_version' => ChefMetal::VERSION,
    'creator' => [1]
  }

  if provisioner_output['provisioner_url'] != provisioner_url
    if (provisioner_output['provisioner_version'].to_f <= 0.3) && provisioner_output['provisioner_url'].start_with?('fog:AWS:') && compute_options[:provider] == 'AWS'
      Chef::Log.warn "The upgrade from chef-metal 0.3 to 0.4 changed the provisioner URL format!  Metal will assume you are in fact using the same AWS account, and modify the provisioner URL to match."
      provisioner_output['provisioner_url'] = provisioner_url
      provisioner_output['provisioner_version'] ||= ChefMetal::VERSION
      provisioner_output['creator'] ||= [1]
    else
      raise "Switching providers for a machine is not currently supported!  Use machine :destroy and then re-create the machine on the new action_handler."
    end
  end

  node['normal']['provisioner_output'] = provisioner_output

  if provisioner_output['server_id']

    # If the server already exists, make sure it is up

    # TODO verify that the server info matches the specification (ami, etc.)\
    server = server_for(node)
    if !server
      Chef::Log.warn "Machine #{node['name']} (#{provisioner_output['server_id']} on #{provisioner_url}) is not associated with the ec2 account.  Recreating ..."
      need_to_create = true
    elsif %w(terminated archive).include?(server.state) # Can't come back from that
      Chef::Log.warn "Machine #{node['name']} (#{server.id} on #{provisioner_url}) is terminated.  Recreating ..."
      need_to_create = true
    else
      need_to_create = false
      if !server.ready?
        action_handler.perform_action "start machine #{node['name']} (#{server.id} on #{provisioner_url})" do
          server.start
        end
        action_handler.perform_action "wait for machine #{node['name']} (#{server.id} on #{provisioner_url}) to be ready" do
          wait_until_ready(server, option_for(node, :start_timeout))
        end
      else
        wait_until_ready(server, option_for(node, :ssh_timeout))
      end
    end
  else
    need_to_create = true
  end

  if need_to_create
    # If the server does not exist, create it
    bootstrap_options = bootstrap_options_for(action_handler.new_resource, node)
    bootstrap_options.merge(:name => action_handler.new_resource.name)

    start_time = Time.now
    timeout = option_for(node, :create_timeout)

    description = [ "create machine #{node['name']} on #{provisioner_url}" ]
    bootstrap_options.each_pair { |key,value| description << "    #{key}: #{value.inspect}" }
    server = nil
    action_handler.perform_action description do
      server = compute.servers.create(bootstrap_options)
      provisioner_output['server_id'] = server.id
      # Save quickly in case something goes wrong
      save_node(action_handler, node, action_handler.new_resource.chef_server)
    end

    if server
      @@ip_pool_lock = Mutex.new
      # Re-retrieve the server in a more malleable form and wait for it to be ready
      server = compute.servers.get(server.id)
      if bootstrap_options[:floating_ip_pool]
        Chef::Log.info 'Attaching IP from pool'
        server.wait_for { ready? }
        action_handler.perform_action "attach floating IP from #{bootstrap_options[:floating_ip_pool]} pool" do
          attach_ip_from_pool(server, bootstrap_options[:floating_ip_pool])
        end
      elsif bootstrap_options[:floating_ip]
        Chef::Log.info 'Attaching given IP'
        server.wait_for { ready? }
        action_handler.perform_action "attach floating IP #{bootstrap_options[:floating_ip]}" do
          attach_ip(server, bootstrap_options[:floating_ip])
        end
      end
      action_handler.perform_action "machine #{node['name']} created as #{server.id} on #{provisioner_url}" do
      end
      # Wait for the machine to come up and for ssh to start listening
      transport = nil
      _self = self
      action_handler.perform_action "wait for machine #{node['name']} to boot" do
        server.wait_for(timeout - (Time.now - start_time)) do
          if ready?
            transport ||= _self.transport_for(server)
            begin
              transport.execute('pwd')
              true
            rescue Errno::ECONNREFUSED, Net::SSH::Disconnect
              false
            rescue
              true
            end
          else
            false
          end
        end
      end

      # If there is some other error, we just wait patiently for SSH
      begin
        server.wait_for(option_for(node, :ssh_timeout)) { transport.available? }
      rescue Fog::Errors::TimeoutError
        # Sometimes (on EC2) the machine comes up but gets stuck or has
        # some other problem.  If this is the case, we restart the server
        # to unstick it.  Reboot covers a multitude of sins.
        Chef::Log.warn "Machine #{node['name']} (#{server.id} on #{provisioner_url}) was started but SSH did not come up.  Rebooting machine in an attempt to unstick it ..."
        action_handler.perform_action "reboot machine #{node['name']} to try to unstick it" do
          server.reboot
        end
        action_handler.perform_action "wait for machine #{node['name']} to be ready after reboot" do
          wait_until_ready(server, option_for(node, :start_timeout))
        end
      end
    end
  end

  # Create machine object for callers to use
  machine_for(node, server)
end

#attach_ip(server, ip) ⇒ Object

Attach given IP to machine Code taken from kitchen-openstack driver

https://github.com/test-kitchen/kitchen-openstack/blob/master/lib/kitchen/driver/openstack.rb#L209-L213



297
298
299
300
301
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 297

def attach_ip(server, ip)
  Chef::Log.info "Attaching floating IP <#{ip}>"
  server.associate_address ip
  (server.addresses['public'] ||= []) << { 'version' => 4, 'addr' => ip }
end

#attach_ip_from_pool(server, pool) ⇒ Object

Attach IP to machine from IP pool Code taken from kitchen-openstack driver

https://github.com/test-kitchen/kitchen-openstack/blob/master/lib/kitchen/driver/openstack.rb#L196-L207



281
282
283
284
285
286
287
288
289
290
291
292
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 281

def attach_ip_from_pool(server, pool)
  @@ip_pool_lock.synchronize do
    Chef::Log.info "Attaching floating IP from <#{pool}> pool"
    free_addrs = compute.addresses.collect do |i|
      i.ip if i.fixed_ip.nil? and i.instance_id.nil? and i.pool == pool
    end.compact
    if free_addrs.empty?
      raise ActionFailed, "No available IPs in pool <#{pool}>"
    end
    attach_ip(server, free_addrs[0])
  end
end

#compute ⇒ Object 



333
334
335
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 333

def compute
  @compute ||= Fog::Compute.new(compute_options)
end

#connect_to_machine(node) ⇒ Object

Connect to machine without acquiring it



304
305
306
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 304

def connect_to_machine(node)
  machine_for(node)
end

#current_base_bootstrap_options ⇒ Object 



87
88
89
90
91
92
93
94
95
96
97
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 87

def current_base_bootstrap_options
  result = @base_bootstrap_options.dup
  if key_pairs.size > 0
    last_pair_name = key_pairs.keys.last
    last_pair = key_pairs[last_pair_name]
    result[:key_name] ||= last_pair_name
    result[:private_key_path] ||= last_pair.private_key_path
    result[:public_key_path] ||= last_pair.public_key_path
  end
  result
end

#delete_machine(action_handler, node) ⇒ Object 



308
309
310
311
312
313
314
315
316
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 308

def delete_machine(action_handler, node)
  if node['normal']['provisioner_output'] && node['normal']['provisioner_output']['server_id']
    server = compute.servers.get(node['normal']['provisioner_output']['server_id'])
    action_handler.perform_action "destroy machine #{node['name']} (#{node['normal']['provisioner_output']['server_id']} at #{provisioner_url})" do
      server.destroy
    end
    convergence_strategy_for(node).cleanup_convergence(action_handler, node)
  end
end

#provisioner_url ⇒ Object 



337
338
339
340
341
342
343
344
345
346
347
348
349
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 337

def provisioner_url
  provider_identifier = case compute_options[:provider]
    when 'AWS'
      [0]
    when 'DigitalOcean'
      compute_options[:digitalocean_client_id]
    when 'OpenStack'
      compute_options[:openstack_auth_url]
    else
      '???'
  end
  "fog:#{compute_options[:provider]}:#{provider_identifier}"
end

#resource_created(machine) ⇒ Object 



328
329
330
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 328

def resource_created(machine)
  @base_bootstrap_options_for[machine] = current_base_bootstrap_options
end

#stop_machine(action_handler, node) ⇒ Object 



318
319
320
321
322
323
324
325
326
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 318

def stop_machine(action_handler, node)
  # If the machine doesn't exist, we silently do nothing
  if node['normal']['provisioner_output'] && node['normal']['provisioner_output']['server_id']
    server = compute.servers.get(node['normal']['provisioner_output']['server_id'])
    action_handler.perform_action "stop machine #{node['name']} (#{server.id} at #{provisioner_url})" do
      server.stop
    end
  end
end

#transport_for(server) ⇒ Object

Not meant to be part of public interface



352
353
354
355
 
# File 'lib/chef_metal/provisioner/fog_provisioner.rb', line 352

def transport_for(server)
  # TODO winrm
  create_ssh_transport(server)
end